[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:23:15.083246  3431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.3.89.254:38703
I20260709 12:23:15.084765  3431 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:23:15.085808  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:15.103475  3437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:15.103672  3438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:15.103832  3440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:15.106244  3431 server_base.cc:1061] running on GCE node
I20260709 12:23:16.334347  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:16.334746  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:16.335026  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783599796335002 us; error 0 us; skew 500 ppm
I20260709 12:23:16.336329  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:16.343293  3431 webserver.cc:533] Webserver started at http://127.3.89.254:35309/ using document root <none> and password file <none>
I20260709 12:23:16.344170  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:16.344370  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:16.344798  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:16.350957  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/master-0-root/instance:
uuid: "a72528fa9f62441eabfabeeec4a468db"
format_stamp: "Formatted at 2026-07-09 12:23:16 on dist-test-slave-3hrd"
I20260709 12:23:16.359530  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.000s	sys 0.008s
I20260709 12:23:16.365443  3446 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:16.366946  3431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:23:16.367295  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/master-0-root
uuid: "a72528fa9f62441eabfabeeec4a468db"
format_stamp: "Formatted at 2026-07-09 12:23:16 on dist-test-slave-3hrd"
I20260709 12:23:16.367604  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:16.426544  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:16.428088  3431 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:23:16.428537  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:16.506039  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:38703
I20260709 12:23:16.506134  3497 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.254:38703 every 8 connection(s)
I20260709 12:23:16.514741  3498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.537443  3498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db: Bootstrap starting.
I20260709 12:23:16.543877  3498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:16.545914  3498 log.cc:826] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db: Log is configured to *not* fsync() on all Append() calls
I20260709 12:23:16.551003  3498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db: No bootstrap required, opened a new log
I20260709 12:23:16.573721  3498 raft_consensus.cc:359] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72528fa9f62441eabfabeeec4a468db" member_type: VOTER }
I20260709 12:23:16.574410  3498 raft_consensus.cc:385] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:16.574693  3498 raft_consensus.cc:740] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a72528fa9f62441eabfabeeec4a468db, State: Initialized, Role: FOLLOWER
I20260709 12:23:16.575606  3498 consensus_queue.cc:260] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72528fa9f62441eabfabeeec4a468db" member_type: VOTER }
I20260709 12:23:16.576169  3498 raft_consensus.cc:399] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:23:16.576488  3498 raft_consensus.cc:493] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:23:16.576875  3498 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:16.581159  3498 raft_consensus.cc:515] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72528fa9f62441eabfabeeec4a468db" member_type: VOTER }
I20260709 12:23:16.581981  3498 leader_election.cc:304] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a72528fa9f62441eabfabeeec4a468db; no voters: 
I20260709 12:23:16.583751  3498 leader_election.cc:290] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:23:16.584172  3501 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:16.586490  3501 raft_consensus.cc:697] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [term 1 LEADER]: Becoming Leader. State: Replica: a72528fa9f62441eabfabeeec4a468db, State: Running, Role: LEADER
I20260709 12:23:16.587496  3501 consensus_queue.cc:237] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72528fa9f62441eabfabeeec4a468db" member_type: VOTER }
I20260709 12:23:16.588567  3498 sys_catalog.cc:565] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:23:16.598695  3502 sys_catalog.cc:455] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a72528fa9f62441eabfabeeec4a468db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72528fa9f62441eabfabeeec4a468db" member_type: VOTER } }
I20260709 12:23:16.600095  3502 sys_catalog.cc:458] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:16.598923  3503 sys_catalog.cc:455] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [sys.catalog]: SysCatalogTable state changed. Reason: New leader a72528fa9f62441eabfabeeec4a468db. Latest consensus state: current_term: 1 leader_uuid: "a72528fa9f62441eabfabeeec4a468db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a72528fa9f62441eabfabeeec4a468db" member_type: VOTER } }
I20260709 12:23:16.600684  3503 sys_catalog.cc:458] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:16.603926  3509 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:23:16.615888  3509 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:23:16.620819  3431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:23:16.635977  3509 catalog_manager.cc:1383] Generated new cluster ID: 4aaf380c240a4ed8b278bb1cedac16ec
I20260709 12:23:16.636338  3509 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:23:16.666908  3509 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:23:16.668591  3509 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:23:16.682608  3509 catalog_manager.cc:6070] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db: Generated new TSK 0
I20260709 12:23:16.683821  3509 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:23:16.752785  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:16.763207  3520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:16.764662  3521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:16.769604  3523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:16.770131  3431 server_base.cc:1061] running on GCE node
I20260709 12:23:16.771039  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:16.771229  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:16.771458  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783599796771423 us; error 0 us; skew 500 ppm
I20260709 12:23:16.772109  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:16.776772  3431 webserver.cc:533] Webserver started at http://127.3.89.193:46595/ using document root <none> and password file <none>
I20260709 12:23:16.777364  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:16.777611  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:16.777954  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:16.779157  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-0-root/instance:
uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
format_stamp: "Formatted at 2026-07-09 12:23:16 on dist-test-slave-3hrd"
I20260709 12:23:16.785184  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 12:23:16.790230  3528 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:16.791548  3431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:23:16.791968  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-0-root
uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
format_stamp: "Formatted at 2026-07-09 12:23:16 on dist-test-slave-3hrd"
I20260709 12:23:16.792369  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:16.820103  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:16.822165  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:16.825070  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:16.830758  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:16.831097  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:16.831419  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:16.831647  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:16.934386  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:41693
I20260709 12:23:16.934465  3591 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.193:41693 every 8 connection(s)
I20260709 12:23:16.942754  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:16.952863  3596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:16.955394  3597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:16.960494  3431 server_base.cc:1061] running on GCE node
W20260709 12:23:16.962277  3599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:16.963251  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:16.963505  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:16.963701  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783599796963683 us; error 0 us; skew 500 ppm
I20260709 12:23:16.964298  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:16.967763  3431 webserver.cc:533] Webserver started at http://127.3.89.194:41031/ using document root <none> and password file <none>
I20260709 12:23:16.968336  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:16.968601  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:16.968966  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:16.970518  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-1-root/instance:
uuid: "242920ef3fb14cbab2b9b913a5891b0b"
format_stamp: "Formatted at 2026-07-09 12:23:16 on dist-test-slave-3hrd"
I20260709 12:23:16.972554  3592 heartbeater.cc:344] Connected to a master server at 127.3.89.254:38703
I20260709 12:23:16.973037  3592 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:16.974285  3592 heartbeater.cc:507] Master 127.3.89.254:38703 requested a full tablet report, sending...
I20260709 12:23:16.977689  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.003s
I20260709 12:23:16.978143  3463 ts_manager.cc:194] Registered new tserver with Master: ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693)
I20260709 12:23:16.981211  3463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40914
I20260709 12:23:16.982538  3604 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:16.983906  3431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:23:16.984198  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-1-root
uuid: "242920ef3fb14cbab2b9b913a5891b0b"
format_stamp: "Formatted at 2026-07-09 12:23:16 on dist-test-slave-3hrd"
I20260709 12:23:16.984496  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:17.017817  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:17.019388  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:17.022574  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:17.025549  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:17.025815  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:17.026115  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:17.026274  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:17.128741  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:40141
I20260709 12:23:17.128852  3666 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.194:40141 every 8 connection(s)
I20260709 12:23:17.135614  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:17.145474  3670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:17.148808  3671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:17.151270  3667 heartbeater.cc:344] Connected to a master server at 127.3.89.254:38703
I20260709 12:23:17.151652  3667 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:17.153014  3667 heartbeater.cc:507] Master 127.3.89.254:38703 requested a full tablet report, sending...
I20260709 12:23:17.155599  3463 ts_manager.cc:194] Registered new tserver with Master: 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:17.157665  3463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40918
W20260709 12:23:17.161746  3673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:17.163735  3431 server_base.cc:1061] running on GCE node
I20260709 12:23:17.165096  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:17.165418  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:17.165848  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783599797165823 us; error 0 us; skew 500 ppm
I20260709 12:23:17.166718  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:17.172438  3431 webserver.cc:533] Webserver started at http://127.3.89.195:35571/ using document root <none> and password file <none>
I20260709 12:23:17.173312  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:17.173585  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:17.173959  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:17.175280  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-2-root/instance:
uuid: "b13e9d12b0b0416a9e214826c4a1c733"
format_stamp: "Formatted at 2026-07-09 12:23:17 on dist-test-slave-3hrd"
I20260709 12:23:17.182188  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:23:17.187750  3678 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:17.188827  3431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:23:17.189437  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-2-root
uuid: "b13e9d12b0b0416a9e214826c4a1c733"
format_stamp: "Formatted at 2026-07-09 12:23:17 on dist-test-slave-3hrd"
I20260709 12:23:17.189872  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599794910850-3431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:17.227077  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:17.228611  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:17.230726  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:17.234269  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:17.234556  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:17.234855  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:17.235059  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:17.327858  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:45347
I20260709 12:23:17.327924  3741 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.195:45347 every 8 connection(s)
I20260709 12:23:17.348572  3742 heartbeater.cc:344] Connected to a master server at 127.3.89.254:38703
I20260709 12:23:17.349030  3742 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:17.350113  3742 heartbeater.cc:507] Master 127.3.89.254:38703 requested a full tablet report, sending...
I20260709 12:23:17.352730  3463 ts_manager.cc:194] Registered new tserver with Master: b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:17.353327  3431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018586626s
I20260709 12:23:17.354823  3463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40924
I20260709 12:23:17.413453  3463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40926:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:23:17.417848  3463 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:23:17.506028  3632 tablet_service.cc:1511] Processing CreateTablet for tablet eafd3f6dfac344f08870c9fdc9d95838 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:17.506847  3631 tablet_service.cc:1511] Processing CreateTablet for tablet 4124039aef754341ad4ac25df7cc1c05 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:17.507752  3632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eafd3f6dfac344f08870c9fdc9d95838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.510058  3631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4124039aef754341ad4ac25df7cc1c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.511782  3630 tablet_service.cc:1511] Processing CreateTablet for tablet 34c2aa61069e4b4c9585861f9061a78d (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:17.511719  3629 tablet_service.cc:1511] Processing CreateTablet for tablet fa991ca904d84c5980cbb5e5859c0c51 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:17.513176  3630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c2aa61069e4b4c9585861f9061a78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.517958  3629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa991ca904d84c5980cbb5e5859c0c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.518519  3628 tablet_service.cc:1511] Processing CreateTablet for tablet 19f46c4700004a08bf63901dae2c7152 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:17.519912  3628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19f46c4700004a08bf63901dae2c7152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.538116  3627 tablet_service.cc:1511] Processing CreateTablet for tablet 99a3567afee845fd82bd52553b4d236f (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:17.540866  3627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99a3567afee845fd82bd52553b4d236f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.556655  3626 tablet_service.cc:1511] Processing CreateTablet for tablet 914a0780eb754f62a3c0221b0c47c2f7 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:17.559432  3626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 914a0780eb754f62a3c0221b0c47c2f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.564895  3625 tablet_service.cc:1511] Processing CreateTablet for tablet 4b8b062b65ee465aa4b2b603c1a73350 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:17.570160  3625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b8b062b65ee465aa4b2b603c1a73350. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:17.612699  3511 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:17.618338  3629 tablet_service.cc:1511] Processing CreateTablet for tablet e732f8f7ac884e1d987d039cfa748f14 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:17.623638  3629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e732f8f7ac884e1d987d039cfa748f14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.640475  3555 tablet_service.cc:1511] Processing CreateTablet for tablet 34c2aa61069e4b4c9585861f9061a78d (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:17.643008  3556 tablet_service.cc:1511] Processing CreateTablet for tablet 4124039aef754341ad4ac25df7cc1c05 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:17.652700  3554 tablet_service.cc:1511] Processing CreateTablet for tablet fa991ca904d84c5980cbb5e5859c0c51 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:17.649220  3706 tablet_service.cc:1511] Processing CreateTablet for tablet eafd3f6dfac344f08870c9fdc9d95838 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:17.655580  3555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c2aa61069e4b4c9585861f9061a78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.659624  3552 tablet_service.cc:1511] Processing CreateTablet for tablet 99a3567afee845fd82bd52553b4d236f (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:17.669132  3556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4124039aef754341ad4ac25df7cc1c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.669154  3550 tablet_service.cc:1511] Processing CreateTablet for tablet 4b8b062b65ee465aa4b2b603c1a73350 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:17.671792  3553 tablet_service.cc:1511] Processing CreateTablet for tablet 19f46c4700004a08bf63901dae2c7152 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:17.677171  3760 tablet_bootstrap.cc:492] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b: Bootstrap starting.
I20260709 12:23:17.667528  3551 tablet_service.cc:1511] Processing CreateTablet for tablet 914a0780eb754f62a3c0221b0c47c2f7 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:17.679160  3706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eafd3f6dfac344f08870c9fdc9d95838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.674181  3704 tablet_service.cc:1511] Processing CreateTablet for tablet 34c2aa61069e4b4c9585861f9061a78d (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:17.681007  3705 tablet_service.cc:1511] Processing CreateTablet for tablet 4124039aef754341ad4ac25df7cc1c05 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:17.670177  3557 tablet_service.cc:1511] Processing CreateTablet for tablet eafd3f6dfac344f08870c9fdc9d95838 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:17.683521  3552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99a3567afee845fd82bd52553b4d236f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.687713  3549 tablet_service.cc:1511] Processing CreateTablet for tablet e732f8f7ac884e1d987d039cfa748f14 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:17.684723  3702 tablet_service.cc:1511] Processing CreateTablet for tablet 19f46c4700004a08bf63901dae2c7152 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:17.699954  3704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c2aa61069e4b4c9585861f9061a78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.693846  3703 tablet_service.cc:1511] Processing CreateTablet for tablet fa991ca904d84c5980cbb5e5859c0c51 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:17.701309  3702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19f46c4700004a08bf63901dae2c7152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.693876  3700 tablet_service.cc:1511] Processing CreateTablet for tablet 914a0780eb754f62a3c0221b0c47c2f7 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:17.701223  3701 tablet_service.cc:1511] Processing CreateTablet for tablet 99a3567afee845fd82bd52553b4d236f (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:17.710289  3553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19f46c4700004a08bf63901dae2c7152. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.710889  3705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4124039aef754341ad4ac25df7cc1c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.704049  3699 tablet_service.cc:1511] Processing CreateTablet for tablet 4b8b062b65ee465aa4b2b603c1a73350 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:17.712792  3698 tablet_service.cc:1511] Processing CreateTablet for tablet e732f8f7ac884e1d987d039cfa748f14 (DEFAULT_TABLE table=test-workload [id=08257d9d98734246883a52f02479de2b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:17.715495  3703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa991ca904d84c5980cbb5e5859c0c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.714205  3557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eafd3f6dfac344f08870c9fdc9d95838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.717231  3701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99a3567afee845fd82bd52553b4d236f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.720997  3698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e732f8f7ac884e1d987d039cfa748f14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.735735  3699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b8b062b65ee465aa4b2b603c1a73350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.741730  3549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e732f8f7ac884e1d987d039cfa748f14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.742286  3700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 914a0780eb754f62a3c0221b0c47c2f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.745540  3551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 914a0780eb754f62a3c0221b0c47c2f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.748401  3554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa991ca904d84c5980cbb5e5859c0c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.764364  3550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b8b062b65ee465aa4b2b603c1a73350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:17.837430  3760 tablet_bootstrap.cc:654] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.861868  3762 tablet_bootstrap.cc:492] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a: Bootstrap starting.
I20260709 12:23:17.883610  3761 tablet_bootstrap.cc:492] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733: Bootstrap starting.
I20260709 12:23:17.886345  3760 tablet_bootstrap.cc:492] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b: No bootstrap required, opened a new log
I20260709 12:23:17.886873  3760 ts_tablet_manager.cc:1403] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent bootstrapping tablet: real 0.218s	user 0.007s	sys 0.013s
I20260709 12:23:17.891026  3760 raft_consensus.cc:359] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:17.892432  3760 raft_consensus.cc:385] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.892791  3760 raft_consensus.cc:740] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.893972  3760 consensus_queue.cc:260] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:17.917743  3667 heartbeater.cc:499] Master 127.3.89.254:38703 was elected leader, sending a full tablet report...
I20260709 12:23:17.917790  3766 raft_consensus.cc:493] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.923003  3766 raft_consensus.cc:515] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:17.923717  3760 ts_tablet_manager.cc:1434] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent starting tablet: real 0.037s	user 0.014s	sys 0.016s
I20260709 12:23:17.925014  3760 tablet_bootstrap.cc:492] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b: Bootstrap starting.
I20260709 12:23:17.932991  3762 tablet_bootstrap.cc:654] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.933308  3761 tablet_bootstrap.cc:654] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.943360  3760 tablet_bootstrap.cc:654] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:17.946511  3762 tablet_bootstrap.cc:492] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a: No bootstrap required, opened a new log
I20260709 12:23:17.947216  3762 ts_tablet_manager.cc:1403] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent bootstrapping tablet: real 0.108s	user 0.012s	sys 0.005s
I20260709 12:23:17.952282  3762 raft_consensus.cc:359] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:17.953078  3762 raft_consensus.cc:385] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.953451  3762 raft_consensus.cc:740] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.954250  3762 consensus_queue.cc:260] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:17.959774  3592 heartbeater.cc:499] Master 127.3.89.254:38703 was elected leader, sending a full tablet report...
I20260709 12:23:17.961741  3762 ts_tablet_manager.cc:1434] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:23:17.961985  3761 tablet_bootstrap.cc:492] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733: No bootstrap required, opened a new log
I20260709 12:23:17.962467  3761 ts_tablet_manager.cc:1403] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733: Time spent bootstrapping tablet: real 0.088s	user 0.015s	sys 0.006s
I20260709 12:23:17.963153  3762 tablet_bootstrap.cc:492] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a: Bootstrap starting.
I20260709 12:23:17.969856  3761 raft_consensus.cc:359] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:17.970937  3761 raft_consensus.cc:385] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:17.971318  3761 raft_consensus.cc:740] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Initialized, Role: FOLLOWER
I20260709 12:23:17.972275  3761 consensus_queue.cc:260] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:17.973327  3766 leader_election.cc:290] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:17.981125  3742 heartbeater.cc:499] Master 127.3.89.254:38703 was elected leader, sending a full tablet report...
I20260709 12:23:17.982151  3761 ts_tablet_manager.cc:1434] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.011s
I20260709 12:23:17.983273  3761 tablet_bootstrap.cc:492] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733: Bootstrap starting.
I20260709 12:23:17.988029  3762 tablet_bootstrap.cc:654] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.014282  3716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733" is_pre_election: true
I20260709 12:23:18.012962  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" is_pre_election: true
I20260709 12:23:18.015640  3760 tablet_bootstrap.cc:492] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b: No bootstrap required, opened a new log
I20260709 12:23:18.016772  3716 raft_consensus.cc:2468] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 0.
W20260709 12:23:18.017444  3607 leader_election.cc:343] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.019814  3608 leader_election.cc:304] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733; no voters: ae0b4bb075da417d9154ab6a1fcbd36a
I20260709 12:23:18.016445  3760 ts_tablet_manager.cc:1403] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent bootstrapping tablet: real 0.092s	user 0.011s	sys 0.051s
I20260709 12:23:18.021040  3766 raft_consensus.cc:2804] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.021447  3766 raft_consensus.cc:493] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.021893  3766 raft_consensus.cc:3060] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.022675  3761 tablet_bootstrap.cc:654] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.020339  3762 tablet_bootstrap.cc:492] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a: No bootstrap required, opened a new log
I20260709 12:23:18.027384  3762 ts_tablet_manager.cc:1403] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent bootstrapping tablet: real 0.064s	user 0.021s	sys 0.003s
I20260709 12:23:18.026432  3760 raft_consensus.cc:359] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.028802  3760 raft_consensus.cc:385] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.029125  3760 raft_consensus.cc:740] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.029901  3760 consensus_queue.cc:260] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.032037  3760 ts_tablet_manager.cc:1434] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:23:18.032970  3760 tablet_bootstrap.cc:492] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b: Bootstrap starting.
I20260709 12:23:18.033707  3766 raft_consensus.cc:515] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.040212  3761 tablet_bootstrap.cc:492] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733: No bootstrap required, opened a new log
I20260709 12:23:18.051216  3761 ts_tablet_manager.cc:1403] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.012s
I20260709 12:23:18.051142  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:18.052510  3716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:18.053110  3716 raft_consensus.cc:3060] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.043890  3760 tablet_bootstrap.cc:654] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:18.053056  3607 leader_election.cc:343] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.056901  3761 raft_consensus.cc:359] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.057649  3761 raft_consensus.cc:385] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.057940  3761 raft_consensus.cc:740] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.058727  3761 consensus_queue.cc:260] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.041529  3762 raft_consensus.cc:359] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.060305  3762 raft_consensus.cc:385] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.060643  3762 raft_consensus.cc:740] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.061064  3761 ts_tablet_manager.cc:1434] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:23:18.061079  3716 raft_consensus.cc:2468] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 1.
I20260709 12:23:18.062054  3761 tablet_bootstrap.cc:492] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733: Bootstrap starting.
I20260709 12:23:18.067579  3608 leader_election.cc:304] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733; no voters: ae0b4bb075da417d9154ab6a1fcbd36a
I20260709 12:23:18.068665  3764 raft_consensus.cc:2804] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.070572  3762 consensus_queue.cc:260] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.072952  3761 tablet_bootstrap.cc:654] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.074690  3766 leader_election.cc:290] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.079502  3764 raft_consensus.cc:697] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Becoming Leader. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:18.080473  3764 consensus_queue.cc:237] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.081061  3762 ts_tablet_manager.cc:1434] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent starting tablet: real 0.042s	user 0.001s	sys 0.005s
I20260709 12:23:18.085019  3762 tablet_bootstrap.cc:492] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a: Bootstrap starting.
I20260709 12:23:18.088480  3761 tablet_bootstrap.cc:492] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733: No bootstrap required, opened a new log
W20260709 12:23:18.088644  3743 tablet.cc:2406] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:23:18.088992  3761 ts_tablet_manager.cc:1403] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.005s
I20260709 12:23:18.096180  3761 raft_consensus.cc:359] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.096993  3761 raft_consensus.cc:385] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.097271  3761 raft_consensus.cc:740] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.098044  3761 consensus_queue.cc:260] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.098949  3762 tablet_bootstrap.cc:654] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.100731  3462 catalog_manager.cc:5697] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194). New cstate: current_term: 1 leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.103199  3760 tablet_bootstrap.cc:492] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b: No bootstrap required, opened a new log
I20260709 12:23:18.103677  3760 ts_tablet_manager.cc:1403] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent bootstrapping tablet: real 0.071s	user 0.013s	sys 0.014s
I20260709 12:23:18.106699  3760 raft_consensus.cc:359] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.107401  3760 raft_consensus.cc:385] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.107707  3760 raft_consensus.cc:740] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.108430  3760 consensus_queue.cc:260] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.110867  3760 ts_tablet_manager.cc:1434] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:23:18.111763  3760 tablet_bootstrap.cc:492] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b: Bootstrap starting.
I20260709 12:23:18.100486  3761 ts_tablet_manager.cc:1434] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:23:18.114408  3761 tablet_bootstrap.cc:492] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733: Bootstrap starting.
I20260709 12:23:18.122560  3760 tablet_bootstrap.cc:654] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.128361  3761 tablet_bootstrap.cc:654] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.128837  3762 tablet_bootstrap.cc:492] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a: No bootstrap required, opened a new log
I20260709 12:23:18.129505  3762 ts_tablet_manager.cc:1403] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.000s
I20260709 12:23:18.131749  3773 raft_consensus.cc:493] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.132290  3773 raft_consensus.cc:515] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.133499  3762 raft_consensus.cc:359] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.135007  3773 leader_election.cc:290] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:18.141101  3760 tablet_bootstrap.cc:492] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b: No bootstrap required, opened a new log
I20260709 12:23:18.149122  3760 ts_tablet_manager.cc:1403] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.002s
I20260709 12:23:18.134577  3762 raft_consensus.cc:385] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.152540  3762 raft_consensus.cc:740] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.152120  3760 raft_consensus.cc:359] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.154261  3760 raft_consensus.cc:385] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.154593  3760 raft_consensus.cc:740] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.155339  3760 consensus_queue.cc:260] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.149875  3761 tablet_bootstrap.cc:492] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733: No bootstrap required, opened a new log
I20260709 12:23:18.158105  3761 ts_tablet_manager.cc:1403] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.007s
I20260709 12:23:18.161073  3761 raft_consensus.cc:359] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.161844  3761 raft_consensus.cc:385] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.161935  3642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a3567afee845fd82bd52553b4d236f" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b" is_pre_election: true
I20260709 12:23:18.162595  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a3567afee845fd82bd52553b4d236f" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" is_pre_election: true
I20260709 12:23:18.162992  3760 ts_tablet_manager.cc:1434] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:23:18.163206  3567 raft_consensus.cc:2468] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 0.
I20260709 12:23:18.164001  3760 tablet_bootstrap.cc:492] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b: Bootstrap starting.
W20260709 12:23:18.164074  3680 leader_election.cc:343] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.164952  3681 leader_election.cc:304] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 242920ef3fb14cbab2b9b913a5891b0b
I20260709 12:23:18.165719  3773 raft_consensus.cc:2804] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.166126  3773 raft_consensus.cc:493] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.153307  3762 consensus_queue.cc:260] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.166488  3773 raft_consensus.cc:3060] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.162236  3761 raft_consensus.cc:740] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.168777  3761 consensus_queue.cc:260] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.171542  3762 ts_tablet_manager.cc:1434] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 12:23:18.172643  3762 tablet_bootstrap.cc:492] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a: Bootstrap starting.
I20260709 12:23:18.174553  3773 raft_consensus.cc:515] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.181034  3760 tablet_bootstrap.cc:654] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.187223  3773 leader_election.cc:290] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:18.197408  3761 ts_tablet_manager.cc:1434] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733: Time spent starting tablet: real 0.039s	user 0.014s	sys 0.010s
I20260709 12:23:18.198494  3761 tablet_bootstrap.cc:492] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733: Bootstrap starting.
I20260709 12:23:18.200460  3762 tablet_bootstrap.cc:654] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.202183  3642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a3567afee845fd82bd52553b4d236f" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
W20260709 12:23:18.204250  3680 leader_election.cc:343] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.206697  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a3567afee845fd82bd52553b4d236f" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:18.207374  3567 raft_consensus.cc:3060] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.209129  3760 tablet_bootstrap.cc:492] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b: No bootstrap required, opened a new log
I20260709 12:23:18.209616  3760 ts_tablet_manager.cc:1403] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.006s
I20260709 12:23:18.214181  3760 raft_consensus.cc:359] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.214905  3760 raft_consensus.cc:385] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.215197  3760 raft_consensus.cc:740] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.215943  3760 consensus_queue.cc:260] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.218484  3760 ts_tablet_manager.cc:1434] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 12:23:18.219549  3760 tablet_bootstrap.cc:492] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b: Bootstrap starting.
I20260709 12:23:18.223627  3567 raft_consensus.cc:2468] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 1.
I20260709 12:23:18.225800  3681 leader_election.cc:304] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 242920ef3fb14cbab2b9b913a5891b0b
I20260709 12:23:18.224393  3761 tablet_bootstrap.cc:654] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.226904  3789 raft_consensus.cc:2804] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.227738  3789 raft_consensus.cc:697] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Becoming Leader. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:18.228693  3789 consensus_queue.cc:237] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.234519  3761 tablet_bootstrap.cc:492] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733: No bootstrap required, opened a new log
I20260709 12:23:18.235801  3761 ts_tablet_manager.cc:1403] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.005s
I20260709 12:23:18.240876  3760 tablet_bootstrap.cc:654] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.243001  3764 raft_consensus.cc:493] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.245085  3764 raft_consensus.cc:515] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.247299  3764 leader_election.cc:290] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.235126  3762 tablet_bootstrap.cc:492] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a: No bootstrap required, opened a new log
I20260709 12:23:18.248397  3762 ts_tablet_manager.cc:1403] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent bootstrapping tablet: real 0.076s	user 0.014s	sys 0.002s
I20260709 12:23:18.249874  3462 catalog_manager.cc:5697] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 reported cstate change: term changed from 0 to 1, leader changed from <none> to b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.251873  3762 raft_consensus.cc:359] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.252632  3762 raft_consensus.cc:385] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.252944  3762 raft_consensus.cc:740] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.253955  3716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733" is_pre_election: true
I20260709 12:23:18.253978  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" is_pre_election: true
I20260709 12:23:18.254268  3761 raft_consensus.cc:359] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.255226  3762 consensus_queue.cc:260] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.257303  3761 raft_consensus.cc:385] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.258719  3761 raft_consensus.cc:740] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.261638  3771 raft_consensus.cc:493] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.261302  3567 raft_consensus.cc:2468] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 0.
I20260709 12:23:18.262336  3771 raft_consensus.cc:515] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.263208  3762 ts_tablet_manager.cc:1434] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.005s
I20260709 12:23:18.265640  3607 leader_election.cc:304] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 
I20260709 12:23:18.266984  3764 raft_consensus.cc:2804] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.267333  3764 raft_consensus.cc:493] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.267633  3764 raft_consensus.cc:3060] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.270085  3762 tablet_bootstrap.cc:492] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a: Bootstrap starting.
I20260709 12:23:18.269196  3761 consensus_queue.cc:260] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.272950  3716 raft_consensus.cc:2468] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 0.
I20260709 12:23:18.274923  3761 ts_tablet_manager.cc:1434] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.001s
I20260709 12:23:18.276271  3761 tablet_bootstrap.cc:492] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733: Bootstrap starting.
I20260709 12:23:18.276906  3771 leader_election.cc:290] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.298556  3764 raft_consensus.cc:515] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.301849  3716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:18.302538  3716 raft_consensus.cc:3060] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.302870  3764 leader_election.cc:290] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.313458  3789 raft_consensus.cc:493] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.314095  3789 raft_consensus.cc:515] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.314916  3760 tablet_bootstrap.cc:492] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b: No bootstrap required, opened a new log
I20260709 12:23:18.314985  3761 tablet_bootstrap.cc:654] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.315483  3760 ts_tablet_manager.cc:1403] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent bootstrapping tablet: real 0.096s	user 0.016s	sys 0.010s
I20260709 12:23:18.317696  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:18.318182  3789 leader_election.cc:290] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:18.318317  3567 raft_consensus.cc:3060] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.319388  3642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b" is_pre_election: true
I20260709 12:23:18.320067  3642 raft_consensus.cc:2468] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 0.
I20260709 12:23:18.320281  3716 raft_consensus.cc:2468] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 1.
I20260709 12:23:18.321374  3608 leader_election.cc:304] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:18.321997  3764 raft_consensus.cc:493] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.324116  3766 raft_consensus.cc:2804] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.323184  3680 leader_election.cc:304] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:18.324350  3777 raft_consensus.cc:493] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.324983  3566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" is_pre_election: true
I20260709 12:23:18.324501  3764 raft_consensus.cc:515] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.326601  3789 raft_consensus.cc:2804] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.328107  3566 raft_consensus.cc:2468] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 0.
I20260709 12:23:18.329519  3789 raft_consensus.cc:493] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.326256  3760 raft_consensus.cc:359] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.325830  3777 raft_consensus.cc:515] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.330118  3789 raft_consensus.cc:3060] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.332628  3760 raft_consensus.cc:385] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.336560  3760 raft_consensus.cc:740] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.337797  3766 raft_consensus.cc:697] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Becoming Leader. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:18.350325  3764 leader_election.cc:290] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.354811  3566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" is_pre_election: true
I20260709 12:23:18.354974  3764 raft_consensus.cc:493] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.355494  3566 raft_consensus.cc:2468] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 0.
I20260709 12:23:18.355530  3764 raft_consensus.cc:515] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.357901  3760 consensus_queue.cc:260] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.358565  3716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733" is_pre_election: true
I20260709 12:23:18.366394  3567 raft_consensus.cc:2468] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 1.
I20260709 12:23:18.368034  3766 consensus_queue.cc:237] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.375385  3715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733" is_pre_election: true
I20260709 12:23:18.368378  3777 leader_election.cc:290] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.368289  3764 leader_election.cc:290] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.382368  3760 ts_tablet_manager.cc:1434] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent starting tablet: real 0.066s	user 0.010s	sys 0.000s
W20260709 12:23:18.383580  3608 leader_election.cc:343] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.364420  3607 leader_election.cc:304] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 
I20260709 12:23:18.384546  3760 tablet_bootstrap.cc:492] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b: Bootstrap starting.
I20260709 12:23:18.385811  3777 raft_consensus.cc:2804] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.385957  3715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733" is_pre_election: true
I20260709 12:23:18.386401  3777 raft_consensus.cc:493] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.386879  3777 raft_consensus.cc:3060] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.389804  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" is_pre_election: true
W20260709 12:23:18.393751  3608 leader_election.cc:343] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.405558  3762 tablet_bootstrap.cc:654] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.417699  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" is_pre_election: true
W20260709 12:23:18.418747  3607 leader_election.cc:343] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.419245  3607 leader_election.cc:304] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b; no voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733
I20260709 12:23:18.422760  3715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733" is_pre_election: true
I20260709 12:23:18.424294  3798 raft_consensus.cc:2749] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:18.424765  3607 leader_election.cc:343] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.427153  3797 raft_consensus.cc:493] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.429836  3771 raft_consensus.cc:493] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.430325  3771 raft_consensus.cc:515] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.429036  3797 raft_consensus.cc:515] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.431002  3607 leader_election.cc:304] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b; no voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733
I20260709 12:23:18.436161  3766 raft_consensus.cc:2749] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.440886  3771 leader_election.cc:290] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.442168  3715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e732f8f7ac884e1d987d039cfa748f14" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733" is_pre_election: true
I20260709 12:23:18.443711  3789 raft_consensus.cc:515] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
W20260709 12:23:18.444356  3532 leader_election.cc:343] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.447223  3716 raft_consensus.cc:2393] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 242920ef3fb14cbab2b9b913a5891b0b in current term 1: Already voted for candidate b13e9d12b0b0416a9e214826c4a1c733 in this term.
I20260709 12:23:18.447443  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914a0780eb754f62a3c0221b0c47c2f7" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" is_pre_election: true
W20260709 12:23:18.449456  3607 leader_election.cc:343] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.449677  3715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914a0780eb754f62a3c0221b0c47c2f7" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733" is_pre_election: true
W20260709 12:23:18.451222  3608 leader_election.cc:343] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.451699  3608 leader_election.cc:304] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b; no voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733
I20260709 12:23:18.454257  3789 leader_election.cc:290] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:18.455976  3642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:18.455855  3777 raft_consensus.cc:515] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.455695  3462 catalog_manager.cc:5697] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194). New cstate: current_term: 1 leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.457333  3760 tablet_bootstrap.cc:654] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.462019  3642 raft_consensus.cc:2393] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b13e9d12b0b0416a9e214826c4a1c733 in current term 1: Already voted for candidate 242920ef3fb14cbab2b9b913a5891b0b in this term.
I20260709 12:23:18.463089  3761 tablet_bootstrap.cc:492] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733: No bootstrap required, opened a new log
I20260709 12:23:18.464411  3761 ts_tablet_manager.cc:1403] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733: Time spent bootstrapping tablet: real 0.188s	user 0.034s	sys 0.046s
I20260709 12:23:18.457949  3766 raft_consensus.cc:2749] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.456859  3797 leader_election.cc:290] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.467401  3642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e732f8f7ac884e1d987d039cfa748f14" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b" is_pre_election: true
I20260709 12:23:18.468528  3762 tablet_bootstrap.cc:492] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a: No bootstrap required, opened a new log
I20260709 12:23:18.467299  3641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b" is_pre_election: true
I20260709 12:23:18.469058  3762 ts_tablet_manager.cc:1403] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent bootstrapping tablet: real 0.199s	user 0.017s	sys 0.005s
I20260709 12:23:18.469398  3641 raft_consensus.cc:2393] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ae0b4bb075da417d9154ab6a1fcbd36a in current term 1: Already voted for candidate 242920ef3fb14cbab2b9b913a5891b0b in this term.
I20260709 12:23:18.466897  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:18.471728  3567 raft_consensus.cc:3060] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.469874  3777 leader_election.cc:290] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
W20260709 12:23:18.475185  3530 leader_election.cc:343] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.475697  3530 leader_election.cc:304] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733
I20260709 12:23:18.486225  3771 raft_consensus.cc:2749] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.486099  3762 raft_consensus.cc:359] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.488240  3762 raft_consensus.cc:385] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.488740  3762 raft_consensus.cc:740] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.487546  3761 raft_consensus.cc:359] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.490465  3761 raft_consensus.cc:385] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.490785  3761 raft_consensus.cc:740] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.490078  3762 consensus_queue.cc:260] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.487375  3530 leader_election.cc:304] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733
I20260709 12:23:18.487536  3566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:18.494556  3716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:18.495358  3716 raft_consensus.cc:2393] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 242920ef3fb14cbab2b9b913a5891b0b in current term 1: Already voted for candidate b13e9d12b0b0416a9e214826c4a1c733 in this term.
I20260709 12:23:18.496169  3567 raft_consensus.cc:2468] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 1.
I20260709 12:23:18.496687  3771 raft_consensus.cc:2749] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.497357  3681 leader_election.cc:304] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 242920ef3fb14cbab2b9b913a5891b0b
I20260709 12:23:18.499423  3789 raft_consensus.cc:2804] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.499912  3789 raft_consensus.cc:697] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Becoming Leader. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:18.500748  3789 consensus_queue.cc:237] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.501827  3761 consensus_queue.cc:260] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.507753  3762 ts_tablet_manager.cc:1434] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent starting tablet: real 0.038s	user 0.013s	sys 0.009s
I20260709 12:23:18.509060  3761 ts_tablet_manager.cc:1434] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.006s
I20260709 12:23:18.510214  3762 tablet_bootstrap.cc:492] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a: Bootstrap starting.
I20260709 12:23:18.510373  3761 tablet_bootstrap.cc:492] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733: Bootstrap starting.
I20260709 12:23:18.525048  3761 tablet_bootstrap.cc:654] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.532380  3762 tablet_bootstrap.cc:654] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.546180  3789 raft_consensus.cc:493] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.547068  3789 raft_consensus.cc:515] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
W20260709 12:23:18.535806  3530 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:40141, user_credentials={real_user=slave}} blocked reactor thread for 48467.2us
I20260709 12:23:18.551127  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" is_pre_election: true
I20260709 12:23:18.551232  3607 leader_election.cc:304] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b; no voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733
I20260709 12:23:18.552337  3777 raft_consensus.cc:2749] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:23:18.555380  3607 consensus_peers.cc:597] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b -> Peer ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693): Couldn't send request to peer ae0b4bb075da417d9154ab6a1fcbd36a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:18.556869  3681 leader_election.cc:343] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.556236  3463 catalog_manager.cc:5697] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 reported cstate change: term changed from 0 to 1, leader changed from <none> to b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.559672  3641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b" is_pre_election: true
I20260709 12:23:18.560452  3760 tablet_bootstrap.cc:492] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b: No bootstrap required, opened a new log
I20260709 12:23:18.560595  3761 tablet_bootstrap.cc:492] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733: No bootstrap required, opened a new log
I20260709 12:23:18.560940  3760 ts_tablet_manager.cc:1403] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b: Time spent bootstrapping tablet: real 0.177s	user 0.033s	sys 0.037s
I20260709 12:23:18.561074  3761 ts_tablet_manager.cc:1403] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.010s
W20260709 12:23:18.561646  3680 leader_election.cc:343] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:18.562101  3680 leader_election.cc:304] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b13e9d12b0b0416a9e214826c4a1c733; no voters: 242920ef3fb14cbab2b9b913a5891b0b, ae0b4bb075da417d9154ab6a1fcbd36a
I20260709 12:23:18.558535  3789 leader_election.cc:290] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:18.563022  3804 raft_consensus.cc:2749] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.564898  3760 raft_consensus.cc:359] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.565635  3760 raft_consensus.cc:385] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.565953  3760 raft_consensus.cc:740] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.566493  3762 tablet_bootstrap.cc:492] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a: No bootstrap required, opened a new log
I20260709 12:23:18.567014  3762 ts_tablet_manager.cc:1403] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.005s
I20260709 12:23:18.566699  3760 consensus_queue.cc:260] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.569397  3760 ts_tablet_manager.cc:1434] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:23:18.570446  3760 tablet_bootstrap.cc:492] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b: Bootstrap starting.
I20260709 12:23:18.570173  3762 raft_consensus.cc:359] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.571138  3762 raft_consensus.cc:385] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.571508  3762 raft_consensus.cc:740] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.572288  3762 consensus_queue.cc:260] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.574730  3762 ts_tablet_manager.cc:1434] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:18.574584  3761 raft_consensus.cc:359] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.575780  3761 raft_consensus.cc:385] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.576109  3761 raft_consensus.cc:740] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.576864  3761 consensus_queue.cc:260] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.579316  3761 ts_tablet_manager.cc:1434] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 12:23:18.576462  3762 tablet_bootstrap.cc:492] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a: Bootstrap starting.
I20260709 12:23:18.580241  3761 tablet_bootstrap.cc:492] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733: Bootstrap starting.
I20260709 12:23:18.586715  3762 tablet_bootstrap.cc:654] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.588521  3761 tablet_bootstrap.cc:654] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.586719  3760 tablet_bootstrap.cc:654] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.609417  3762 tablet_bootstrap.cc:492] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a: No bootstrap required, opened a new log
I20260709 12:23:18.610240  3762 ts_tablet_manager.cc:1403] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.008s
I20260709 12:23:18.612139  3761 tablet_bootstrap.cc:492] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733: No bootstrap required, opened a new log
I20260709 12:23:18.612746  3761 ts_tablet_manager.cc:1403] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.017s
I20260709 12:23:18.614599  3762 raft_consensus.cc:359] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.616793  3762 raft_consensus.cc:385] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:23:18.617170  3511 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:18.617151  3762 raft_consensus.cc:740] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.618088  3760 tablet_bootstrap.cc:492] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b: No bootstrap required, opened a new log
I20260709 12:23:18.618606  3760 ts_tablet_manager.cc:1403] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.022s
I20260709 12:23:18.618152  3762 consensus_queue.cc:260] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.620749  3762 ts_tablet_manager.cc:1434] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:23:18.621744  3762 tablet_bootstrap.cc:492] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a: Bootstrap starting.
I20260709 12:23:18.621752  3760 raft_consensus.cc:359] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.622550  3760 raft_consensus.cc:385] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.622812  3760 raft_consensus.cc:740] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.623351  3761 raft_consensus.cc:359] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.623481  3760 consensus_queue.cc:260] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.624125  3761 raft_consensus.cc:385] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.624539  3761 raft_consensus.cc:740] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.625269  3761 consensus_queue.cc:260] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.625916  3760 ts_tablet_manager.cc:1434] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:18.626824  3760 tablet_bootstrap.cc:492] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b: Bootstrap starting.
I20260709 12:23:18.627753  3761 ts_tablet_manager.cc:1434] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 12:23:18.628687  3761 tablet_bootstrap.cc:492] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733: Bootstrap starting.
I20260709 12:23:18.632661  3760 tablet_bootstrap.cc:654] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.634660  3761 tablet_bootstrap.cc:654] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.643711  3760 tablet_bootstrap.cc:492] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b: No bootstrap required, opened a new log
I20260709 12:23:18.644178  3760 ts_tablet_manager.cc:1403] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.008s
I20260709 12:23:18.646746  3777 consensus_queue.cc:1048] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.647027  3760 raft_consensus.cc:359] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.647701  3760 raft_consensus.cc:385] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.647986  3762 tablet_bootstrap.cc:654] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.648015  3760 raft_consensus.cc:740] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.649945  3760 consensus_queue.cc:260] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.651962  3760 ts_tablet_manager.cc:1434] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:23:18.655107  3777 raft_consensus.cc:493] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.655644  3777 raft_consensus.cc:515] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.659003  3566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" is_pre_election: true
I20260709 12:23:18.659678  3566 raft_consensus.cc:2468] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 0.
I20260709 12:23:18.661513  3607 leader_election.cc:304] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 
I20260709 12:23:18.662693  3797 raft_consensus.cc:2804] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.663065  3797 raft_consensus.cc:493] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.663393  3797 raft_consensus.cc:3060] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.666559  3777 leader_election.cc:290] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.668346  3715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733" is_pre_election: true
I20260709 12:23:18.669045  3715 raft_consensus.cc:2468] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 0.
I20260709 12:23:18.673288  3804 consensus_queue.cc:1048] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.679071  3797 raft_consensus.cc:515] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.683503  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:18.684124  3567 raft_consensus.cc:3060] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.691180  3567 raft_consensus.cc:2468] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 1.
I20260709 12:23:18.692461  3607 leader_election.cc:304] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 
I20260709 12:23:18.693442  3777 raft_consensus.cc:2804] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.693958  3777 raft_consensus.cc:697] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Becoming Leader. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:18.694823  3777 consensus_queue.cc:237] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.721625  3641 raft_consensus.cc:3060] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.722047  3761 tablet_bootstrap.cc:492] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733: No bootstrap required, opened a new log
I20260709 12:23:18.722551  3761 ts_tablet_manager.cc:1403] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733: Time spent bootstrapping tablet: real 0.094s	user 0.010s	sys 0.001s
I20260709 12:23:18.726622  3797 leader_election.cc:290] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.727294  3715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:18.728031  3715 raft_consensus.cc:3060] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.725257  3761 raft_consensus.cc:359] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.733590  3761 raft_consensus.cc:385] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.733919  3761 raft_consensus.cc:740] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.735333  3761 consensus_queue.cc:260] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.735849  3715 raft_consensus.cc:2468] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 1.
I20260709 12:23:18.736353  3462 catalog_manager.cc:5697] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194). New cstate: current_term: 1 leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.737840  3761 ts_tablet_manager.cc:1434] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:23:18.742169  3762 tablet_bootstrap.cc:492] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a: No bootstrap required, opened a new log
I20260709 12:23:18.742705  3762 ts_tablet_manager.cc:1403] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent bootstrapping tablet: real 0.121s	user 0.030s	sys 0.042s
I20260709 12:23:18.742616  3804 consensus_queue.cc:1048] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.745512  3762 raft_consensus.cc:359] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.746407  3762 raft_consensus.cc:385] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.746764  3762 raft_consensus.cc:740] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.747680  3762 consensus_queue.cc:260] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.750345  3762 ts_tablet_manager.cc:1434] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:18.751386  3762 tablet_bootstrap.cc:492] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a: Bootstrap starting.
I20260709 12:23:18.754040  3803 raft_consensus.cc:493] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.754693  3803 raft_consensus.cc:515] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.757431  3762 tablet_bootstrap.cc:654] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:18.757840  3642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b" is_pre_election: true
I20260709 12:23:18.762418  3715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733" is_pre_election: true
I20260709 12:23:18.765661  3532 leader_election.cc:304] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733
I20260709 12:23:18.766759  3771 raft_consensus.cc:2749] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:18.769649  3803 leader_election.cc:290] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.775957  3797 raft_consensus.cc:493] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.776737  3797 raft_consensus.cc:515] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.780642  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e732f8f7ac884e1d987d039cfa748f14" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" is_pre_election: true
I20260709 12:23:18.781318  3567 raft_consensus.cc:2468] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 0.
I20260709 12:23:18.782141  3762 tablet_bootstrap.cc:492] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a: No bootstrap required, opened a new log
I20260709 12:23:18.782634  3762 ts_tablet_manager.cc:1403] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent bootstrapping tablet: real 0.031s	user 0.020s	sys 0.008s
I20260709 12:23:18.782936  3607 leader_election.cc:304] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 
I20260709 12:23:18.783809  3777 raft_consensus.cc:2804] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.784221  3777 raft_consensus.cc:493] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.784161  3715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e732f8f7ac884e1d987d039cfa748f14" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733" is_pre_election: true
I20260709 12:23:18.784655  3777 raft_consensus.cc:3060] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.784864  3715 raft_consensus.cc:2468] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 0.
I20260709 12:23:18.785456  3762 raft_consensus.cc:359] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.786306  3797 leader_election.cc:290] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.786144  3762 raft_consensus.cc:385] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:18.788376  3762 raft_consensus.cc:740] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Initialized, Role: FOLLOWER
I20260709 12:23:18.789172  3762 consensus_queue.cc:260] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.791635  3766 consensus_queue.cc:1048] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.791664  3762 ts_tablet_manager.cc:1434] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:23:18.791996  3804 raft_consensus.cc:493] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.792676  3804 raft_consensus.cc:515] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.794921  3804 leader_election.cc:290] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:18.796356  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" is_pre_election: true
I20260709 12:23:18.797159  3567 raft_consensus.cc:2468] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 0.
I20260709 12:23:18.798434  3681 leader_election.cc:304] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:18.799345  3641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b" is_pre_election: true
I20260709 12:23:18.799554  3773 raft_consensus.cc:2804] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.791568  3777 raft_consensus.cc:515] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.799994  3773 raft_consensus.cc:493] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.800108  3641 raft_consensus.cc:2468] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 0.
I20260709 12:23:18.800452  3773 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.804049  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e732f8f7ac884e1d987d039cfa748f14" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:18.804842  3567 raft_consensus.cc:3060] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.807797  3773 raft_consensus.cc:515] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.811137  3566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:18.811949  3566 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.819345  3566 raft_consensus.cc:2468] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 1.
I20260709 12:23:18.820549  3681 leader_election.cc:304] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:18.821379  3804 raft_consensus.cc:2804] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.821256  3716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e732f8f7ac884e1d987d039cfa748f14" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:18.821967  3716 raft_consensus.cc:3060] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.821031  3777 leader_election.cc:290] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.822866  3773 leader_election.cc:290] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:18.824640  3567 raft_consensus.cc:2468] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 1.
I20260709 12:23:18.821900  3804 raft_consensus.cc:697] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Becoming Leader. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:18.825925  3607 leader_election.cc:304] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 
I20260709 12:23:18.825954  3641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:18.826659  3641 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.826351  3804 consensus_queue.cc:237] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.826913  3777 raft_consensus.cc:2804] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.827396  3777 raft_consensus.cc:697] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Becoming Leader. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:18.828199  3777 consensus_queue.cc:237] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.838119  3716 raft_consensus.cc:2468] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 1.
I20260709 12:23:18.839063  3641 raft_consensus.cc:2468] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 1.
I20260709 12:23:18.853874  3797 consensus_queue.cc:1048] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:18.854693  3461 catalog_manager.cc:5697] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194). New cstate: current_term: 1 leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.855206  3462 catalog_manager.cc:5697] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 reported cstate change: term changed from 0 to 1, leader changed from <none> to b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.888176  3804 raft_consensus.cc:493] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.888691  3804 raft_consensus.cc:515] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.891922  3804 leader_election.cc:290] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:18.892167  3641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914a0780eb754f62a3c0221b0c47c2f7" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b" is_pre_election: true
I20260709 12:23:18.892601  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914a0780eb754f62a3c0221b0c47c2f7" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" is_pre_election: true
I20260709 12:23:18.892830  3641 raft_consensus.cc:2468] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 0.
I20260709 12:23:18.893190  3567 raft_consensus.cc:2468] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 0.
I20260709 12:23:18.894225  3680 leader_election.cc:304] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:18.894912  3804 raft_consensus.cc:2804] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.895301  3804 raft_consensus.cc:493] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.895653  3804 raft_consensus.cc:3060] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.902694  3804 raft_consensus.cc:515] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.904946  3804 leader_election.cc:290] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:18.905975  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914a0780eb754f62a3c0221b0c47c2f7" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:18.905997  3641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914a0780eb754f62a3c0221b0c47c2f7" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:18.906629  3567 raft_consensus.cc:3060] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.906701  3641 raft_consensus.cc:3060] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.911860  3641 raft_consensus.cc:2468] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 1.
I20260709 12:23:18.912973  3680 leader_election.cc:304] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:18.913884  3804 raft_consensus.cc:2804] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.914420  3804 raft_consensus.cc:697] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Becoming Leader. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:18.914860  3567 raft_consensus.cc:2468] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 1.
I20260709 12:23:18.915318  3804 consensus_queue.cc:237] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.923549  3461 catalog_manager.cc:5697] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 reported cstate change: term changed from 0 to 1, leader changed from <none> to b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:18.938411  3803 raft_consensus.cc:493] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.939038  3803 raft_consensus.cc:515] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.941224  3803 leader_election.cc:290] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.942214  3641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b" is_pre_election: true
I20260709 12:23:18.942408  3716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733" is_pre_election: true
I20260709 12:23:18.942912  3641 raft_consensus.cc:2468] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 0.
I20260709 12:23:18.943037  3716 raft_consensus.cc:2468] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 0.
I20260709 12:23:18.944054  3532 leader_election.cc:304] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:18.944731  3803 raft_consensus.cc:2804] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:18.945097  3803 raft_consensus.cc:493] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:18.945410  3803 raft_consensus.cc:3060] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.950202  3803 raft_consensus.cc:515] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.951845  3803 leader_election.cc:290] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 1 election: Requested vote from peers 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:18.952775  3641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:18.952951  3716 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:18.953301  3641 raft_consensus.cc:3060] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.953557  3716 raft_consensus.cc:3060] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:18.960626  3716 raft_consensus.cc:2468] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 1.
I20260709 12:23:18.960626  3641 raft_consensus.cc:2468] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 1.
I20260709 12:23:18.961303  3804 consensus_queue.cc:1048] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.962371  3532 leader_election.cc:304] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:18.963378  3803 raft_consensus.cc:2804] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:18.964025  3803 raft_consensus.cc:697] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 LEADER]: Becoming Leader. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Running, Role: LEADER
I20260709 12:23:18.965248  3803 consensus_queue.cc:237] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:18.978135  3773 consensus_queue.cc:1048] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:18.979087  3463 catalog_manager.cc:5697] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a reported cstate change: term changed from 0 to 1, leader changed from <none> to ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193). New cstate: current_term: 1 leader_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.055187  3567 raft_consensus.cc:3060] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:19.078792  3641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
mode: GRACEFUL
new_leader_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
 from {username='slave'} at 127.0.0.1:35802
I20260709 12:23:19.079427  3641 raft_consensus.cc:606] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Received request to transfer leadership to TS ae0b4bb075da417d9154ab6a1fcbd36a
I20260709 12:23:19.084816  3641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
mode: GRACEFUL
new_leader_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
 from {username='slave'} at 127.0.0.1:35802
I20260709 12:23:19.085340  3641 raft_consensus.cc:606] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Received request to transfer leadership to TS ae0b4bb075da417d9154ab6a1fcbd36a
I20260709 12:23:19.100185  3716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d"
dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
mode: GRACEFUL
new_leader_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
 from {username='slave'} at 127.0.0.1:50170
I20260709 12:23:19.100783  3716 raft_consensus.cc:606] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Received request to transfer leadership to TS ae0b4bb075da417d9154ab6a1fcbd36a
I20260709 12:23:19.109454  3716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152"
dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
mode: GRACEFUL
new_leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:50170
I20260709 12:23:19.110069  3716 raft_consensus.cc:606] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Received request to transfer leadership to TS 242920ef3fb14cbab2b9b913a5891b0b
I20260709 12:23:19.115366  3716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99a3567afee845fd82bd52553b4d236f"
dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
mode: GRACEFUL
new_leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:50170
I20260709 12:23:19.115950  3716 raft_consensus.cc:606] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Received request to transfer leadership to TS 242920ef3fb14cbab2b9b913a5891b0b
I20260709 12:23:19.121452  3716 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "914a0780eb754f62a3c0221b0c47c2f7"
dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
mode: GRACEFUL
new_leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:50170
I20260709 12:23:19.122092  3716 raft_consensus.cc:606] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Received request to transfer leadership to TS 242920ef3fb14cbab2b9b913a5891b0b
I20260709 12:23:19.131850  3641 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e732f8f7ac884e1d987d039cfa748f14"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
mode: GRACEFUL
new_leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
 from {username='slave'} at 127.0.0.1:35802
I20260709 12:23:19.132411  3641 raft_consensus.cc:606] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Received request to transfer leadership to TS b13e9d12b0b0416a9e214826c4a1c733
I20260709 12:23:19.245115  3804 consensus_queue.cc:1048] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.260452  3789 consensus_queue.cc:1048] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.266121  3766 consensus_queue.cc:1048] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.266659  3797 consensus_queue.cc:1048] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.268200  3797 consensus_queue.cc:1048] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.299175  3777 consensus_queue.cc:1048] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.307727  3797 raft_consensus.cc:993] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b: : Instructing follower ae0b4bb075da417d9154ab6a1fcbd36a to start an election
I20260709 12:23:19.308465  3797 raft_consensus.cc:1081] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Signalling peer ae0b4bb075da417d9154ab6a1fcbd36a to start an election
I20260709 12:23:19.318727  3789 raft_consensus.cc:993] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733: : Instructing follower 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:19.319340  3789 raft_consensus.cc:1081] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Signalling peer 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:19.322422  3641 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99a3567afee845fd82bd52553b4d236f"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:35792
I20260709 12:23:19.323297  3641 raft_consensus.cc:493] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:19.323812  3641 raft_consensus.cc:3060] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.326956  3567 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838"
dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
 from {username='slave'} at 127.0.0.1:59376
I20260709 12:23:19.327574  3567 raft_consensus.cc:493] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:19.327979  3567 raft_consensus.cc:3060] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.339859  3641 raft_consensus.cc:515] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.343542  3777 raft_consensus.cc:993] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b: : Instructing follower b13e9d12b0b0416a9e214826c4a1c733 to start an election
I20260709 12:23:19.344003  3777 raft_consensus.cc:1081] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Signalling peer b13e9d12b0b0416a9e214826c4a1c733 to start an election
I20260709 12:23:19.344879  3567 raft_consensus.cc:515] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.347738  3714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e732f8f7ac884e1d987d039cfa748f14"
dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
 from {username='slave'} at 127.0.0.1:50154
I20260709 12:23:19.353229  3564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a3567afee845fd82bd52553b4d236f" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:19.354236  3564 raft_consensus.cc:3060] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.359844  3715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:19.360498  3715 raft_consensus.cc:3060] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.360639  3642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:19.361474  3642 raft_consensus.cc:3055] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:19.361897  3642 raft_consensus.cc:740] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:19.362985  3642 consensus_queue.cc:260] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.365164  3642 raft_consensus.cc:3060] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.367318  3715 raft_consensus.cc:2468] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 2.
I20260709 12:23:19.368688  3532 leader_election.cc:304] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:19.369623  3771 raft_consensus.cc:2804] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:19.370100  3771 raft_consensus.cc:697] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Becoming Leader. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Running, Role: LEADER
I20260709 12:23:19.370883  3771 consensus_queue.cc:237] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.374469  3567 leader_election.cc:290] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 2 election: Requested vote from peers 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:19.377892  3641 leader_election.cc:290] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 2 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:19.379359  3715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a3567afee845fd82bd52553b4d236f" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:19.380026  3715 raft_consensus.cc:3055] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:19.380425  3715 raft_consensus.cc:740] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:19.381284  3715 consensus_queue.cc:260] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.382911  3715 raft_consensus.cc:3060] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.398880  3715 raft_consensus.cc:2468] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 2.
I20260709 12:23:19.399551  3714 raft_consensus.cc:493] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:19.400820  3714 raft_consensus.cc:3060] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.412109  3608 leader_election.cc:304] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:19.413198  3798 raft_consensus.cc:2804] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:19.413797  3798 raft_consensus.cc:697] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Becoming Leader. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:19.414564  3804 raft_consensus.cc:993] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733: : Instructing follower ae0b4bb075da417d9154ab6a1fcbd36a to start an election
I20260709 12:23:19.414669  3564 raft_consensus.cc:2468] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 2.
I20260709 12:23:19.415149  3804 raft_consensus.cc:1081] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Signalling peer ae0b4bb075da417d9154ab6a1fcbd36a to start an election
I20260709 12:23:19.414748  3798 consensus_queue.cc:237] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.416982  3567 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d"
dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
 from {username='slave'} at 127.0.0.1:59386
I20260709 12:23:19.417548  3567 raft_consensus.cc:493] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:19.417953  3567 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.419025  3642 raft_consensus.cc:2468] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 2.
I20260709 12:23:19.424686  3567 raft_consensus.cc:515] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.425335  3714 raft_consensus.cc:515] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.428752  3715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:19.429359  3715 raft_consensus.cc:3055] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:19.429713  3715 raft_consensus.cc:740] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:19.430491  3715 consensus_queue.cc:260] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.431531  3715 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.430727  3462 catalog_manager.cc:5697] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b reported cstate change: term changed from 1 to 2, leader changed from b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195) to 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194). New cstate: current_term: 2 leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.437459  3566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e732f8f7ac884e1d987d039cfa748f14" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:19.438139  3566 raft_consensus.cc:3060] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.438305  3715 raft_consensus.cc:2468] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 2.
I20260709 12:23:19.439530  3532 leader_election.cc:304] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:19.440366  3771 raft_consensus.cc:2804] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:19.440804  3771 raft_consensus.cc:697] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Becoming Leader. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Running, Role: LEADER
I20260709 12:23:19.441674  3567 leader_election.cc:290] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 2 election: Requested vote from peers 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:19.441648  3771 consensus_queue.cc:237] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.446595  3566 raft_consensus.cc:2468] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 2.
I20260709 12:23:19.447160  3714 leader_election.cc:290] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 2 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:19.447404  3642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e732f8f7ac884e1d987d039cfa748f14" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:19.448052  3642 raft_consensus.cc:3055] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:19.447965  3681 leader_election.cc:304] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:19.448477  3642 raft_consensus.cc:740] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:19.449195  3773 raft_consensus.cc:2804] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:19.449429  3642 consensus_queue.cc:260] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.450009  3773 raft_consensus.cc:697] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 LEADER]: Becoming Leader. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:19.451001  3642 raft_consensus.cc:3060] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.451773  3773 consensus_queue.cc:237] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.453970  3641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:19.454591  3641 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.457314  3789 raft_consensus.cc:993] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733: : Instructing follower 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:19.457857  3804 raft_consensus.cc:1081] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Signalling peer 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:19.459570  3640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:35792
I20260709 12:23:19.460114  3640 raft_consensus.cc:493] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:19.460466  3640 raft_consensus.cc:3060] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.465821  3463 catalog_manager.cc:5697] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a reported cstate change: term changed from 1 to 2, leader changed from b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195) to ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193). New cstate: current_term: 2 leader_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.467754  3463 catalog_manager.cc:5697] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a reported cstate change: term changed from 1 to 2, leader changed from 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194) to ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193). New cstate: current_term: 2 leader_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.468884  3641 raft_consensus.cc:2468] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 2.
I20260709 12:23:19.468705  3640 raft_consensus.cc:515] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.472091  3566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:19.472721  3566 raft_consensus.cc:3060] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.474246  3714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19f46c4700004a08bf63901dae2c7152" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:19.474821  3714 raft_consensus.cc:3055] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:19.475158  3714 raft_consensus.cc:740] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:19.476226  3714 consensus_queue.cc:260] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.477309  3714 raft_consensus.cc:3060] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.478981  3462 catalog_manager.cc:5697] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 reported cstate change: term changed from 1 to 2, leader changed from 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194) to b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:19.479839  3566 raft_consensus.cc:2468] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 2.
I20260709 12:23:19.481197  3607 leader_election.cc:304] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 
I20260709 12:23:19.482005  3797 raft_consensus.cc:2804] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:19.482522  3797 raft_consensus.cc:697] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Becoming Leader. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:19.483398  3797 consensus_queue.cc:237] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.489158  3642 raft_consensus.cc:2468] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 2.
I20260709 12:23:19.472190  3640 leader_election.cc:290] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 2 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:19.492832  3771 consensus_queue.cc:1048] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.493027  3463 catalog_manager.cc:5697] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b reported cstate change: term changed from 1 to 2, leader changed from b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195) to 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194). New cstate: current_term: 2 leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.497781  3714 raft_consensus.cc:2468] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 2.
I20260709 12:23:19.498155  3797 raft_consensus.cc:993] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b: : Instructing follower ae0b4bb075da417d9154ab6a1fcbd36a to start an election
I20260709 12:23:19.498596  3797 raft_consensus.cc:1081] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Signalling peer ae0b4bb075da417d9154ab6a1fcbd36a to start an election
I20260709 12:23:19.500308  3566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05"
dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
 from {username='slave'} at 127.0.0.1:59376
I20260709 12:23:19.501017  3566 raft_consensus.cc:493] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:19.501533  3566 raft_consensus.cc:3060] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.505373  3773 consensus_queue.cc:1048] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.514763  3566 raft_consensus.cc:515] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.525847  3789 consensus_queue.cc:1048] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:19.529529  3639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:19.530301  3639 raft_consensus.cc:3055] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:19.530745  3639 raft_consensus.cc:740] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:19.531803  3639 consensus_queue.cc:260] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.536705  3639 raft_consensus.cc:3060] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.547972  3789 raft_consensus.cc:993] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733: : Instructing follower 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:19.547894  3714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:19.548487  3789 raft_consensus.cc:1081] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Signalling peer 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:19.548635  3714 raft_consensus.cc:3060] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.549558  3566 leader_election.cc:290] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 2 election: Requested vote from peers 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:19.550565  3642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "914a0780eb754f62a3c0221b0c47c2f7"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:35792
I20260709 12:23:19.551141  3642 raft_consensus.cc:493] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:19.551522  3642 raft_consensus.cc:3060] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.555830  3714 raft_consensus.cc:2468] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 2.
I20260709 12:23:19.557264  3532 leader_election.cc:304] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:19.558260  3771 raft_consensus.cc:2804] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:19.559324  3639 raft_consensus.cc:2468] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 2.
I20260709 12:23:19.559461  3771 raft_consensus.cc:697] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Becoming Leader. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Running, Role: LEADER
I20260709 12:23:19.561538  3642 raft_consensus.cc:515] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.562638  3771 consensus_queue.cc:237] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.566558  3642 leader_election.cc:290] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 2 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:19.568856  3714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914a0780eb754f62a3c0221b0c47c2f7" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:19.569507  3714 raft_consensus.cc:3055] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:19.569878  3714 raft_consensus.cc:740] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:19.572018  3461 catalog_manager.cc:5697] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a reported cstate change: term changed from 1 to 2, leader changed from 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194) to ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193). New cstate: current_term: 2 leader_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.579735  3566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "914a0780eb754f62a3c0221b0c47c2f7" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:19.580368  3566 raft_consensus.cc:3060] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.573552  3714 consensus_queue.cc:260] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.586649  3714 raft_consensus.cc:3060] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:19.587193  3566 raft_consensus.cc:2468] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 2.
I20260709 12:23:19.591248  3607 leader_election.cc:304] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 
I20260709 12:23:19.592612  3798 raft_consensus.cc:2804] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:19.592823  3845 consensus_queue.cc:1048] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:19.593160  3798 raft_consensus.cc:697] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Becoming Leader. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:19.594060  3798 consensus_queue.cc:237] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:19.594615  3714 raft_consensus.cc:2468] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 2.
I20260709 12:23:19.606568  3463 catalog_manager.cc:5697] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b reported cstate change: term changed from 1 to 2, leader changed from b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195) to 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194). New cstate: current_term: 2 leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:19.777815  3715 raft_consensus.cc:1275] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Refusing update from remote peer ae0b4bb075da417d9154ab6a1fcbd36a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:19.779564  3803 consensus_queue.cc:1048] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:19.789395  3641 raft_consensus.cc:1275] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Refusing update from remote peer ae0b4bb075da417d9154ab6a1fcbd36a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:19.791569  3845 consensus_queue.cc:1048] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:19.823604  3715 raft_consensus.cc:1275] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Refusing update from remote peer ae0b4bb075da417d9154ab6a1fcbd36a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:19.824951  3845 consensus_queue.cc:1048] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:19.833511  3641 raft_consensus.cc:1275] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Refusing update from remote peer ae0b4bb075da417d9154ab6a1fcbd36a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:19.835307  3845 consensus_queue.cc:1048] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:19.927640  3641 raft_consensus.cc:1275] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Refusing update from remote peer b13e9d12b0b0416a9e214826c4a1c733: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:19.929217  3773 consensus_queue.cc:1048] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:19.931958  3715 raft_consensus.cc:1275] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:19.933616  3798 consensus_queue.cc:1048] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:19.937674  3566 raft_consensus.cc:1275] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Refusing update from remote peer b13e9d12b0b0416a9e214826c4a1c733: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:19.940043  3773 consensus_queue.cc:1048] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:19.943318  3567 raft_consensus.cc:1275] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:19.944954  3766 consensus_queue.cc:1048] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:19.980181  3715 raft_consensus.cc:1275] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:19.981515  3766 consensus_queue.cc:1048] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:19.990945  3566 raft_consensus.cc:1275] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:19.992766  3766 consensus_queue.cc:1048] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:20.001283  3641 raft_consensus.cc:1275] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Refusing update from remote peer ae0b4bb075da417d9154ab6a1fcbd36a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:20.002913  3845 consensus_queue.cc:1048] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:20.014552  3714 raft_consensus.cc:1275] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Refusing update from remote peer ae0b4bb075da417d9154ab6a1fcbd36a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:20.016163  3844 consensus_queue.cc:1048] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:20.037506  3566 raft_consensus.cc:1275] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:20.038981  3766 consensus_queue.cc:1048] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:20.045187  3714 raft_consensus.cc:1275] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:20.046710  3766 consensus_queue.cc:1048] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:21.785904  3588 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5060 after lost signal to thread 3434
W20260709 12:23:21.786720  3588 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5740 after lost signal to thread 3497
W20260709 12:23:21.787494  3588 debug-util.cc:398] Leaking SignalData structure 0x7b08001be7e0 after lost signal to thread 3591
W20260709 12:23:21.788163  3588 debug-util.cc:398] Leaking SignalData structure 0x7b08001bee00 after lost signal to thread 3666
W20260709 12:23:21.788805  3588 debug-util.cc:398] Leaking SignalData structure 0x7b0800082160 after lost signal to thread 3741
I20260709 12:23:22.189265  3431 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:23:22.192010  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:22.224869  3566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d"
dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
mode: GRACEFUL
new_leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
 from {username='slave'} at 127.0.0.1:59392
I20260709 12:23:22.225684  3566 raft_consensus.cc:606] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Received request to transfer leadership to TS b13e9d12b0b0416a9e214826c4a1c733
I20260709 12:23:22.239635  3639 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
mode: GRACEFUL
new_leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
 from {username='slave'} at 127.0.0.1:35808
I20260709 12:23:22.240257  3639 raft_consensus.cc:606] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Received request to transfer leadership to TS b13e9d12b0b0416a9e214826c4a1c733
I20260709 12:23:22.241626  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:23:22.242127  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:22.258459  3853 raft_consensus.cc:993] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b: : Instructing follower b13e9d12b0b0416a9e214826c4a1c733 to start an election
I20260709 12:23:22.258976  3766 raft_consensus.cc:1081] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Signalling peer b13e9d12b0b0416a9e214826c4a1c733 to start an election
I20260709 12:23:22.260726  3714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350"
dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
 from {username='slave'} at 127.0.0.1:50154
I20260709 12:23:22.261240  3714 raft_consensus.cc:493] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:22.261665  3714 raft_consensus.cc:3060] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.266641  3714 raft_consensus.cc:515] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:22.268530  3714 leader_election.cc:290] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 2 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:22.269523  3566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:22.269743  3639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:22.270417  3639 raft_consensus.cc:3055] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:22.270221  3566 raft_consensus.cc:3060] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.271049  3639 raft_consensus.cc:740] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:22.272045  3639 consensus_queue.cc:260] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:22.273048  3639 raft_consensus.cc:3060] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:22.278079  3639 raft_consensus.cc:2468] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 2.
I20260709 12:23:22.278368  3566 raft_consensus.cc:2468] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 2.
I20260709 12:23:22.279358  3680 leader_election.cc:304] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:22.280395  3804 raft_consensus.cc:2804] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:22.281951  3804 raft_consensus.cc:697] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 LEADER]: Becoming Leader. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:22.282883  3804 consensus_queue.cc:237] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:22.291517  3461 catalog_manager.cc:5697] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 reported cstate change: term changed from 1 to 2, leader changed from 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194) to b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:22.348156  3863 raft_consensus.cc:993] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a: : Instructing follower b13e9d12b0b0416a9e214826c4a1c733 to start an election
I20260709 12:23:22.348706  3845 raft_consensus.cc:1081] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Signalling peer b13e9d12b0b0416a9e214826c4a1c733 to start an election
I20260709 12:23:22.350872  3714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d"
dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
 from {username='slave'} at 127.0.0.1:50162
I20260709 12:23:22.351572  3714 raft_consensus.cc:493] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:22.351966  3714 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.357879  3714 raft_consensus.cc:515] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:22.360265  3714 leader_election.cc:290] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 3 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:22.360659  3566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:22.361246  3639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:22.361338  3566 raft_consensus.cc:3055] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:22.361972  3639 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.362049  3566 raft_consensus.cc:740] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Running, Role: LEADER
I20260709 12:23:22.363183  3566 consensus_queue.cc:260] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:22.364898  3566 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:22.367180  3639 raft_consensus.cc:2468] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 3.
I20260709 12:23:22.368289  3680 leader_election.cc:304] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:22.369642  3804 raft_consensus.cc:2804] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:22.370153  3804 raft_consensus.cc:697] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 3 LEADER]: Becoming Leader. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:22.370975  3804 consensus_queue.cc:237] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:22.374188  3566 raft_consensus.cc:2468] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 3.
I20260709 12:23:22.382436  3461 catalog_manager.cc:5697] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 reported cstate change: term changed from 2 to 3, leader changed from ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193) to b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:22.847293  3639 raft_consensus.cc:1275] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Refusing update from remote peer b13e9d12b0b0416a9e214826c4a1c733: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:22.848799  3804 consensus_queue.cc:1048] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:22.851902  3566 raft_consensus.cc:1275] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Refusing update from remote peer b13e9d12b0b0416a9e214826c4a1c733: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:22.853467  3857 consensus_queue.cc:1048] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:22.863806  3567 raft_consensus.cc:1275] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Refusing update from remote peer b13e9d12b0b0416a9e214826c4a1c733: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:22.865417  3804 consensus_queue.cc:1048] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:22.899324  3639 raft_consensus.cc:1275] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Refusing update from remote peer b13e9d12b0b0416a9e214826c4a1c733: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:22.905515  3804 consensus_queue.cc:1048] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:23.243085  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:23.248980  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:23.249470  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:24.250763  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:24.256421  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:24.256901  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:25.258723  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:25.265954  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:25.266404  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:26.267577  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:26.272982  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:26.273613  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:27.274591  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:27.279616  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:27.280064  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:28.281121  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:28.286410  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:28.286895  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:29.287869  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:29.293653  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:29.294114  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:30.295075  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:30.301003  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:30.301394  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:31.302330  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:31.307561  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:31.308122  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:32.336665  3431 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:23:32.359612  3567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838"
dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
mode: GRACEFUL
new_leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:49022
I20260709 12:23:32.360661  3567 raft_consensus.cc:606] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Received request to transfer leadership to TS 242920ef3fb14cbab2b9b913a5891b0b
I20260709 12:23:32.366735  3567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05"
dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
mode: GRACEFUL
new_leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:49022
I20260709 12:23:32.367268  3567 raft_consensus.cc:606] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Received request to transfer leadership to TS 242920ef3fb14cbab2b9b913a5891b0b
I20260709 12:23:32.372879  3714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d"
dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
mode: GRACEFUL
new_leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:50170
I20260709 12:23:32.373374  3714 raft_consensus.cc:606] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 3 LEADER]: Received request to transfer leadership to TS 242920ef3fb14cbab2b9b913a5891b0b
I20260709 12:23:32.378865  3567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51"
dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
mode: GRACEFUL
new_leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:49022
I20260709 12:23:32.379444  3567 raft_consensus.cc:606] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 LEADER]: Received request to transfer leadership to TS 242920ef3fb14cbab2b9b913a5891b0b
I20260709 12:23:32.392669  3714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350"
dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
mode: GRACEFUL
new_leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:50170
I20260709 12:23:32.393105  3714 raft_consensus.cc:606] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 LEADER]: Received request to transfer leadership to TS 242920ef3fb14cbab2b9b913a5891b0b
I20260709 12:23:32.560967  3883 raft_consensus.cc:993] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a: : Instructing follower 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:32.561478  3876 raft_consensus.cc:1081] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Signalling peer 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:32.562975  3642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:35794
I20260709 12:23:32.563534  3642 raft_consensus.cc:493] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:32.563880  3642 raft_consensus.cc:3060] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.569103  3642 raft_consensus.cc:515] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.571108  3642 leader_election.cc:290] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 3 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:32.571899  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:32.572331  3714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:32.572630  3567 raft_consensus.cc:3055] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:32.572896  3714 raft_consensus.cc:3060] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.572921  3567 raft_consensus.cc:740] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Running, Role: LEADER
I20260709 12:23:32.573896  3567 consensus_queue.cc:260] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.574858  3567 raft_consensus.cc:3060] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.577919  3714 raft_consensus.cc:2468] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 3.
I20260709 12:23:32.580129  3608 leader_election.cc:304] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:32.580572  3567 raft_consensus.cc:2468] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 3.
I20260709 12:23:32.581028  3766 raft_consensus.cc:2804] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:32.582216  3766 raft_consensus.cc:697] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 LEADER]: Becoming Leader. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:32.583407  3885 raft_consensus.cc:993] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733: : Instructing follower 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:32.583153  3766 consensus_queue.cc:237] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.583981  3871 raft_consensus.cc:1081] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 LEADER]: Signalling peer 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:32.585840  3642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:35792
I20260709 12:23:32.586313  3642 raft_consensus.cc:493] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:32.586659  3642 raft_consensus.cc:3060] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.591077  3463 catalog_manager.cc:5697] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b reported cstate change: term changed from 2 to 3, leader changed from ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193) to 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194). New cstate: current_term: 3 leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:32.591631  3642 raft_consensus.cc:515] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.593634  3642 leader_election.cc:290] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 3 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:32.594707  3714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:32.595289  3714 raft_consensus.cc:3055] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:32.595476  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:32.595608  3714 raft_consensus.cc:740] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:32.595988  3567 raft_consensus.cc:3060] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.596386  3714 consensus_queue.cc:260] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.597536  3714 raft_consensus.cc:3060] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.602638  3567 raft_consensus.cc:2468] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 3.
I20260709 12:23:32.603853  3607 leader_election.cc:304] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 
I20260709 12:23:32.603987  3714 raft_consensus.cc:2468] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 3.
I20260709 12:23:32.604686  3766 raft_consensus.cc:2804] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:32.605136  3766 raft_consensus.cc:697] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 LEADER]: Becoming Leader. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:32.605913  3766 consensus_queue.cc:237] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.614030  3463 catalog_manager.cc:5697] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b reported cstate change: term changed from 2 to 3, leader changed from b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195) to 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194). New cstate: current_term: 3 leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:32.667685  3883 raft_consensus.cc:993] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a: : Instructing follower 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:32.668179  3876 raft_consensus.cc:1081] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Signalling peer 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:32.669862  3642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:35794
I20260709 12:23:32.670373  3642 raft_consensus.cc:493] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:32.670692  3642 raft_consensus.cc:3060] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.675274  3642 raft_consensus.cc:515] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.677165  3642 leader_election.cc:290] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 3 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:32.678051  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:32.678278  3714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4124039aef754341ad4ac25df7cc1c05" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:32.678630  3567 raft_consensus.cc:3055] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:32.678716  3714 raft_consensus.cc:3060] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.679121  3567 raft_consensus.cc:740] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Running, Role: LEADER
I20260709 12:23:32.680116  3567 consensus_queue.cc:260] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.681255  3883 raft_consensus.cc:993] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a: : Instructing follower 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:32.681310  3567 raft_consensus.cc:3060] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:32.681814  3876 raft_consensus.cc:1081] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 LEADER]: Signalling peer 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:32.683209  3642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:35794
I20260709 12:23:32.683689  3642 raft_consensus.cc:493] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:32.684008  3642 raft_consensus.cc:3060] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:32.685179  3714 raft_consensus.cc:2468] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 3.
I20260709 12:23:32.686239  3608 leader_election.cc:304] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:32.686980  3766 raft_consensus.cc:2804] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:32.687615  3567 raft_consensus.cc:2468] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 3.
I20260709 12:23:32.687413  3766 raft_consensus.cc:697] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 LEADER]: Becoming Leader. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:32.688745  3766 consensus_queue.cc:237] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.689638  3642 raft_consensus.cc:515] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.691676  3642 leader_election.cc:290] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 2 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:32.692399  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:32.692560  3714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:32.693681  3567 raft_consensus.cc:3055] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:32.693853  3714 raft_consensus.cc:3060] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:32.694041  3567 raft_consensus.cc:740] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Running, Role: LEADER
I20260709 12:23:32.695031  3567 consensus_queue.cc:260] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.696461  3567 raft_consensus.cc:3060] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:32.701299  3714 raft_consensus.cc:2468] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 2.
I20260709 12:23:32.702057  3567 raft_consensus.cc:2468] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 2.
I20260709 12:23:32.702180  3608 leader_election.cc:304] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:32.702855  3766 raft_consensus.cc:2804] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:32.703249  3766 raft_consensus.cc:697] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Becoming Leader. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:32.702595  3463 catalog_manager.cc:5697] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b reported cstate change: term changed from 2 to 3, leader changed from ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193) to 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194). New cstate: current_term: 3 leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:32.703964  3766 consensus_queue.cc:237] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.714506  3463 catalog_manager.cc:5697] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b reported cstate change: term changed from 1 to 2, leader changed from ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193) to 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194). New cstate: current_term: 2 leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:32.772756  3885 raft_consensus.cc:993] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733: : Instructing follower 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:32.773201  3871 raft_consensus.cc:1081] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 3 LEADER]: Signalling peer 242920ef3fb14cbab2b9b913a5891b0b to start an election
I20260709 12:23:32.774653  3642 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
 from {username='slave'} at 127.0.0.1:35792
I20260709 12:23:32.775130  3642 raft_consensus.cc:493] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:32.775478  3642 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:32.779755  3642 raft_consensus.cc:515] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.781090  3642 leader_election.cc:290] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 4 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:32.781924  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:32.782106  3714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "242920ef3fb14cbab2b9b913a5891b0b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:32.782423  3567 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:32.782721  3714 raft_consensus.cc:3055] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:32.782982  3714 raft_consensus.cc:740] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:32.783612  3714 consensus_queue.cc:260] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.784415  3714 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:32.786597  3567 raft_consensus.cc:2468] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 4.
I20260709 12:23:32.787425  3607 leader_election.cc:304] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 
I20260709 12:23:32.788057  3766 raft_consensus.cc:2804] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:32.788398  3766 raft_consensus.cc:697] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 4 LEADER]: Becoming Leader. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:32.788878  3714 raft_consensus.cc:2468] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 242920ef3fb14cbab2b9b913a5891b0b in term 4.
I20260709 12:23:32.789068  3766 consensus_queue.cc:237] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:32.795449  3463 catalog_manager.cc:5697] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b reported cstate change: term changed from 3 to 4, leader changed from b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195) to 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194). New cstate: current_term: 4 leader_uuid: "242920ef3fb14cbab2b9b913a5891b0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:33.023420  3714 raft_consensus.cc:1275] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:33.024611  3766 consensus_queue.cc:1048] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:33.034229  3567 raft_consensus.cc:1275] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:33.035508  3766 consensus_queue.cc:1048] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:33.101634  3714 raft_consensus.cc:1275] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:33.102811  3766 consensus_queue.cc:1048] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:33.108448  3716 raft_consensus.cc:1275] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:33.110217  3766 consensus_queue.cc:1048] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:33.114539  3567 raft_consensus.cc:1275] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:33.116637  3901 consensus_queue.cc:1048] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:33.131712  3567 raft_consensus.cc:1275] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:33.133327  3902 consensus_queue.cc:1048] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:33.175321  3567 raft_consensus.cc:1275] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 4 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:33.176486  3902 consensus_queue.cc:1048] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:33.185333  3714 raft_consensus.cc:1275] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 4 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:33.187031  3901 consensus_queue.cc:1048] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:33.222455  3567 raft_consensus.cc:1275] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:33.223658  3901 consensus_queue.cc:1048] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:33.231900  3714 raft_consensus.cc:1275] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Refusing update from remote peer 242920ef3fb14cbab2b9b913a5891b0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:33.233237  3901 consensus_queue.cc:1048] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:35.427610  3431 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:23:35.428403  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:35.435537  3642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
mode: GRACEFUL
new_leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
 from {username='slave'} at 127.0.0.1:35808
I20260709 12:23:35.436236  3642 raft_consensus.cc:606] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 4 LEADER]: Received request to transfer leadership to TS b13e9d12b0b0416a9e214826c4a1c733
I20260709 12:23:35.439059  3642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
mode: GRACEFUL
new_leader_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
 from {username='slave'} at 127.0.0.1:35808
I20260709 12:23:35.439555  3642 raft_consensus.cc:606] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 LEADER]: Received request to transfer leadership to TS ae0b4bb075da417d9154ab6a1fcbd36a
I20260709 12:23:35.441715  3642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99a3567afee845fd82bd52553b4d236f"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
mode: GRACEFUL
new_leader_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
 from {username='slave'} at 127.0.0.1:35808
I20260709 12:23:35.442143  3642 raft_consensus.cc:606] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Received request to transfer leadership to TS ae0b4bb075da417d9154ab6a1fcbd36a
I20260709 12:23:35.444486  3642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
mode: GRACEFUL
new_leader_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
 from {username='slave'} at 127.0.0.1:35808
I20260709 12:23:35.444970  3642 raft_consensus.cc:606] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 LEADER]: Received request to transfer leadership to TS ae0b4bb075da417d9154ab6a1fcbd36a
I20260709 12:23:35.447428  3642 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51"
dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
mode: GRACEFUL
new_leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
 from {username='slave'} at 127.0.0.1:35808
I20260709 12:23:35.447913  3642 raft_consensus.cc:606] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Received request to transfer leadership to TS b13e9d12b0b0416a9e214826c4a1c733
I20260709 12:23:35.449079  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:35.449560  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:35.499045  3902 raft_consensus.cc:993] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b: : Instructing follower b13e9d12b0b0416a9e214826c4a1c733 to start an election
I20260709 12:23:35.499754  3912 raft_consensus.cc:1081] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Signalling peer b13e9d12b0b0416a9e214826c4a1c733 to start an election
I20260709 12:23:35.501652  3714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51"
dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
 from {username='slave'} at 127.0.0.1:50154
I20260709 12:23:35.502135  3714 raft_consensus.cc:493] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:35.502440  3714 raft_consensus.cc:3060] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.508049  3714 raft_consensus.cc:515] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.510800  3714 leader_election.cc:290] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 3 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:35.511380  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:35.511353  3642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa991ca904d84c5980cbb5e5859c0c51" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:35.512470  3567 raft_consensus.cc:3060] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.512528  3642 raft_consensus.cc:3055] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:35.513011  3642 raft_consensus.cc:740] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:35.513990  3642 consensus_queue.cc:260] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.515277  3642 raft_consensus.cc:3060] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.519974  3567 raft_consensus.cc:2468] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 3.
I20260709 12:23:35.521355  3681 leader_election.cc:304] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:35.522037  3642 raft_consensus.cc:2468] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 3.
I20260709 12:23:35.522289  3885 raft_consensus.cc:2804] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:35.523576  3885 raft_consensus.cc:697] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 LEADER]: Becoming Leader. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:35.524426  3885 consensus_queue.cc:237] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.531754  3461 catalog_manager.cc:5697] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 reported cstate change: term changed from 2 to 3, leader changed from 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194) to b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:35.681188  3902 raft_consensus.cc:993] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b: : Instructing follower ae0b4bb075da417d9154ab6a1fcbd36a to start an election
I20260709 12:23:35.681730  3912 raft_consensus.cc:1081] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 LEADER]: Signalling peer ae0b4bb075da417d9154ab6a1fcbd36a to start an election
I20260709 12:23:35.683367  3567 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350"
dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
 from {username='slave'} at 127.0.0.1:59376
I20260709 12:23:35.683851  3567 raft_consensus.cc:493] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:35.684137  3567 raft_consensus.cc:3060] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.688544  3567 raft_consensus.cc:515] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.690442  3567 leader_election.cc:290] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 4 election: Requested vote from peers 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:35.691425  3642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:35.691723  3714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b8b062b65ee465aa4b2b603c1a73350" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:35.691994  3642 raft_consensus.cc:3055] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:35.692344  3642 raft_consensus.cc:740] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:35.692369  3714 raft_consensus.cc:3060] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.693063  3642 consensus_queue.cc:260] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.693955  3642 raft_consensus.cc:3060] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.697713  3714 raft_consensus.cc:2468] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 4.
I20260709 12:23:35.698277  3642 raft_consensus.cc:2468] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 4.
I20260709 12:23:35.699306  3532 leader_election.cc:304] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:35.701315  3920 raft_consensus.cc:2804] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:35.702463  3920 raft_consensus.cc:697] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 4 LEADER]: Becoming Leader. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Running, Role: LEADER
I20260709 12:23:35.703339  3920 consensus_queue.cc:237] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.710597  3461 catalog_manager.cc:5697] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a reported cstate change: term changed from 3 to 4, leader changed from 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194) to ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193). New cstate: current_term: 4 leader_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:35.823830  3902 raft_consensus.cc:993] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b: : Instructing follower b13e9d12b0b0416a9e214826c4a1c733 to start an election
I20260709 12:23:35.824414  3912 raft_consensus.cc:1081] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 4 LEADER]: Signalling peer b13e9d12b0b0416a9e214826c4a1c733 to start an election
I20260709 12:23:35.825857  3714 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d"
dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
 from {username='slave'} at 127.0.0.1:50154
I20260709 12:23:35.826243  3714 raft_consensus.cc:493] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:35.826742  3714 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:35.831161  3714 raft_consensus.cc:515] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.832767  3714 leader_election.cc:290] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 5 election: Requested vote from peers ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693), 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141)
I20260709 12:23:35.833702  3567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
I20260709 12:23:35.834306  3567 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:35.834182  3642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c2aa61069e4b4c9585861f9061a78d" candidate_uuid: "b13e9d12b0b0416a9e214826c4a1c733" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:35.834782  3642 raft_consensus.cc:3055] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:23:35.835083  3642 raft_consensus.cc:740] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:35.836066  3642 consensus_queue.cc:260] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.837291  3642 raft_consensus.cc:3060] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:23:35.841454  3567 raft_consensus.cc:2468] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 5.
I20260709 12:23:35.842947  3681 leader_election.cc:304] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:35.843847  3885 raft_consensus.cc:2804] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:23:35.844272  3642 raft_consensus.cc:2468] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13e9d12b0b0416a9e214826c4a1c733 in term 5.
I20260709 12:23:35.844306  3885 raft_consensus.cc:697] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 5 LEADER]: Becoming Leader. State: Replica: b13e9d12b0b0416a9e214826c4a1c733, State: Running, Role: LEADER
I20260709 12:23:35.845507  3885 consensus_queue.cc:237] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.853025  3461 catalog_manager.cc:5697] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 reported cstate change: term changed from 4 to 5, leader changed from 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194) to b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195). New cstate: current_term: 5 leader_uuid: "b13e9d12b0b0416a9e214826c4a1c733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:35.870179  3902 raft_consensus.cc:993] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b: : Instructing follower ae0b4bb075da417d9154ab6a1fcbd36a to start an election
I20260709 12:23:35.870682  3912 raft_consensus.cc:1081] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 LEADER]: Signalling peer ae0b4bb075da417d9154ab6a1fcbd36a to start an election
I20260709 12:23:35.872224  3567 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838"
dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
 from {username='slave'} at 127.0.0.1:59376
I20260709 12:23:35.872663  3567 raft_consensus.cc:493] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:35.872918  3567 raft_consensus.cc:3060] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.878343  3567 raft_consensus.cc:515] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.880046  3567 leader_election.cc:290] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 4 election: Requested vote from peers 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:35.881022  3642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:35.881022  3714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eafd3f6dfac344f08870c9fdc9d95838" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:35.881899  3642 raft_consensus.cc:3055] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:35.881910  3714 raft_consensus.cc:3060] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.882407  3642 raft_consensus.cc:740] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:35.883258  3642 consensus_queue.cc:260] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.884433  3642 raft_consensus.cc:3060] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.889117  3714 raft_consensus.cc:2468] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 4.
I20260709 12:23:35.890399  3532 leader_election.cc:304] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0b4bb075da417d9154ab6a1fcbd36a, b13e9d12b0b0416a9e214826c4a1c733; no voters: 
I20260709 12:23:35.891199  3642 raft_consensus.cc:2468] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 4.
I20260709 12:23:35.891307  3920 raft_consensus.cc:2804] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:35.891917  3920 raft_consensus.cc:697] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 4 LEADER]: Becoming Leader. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Running, Role: LEADER
I20260709 12:23:35.892839  3920 consensus_queue.cc:237] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.899633  3461 catalog_manager.cc:5697] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a reported cstate change: term changed from 3 to 4, leader changed from 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194) to ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193). New cstate: current_term: 4 leader_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:35.947304  3902 raft_consensus.cc:993] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b: : Instructing follower ae0b4bb075da417d9154ab6a1fcbd36a to start an election
I20260709 12:23:35.947446  3642 raft_consensus.cc:1275] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Refusing update from remote peer b13e9d12b0b0416a9e214826c4a1c733: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:35.947903  3912 raft_consensus.cc:1081] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Signalling peer ae0b4bb075da417d9154ab6a1fcbd36a to start an election
I20260709 12:23:35.949038  3885 consensus_queue.cc:1048] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:35.949745  3567 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99a3567afee845fd82bd52553b4d236f"
dest_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a"
 from {username='slave'} at 127.0.0.1:59376
I20260709 12:23:35.950264  3567 raft_consensus.cc:493] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:35.950634  3567 raft_consensus.cc:3060] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.957103  3567 raft_consensus.cc:515] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.961623  3639 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a3567afee845fd82bd52553b4d236f" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "242920ef3fb14cbab2b9b913a5891b0b"
I20260709 12:23:35.962277  3639 raft_consensus.cc:3055] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:35.962594  3639 raft_consensus.cc:740] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 242920ef3fb14cbab2b9b913a5891b0b, State: Running, Role: LEADER
I20260709 12:23:35.962769  3714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99a3567afee845fd82bd52553b4d236f" candidate_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b13e9d12b0b0416a9e214826c4a1c733"
I20260709 12:23:35.963466  3639 consensus_queue.cc:260] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.964529  3564 raft_consensus.cc:1275] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Refusing update from remote peer b13e9d12b0b0416a9e214826c4a1c733: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:35.964709  3639 raft_consensus.cc:3060] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.967125  3567 leader_election.cc:290] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 3 election: Requested vote from peers 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141), b13e9d12b0b0416a9e214826c4a1c733 (127.3.89.195:45347)
I20260709 12:23:35.967862  3714 raft_consensus.cc:3060] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.967545  3927 consensus_queue.cc:1048] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:35.974437  3639 raft_consensus.cc:2468] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 3.
I20260709 12:23:35.975835  3530 leader_election.cc:304] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 242920ef3fb14cbab2b9b913a5891b0b, ae0b4bb075da417d9154ab6a1fcbd36a; no voters: 
I20260709 12:23:35.977324  3714 raft_consensus.cc:2468] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0b4bb075da417d9154ab6a1fcbd36a in term 3.
I20260709 12:23:35.977469  3920 raft_consensus.cc:2804] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:35.978667  3920 raft_consensus.cc:697] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 LEADER]: Becoming Leader. State: Replica: ae0b4bb075da417d9154ab6a1fcbd36a, State: Running, Role: LEADER
I20260709 12:23:35.979558  3920 consensus_queue.cc:237] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } }
I20260709 12:23:35.989827  3462 catalog_manager.cc:5697] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a reported cstate change: term changed from 2 to 3, leader changed from 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194) to ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193). New cstate: current_term: 3 leader_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:36.164462  3639 raft_consensus.cc:1275] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 4 FOLLOWER]: Refusing update from remote peer ae0b4bb075da417d9154ab6a1fcbd36a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:36.165832  3920 consensus_queue.cc:1048] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:36.176299  3714 raft_consensus.cc:1275] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 4 FOLLOWER]: Refusing update from remote peer ae0b4bb075da417d9154ab6a1fcbd36a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:36.177901  3933 consensus_queue.cc:1048] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:36.296329  3714 raft_consensus.cc:1275] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 4 FOLLOWER]: Refusing update from remote peer ae0b4bb075da417d9154ab6a1fcbd36a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:36.297606  3933 consensus_queue.cc:1048] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:36.307550  3639 raft_consensus.cc:1275] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 4 FOLLOWER]: Refusing update from remote peer ae0b4bb075da417d9154ab6a1fcbd36a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:36.308996  3933 consensus_queue.cc:1048] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:36.396759  3714 raft_consensus.cc:1275] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Refusing update from remote peer ae0b4bb075da417d9154ab6a1fcbd36a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:36.397897  3933 consensus_queue.cc:1048] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b13e9d12b0b0416a9e214826c4a1c733" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:36.408142  3639 raft_consensus.cc:1275] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Refusing update from remote peer ae0b4bb075da417d9154ab6a1fcbd36a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:36.410295  3921 consensus_queue.cc:1048] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:36.426627  3567 raft_consensus.cc:1275] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 5 FOLLOWER]: Refusing update from remote peer b13e9d12b0b0416a9e214826c4a1c733: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:23:36.427932  3927 consensus_queue.cc:1048] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0b4bb075da417d9154ab6a1fcbd36a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:36.436638  3639 raft_consensus.cc:1275] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 5 FOLLOWER]: Refusing update from remote peer b13e9d12b0b0416a9e214826c4a1c733: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:23:36.438238  3940 consensus_queue.cc:1048] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "242920ef3fb14cbab2b9b913a5891b0b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:36.450695  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:36.456173  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:36.456604  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:37.457695  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:37.462690  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:37.463128  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:38.464098  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:38.469077  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:38.469542  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:39.470685  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:39.475548  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:39.476012  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:40.477224  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:40.483496  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:40.483947  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:41.484961  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:41.490149  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:41.490559  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:42.491585  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:42.496539  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:42.496958  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:43.497974  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:43.502964  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:43.503489  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:44.504530  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:44.509315  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:44.509750  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:45.542059  3431 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:23:45.542682  3431 tablet_server.cc:179] TabletServer@127.3.89.193:0 shutting down...
I20260709 12:23:45.572588  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:45.573485  3431 tablet_replica.cc:333] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a: stopping tablet replica
I20260709 12:23:45.574399  3431 raft_consensus.cc:2243] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.575119  3431 raft_consensus.cc:2272] T 34c2aa61069e4b4c9585861f9061a78d P ae0b4bb075da417d9154ab6a1fcbd36a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.577689  3431 tablet_replica.cc:333] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a: stopping tablet replica
I20260709 12:23:45.578189  3431 raft_consensus.cc:2243] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.578748  3431 raft_consensus.cc:2272] T fa991ca904d84c5980cbb5e5859c0c51 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.580654  3431 tablet_replica.cc:333] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a: stopping tablet replica
I20260709 12:23:45.581212  3431 raft_consensus.cc:2243] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:23:45.582249  3431 raft_consensus.cc:2272] T 4b8b062b65ee465aa4b2b603c1a73350 P ae0b4bb075da417d9154ab6a1fcbd36a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.584263  3431 tablet_replica.cc:333] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a: stopping tablet replica
I20260709 12:23:45.584865  3431 raft_consensus.cc:2243] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.585361  3431 raft_consensus.cc:2272] T 914a0780eb754f62a3c0221b0c47c2f7 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.587505  3431 tablet_replica.cc:333] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a: stopping tablet replica
I20260709 12:23:45.588065  3431 raft_consensus.cc:2243] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.588558  3431 raft_consensus.cc:2272] T 19f46c4700004a08bf63901dae2c7152 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.590571  3431 tablet_replica.cc:333] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a: stopping tablet replica
I20260709 12:23:45.591126  3431 raft_consensus.cc:2243] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:23:45.592137  3431 raft_consensus.cc:2272] T eafd3f6dfac344f08870c9fdc9d95838 P ae0b4bb075da417d9154ab6a1fcbd36a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.594126  3431 tablet_replica.cc:333] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a: stopping tablet replica
I20260709 12:23:45.594587  3431 raft_consensus.cc:2243] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.595039  3431 raft_consensus.cc:2272] T 4124039aef754341ad4ac25df7cc1c05 P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.596802  3431 tablet_replica.cc:333] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a: stopping tablet replica
I20260709 12:23:45.597342  3431 raft_consensus.cc:2243] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:45.598206  3431 raft_consensus.cc:2272] T 99a3567afee845fd82bd52553b4d236f P ae0b4bb075da417d9154ab6a1fcbd36a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.600234  3431 tablet_replica.cc:333] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a: stopping tablet replica
I20260709 12:23:45.600756  3431 raft_consensus.cc:2243] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.601362  3431 raft_consensus.cc:2272] T e732f8f7ac884e1d987d039cfa748f14 P ae0b4bb075da417d9154ab6a1fcbd36a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:45.752696  3607 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.193:41693: connect: Connection refused (error 111)
W20260709 12:23:45.757519  3681 consensus_peers.cc:597] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 -> Peer ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693): Couldn't send request to peer ae0b4bb075da417d9154ab6a1fcbd36a. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:41693: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:45.758293  3607 consensus_peers.cc:597] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b -> Peer ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693): Couldn't send request to peer ae0b4bb075da417d9154ab6a1fcbd36a. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:41693: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:45.758680  3607 consensus_peers.cc:597] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b -> Peer ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693): Couldn't send request to peer ae0b4bb075da417d9154ab6a1fcbd36a. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:41693: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:45.775691  3431 tablet_server.cc:196] TabletServer@127.3.89.193:0 shutdown complete.
I20260709 12:23:45.796200  3431 tablet_server.cc:179] TabletServer@127.3.89.194:0 shutting down...
I20260709 12:23:45.826830  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:45.827762  3431 tablet_replica.cc:333] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b: stopping tablet replica
I20260709 12:23:45.828478  3431 raft_consensus.cc:2243] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.828961  3431 raft_consensus.cc:2272] T e732f8f7ac884e1d987d039cfa748f14 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.831045  3431 tablet_replica.cc:333] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b: stopping tablet replica
I20260709 12:23:45.831588  3431 raft_consensus.cc:2243] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.832000  3431 raft_consensus.cc:2272] T 4b8b062b65ee465aa4b2b603c1a73350 P 242920ef3fb14cbab2b9b913a5891b0b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.834263  3431 tablet_replica.cc:333] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b: stopping tablet replica
I20260709 12:23:45.835114  3431 raft_consensus.cc:2243] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:45.835953  3431 raft_consensus.cc:2272] T 914a0780eb754f62a3c0221b0c47c2f7 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.837898  3431 tablet_replica.cc:333] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b: stopping tablet replica
I20260709 12:23:45.838367  3431 raft_consensus.cc:2243] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.838728  3431 raft_consensus.cc:2272] T 34c2aa61069e4b4c9585861f9061a78d P 242920ef3fb14cbab2b9b913a5891b0b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.840215  3431 tablet_replica.cc:333] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b: stopping tablet replica
I20260709 12:23:45.840816  3431 raft_consensus.cc:2243] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.841187  3431 raft_consensus.cc:2272] T eafd3f6dfac344f08870c9fdc9d95838 P 242920ef3fb14cbab2b9b913a5891b0b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.842871  3431 tablet_replica.cc:333] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b: stopping tablet replica
I20260709 12:23:45.843307  3431 raft_consensus.cc:2243] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:45.843999  3431 raft_consensus.cc:2272] T 19f46c4700004a08bf63901dae2c7152 P 242920ef3fb14cbab2b9b913a5891b0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.845628  3431 tablet_replica.cc:333] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b: stopping tablet replica
I20260709 12:23:45.846065  3431 raft_consensus.cc:2243] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.846411  3431 raft_consensus.cc:2272] T fa991ca904d84c5980cbb5e5859c0c51 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.848048  3431 tablet_replica.cc:333] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b: stopping tablet replica
I20260709 12:23:45.848515  3431 raft_consensus.cc:2243] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.848987  3431 raft_consensus.cc:2272] T 99a3567afee845fd82bd52553b4d236f P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.850528  3431 tablet_replica.cc:333] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b: stopping tablet replica
I20260709 12:23:45.850971  3431 raft_consensus.cc:2243] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:45.851718  3431 raft_consensus.cc:2272] T 4124039aef754341ad4ac25df7cc1c05 P 242920ef3fb14cbab2b9b913a5891b0b [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:45.862177  3680 consensus_peers.cc:597] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 -> Peer 242920ef3fb14cbab2b9b913a5891b0b (127.3.89.194:40141): Couldn't send request to peer 242920ef3fb14cbab2b9b913a5891b0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:45.881453  3431 tablet_server.cc:196] TabletServer@127.3.89.194:0 shutdown complete.
I20260709 12:23:45.899921  3431 tablet_server.cc:179] TabletServer@127.3.89.195:0 shutting down...
I20260709 12:23:45.927803  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:45.928577  3431 tablet_replica.cc:333] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733: stopping tablet replica
I20260709 12:23:45.929618  3431 raft_consensus.cc:2243] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:45.930644  3431 raft_consensus.cc:2272] T fa991ca904d84c5980cbb5e5859c0c51 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.932502  3431 tablet_replica.cc:333] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733: stopping tablet replica
I20260709 12:23:45.933236  3431 raft_consensus.cc:2243] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:23:45.934294  3431 raft_consensus.cc:2272] T 34c2aa61069e4b4c9585861f9061a78d P b13e9d12b0b0416a9e214826c4a1c733 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.936444  3431 tablet_replica.cc:333] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733: stopping tablet replica
I20260709 12:23:45.937166  3431 raft_consensus.cc:2243] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:23:45.937160  3681 consensus_peers.cc:597] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 -> Peer ae0b4bb075da417d9154ab6a1fcbd36a (127.3.89.193:41693): Couldn't send request to peer ae0b4bb075da417d9154ab6a1fcbd36a. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:41693: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:45.938088  3431 raft_consensus.cc:2272] T e732f8f7ac884e1d987d039cfa748f14 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.939792  3431 tablet_replica.cc:333] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733: stopping tablet replica
I20260709 12:23:45.940241  3431 raft_consensus.cc:2243] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.940694  3431 raft_consensus.cc:2272] T 914a0780eb754f62a3c0221b0c47c2f7 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.942296  3431 tablet_replica.cc:333] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733: stopping tablet replica
I20260709 12:23:45.942715  3431 raft_consensus.cc:2243] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.943109  3431 raft_consensus.cc:2272] T 4b8b062b65ee465aa4b2b603c1a73350 P b13e9d12b0b0416a9e214826c4a1c733 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.944725  3431 tablet_replica.cc:333] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733: stopping tablet replica
I20260709 12:23:45.945158  3431 raft_consensus.cc:2243] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.945638  3431 raft_consensus.cc:2272] T eafd3f6dfac344f08870c9fdc9d95838 P b13e9d12b0b0416a9e214826c4a1c733 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.947124  3431 tablet_replica.cc:333] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733: stopping tablet replica
I20260709 12:23:45.947530  3431 raft_consensus.cc:2243] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.947906  3431 raft_consensus.cc:2272] T 4124039aef754341ad4ac25df7cc1c05 P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.949470  3431 tablet_replica.cc:333] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733: stopping tablet replica
I20260709 12:23:45.949980  3431 raft_consensus.cc:2243] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.950497  3431 raft_consensus.cc:2272] T 99a3567afee845fd82bd52553b4d236f P b13e9d12b0b0416a9e214826c4a1c733 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.952035  3431 tablet_replica.cc:333] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733: stopping tablet replica
I20260709 12:23:45.952417  3431 raft_consensus.cc:2243] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:45.952764  3431 raft_consensus.cc:2272] T 19f46c4700004a08bf63901dae2c7152 P b13e9d12b0b0416a9e214826c4a1c733 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:45.979838  3431 tablet_server.cc:196] TabletServer@127.3.89.195:0 shutdown complete.
I20260709 12:23:45.998389  3431 master.cc:562] Master@127.3.89.254:38703 shutting down...
I20260709 12:23:46.128206  3431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:46.128950  3431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:46.129313  3431 tablet_replica.cc:333] T 00000000000000000000000000000000 P a72528fa9f62441eabfabeeec4a468db: stopping tablet replica
I20260709 12:23:46.149935  3431 master.cc:584] Master@127.3.89.254:38703 shutdown complete.
242920ef3fb14cbab2b9b913a5891b0b  4
ae0b4bb075da417d9154ab6a1fcbd36a  4
b13e9d12b0b0416a9e214826c4a1c733  1
242920ef3fb14cbab2b9b913a5891b0b  3
ae0b4bb075da417d9154ab6a1fcbd36a  3
b13e9d12b0b0416a9e214826c4a1c733  3
242920ef3fb14cbab2b9b913a5891b0b  8
ae0b4bb075da417d9154ab6a1fcbd36a  0
b13e9d12b0b0416a9e214826c4a1c733  1
242920ef3fb14cbab2b9b913a5891b0b  3
ae0b4bb075da417d9154ab6a1fcbd36a  3
b13e9d12b0b0416a9e214826c4a1c733  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (31111 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:23:46.194092  3431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.3.89.254:39031
I20260709 12:23:46.195698  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:46.202137  3955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:46.202280  3956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:46.203270  3431 server_base.cc:1061] running on GCE node
W20260709 12:23:46.203753  3958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:46.204635  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:46.204860  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:46.205025  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783599826205009 us; error 0 us; skew 500 ppm
I20260709 12:23:46.205560  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:46.208220  3431 webserver.cc:533] Webserver started at http://127.3.89.254:37571/ using document root <none> and password file <none>
I20260709 12:23:46.208752  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:46.208963  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:46.209245  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:46.210466  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/master-0-root/instance:
uuid: "3d73f714f61942ff89321862d15c8307"
format_stamp: "Formatted at 2026-07-09 12:23:46 on dist-test-slave-3hrd"
I20260709 12:23:46.215525  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:23:46.219251  3963 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:46.220144  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:23:46.220572  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/master-0-root
uuid: "3d73f714f61942ff89321862d15c8307"
format_stamp: "Formatted at 2026-07-09 12:23:46 on dist-test-slave-3hrd"
I20260709 12:23:46.220928  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:46.244266  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:46.245770  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:46.294900  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:39031
I20260709 12:23:46.294996  4014 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.254:39031 every 8 connection(s)
I20260709 12:23:46.299146  4015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.308723  4015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307: Bootstrap starting.
I20260709 12:23:46.313496  4015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:46.317948  4015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307: No bootstrap required, opened a new log
I20260709 12:23:46.320386  4015 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d73f714f61942ff89321862d15c8307" member_type: VOTER }
I20260709 12:23:46.320890  4015 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:46.321132  4015 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d73f714f61942ff89321862d15c8307, State: Initialized, Role: FOLLOWER
I20260709 12:23:46.321831  4015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d73f714f61942ff89321862d15c8307" member_type: VOTER }
I20260709 12:23:46.322371  4015 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:23:46.322641  4015 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:23:46.322952  4015 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:46.327598  4015 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d73f714f61942ff89321862d15c8307" member_type: VOTER }
I20260709 12:23:46.328243  4015 leader_election.cc:304] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d73f714f61942ff89321862d15c8307; no voters: 
I20260709 12:23:46.329519  4015 leader_election.cc:290] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:23:46.329991  4018 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:46.331447  4018 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [term 1 LEADER]: Becoming Leader. State: Replica: 3d73f714f61942ff89321862d15c8307, State: Running, Role: LEADER
I20260709 12:23:46.332207  4018 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d73f714f61942ff89321862d15c8307" member_type: VOTER }
I20260709 12:23:46.332918  4015 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:23:46.337481  4019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d73f714f61942ff89321862d15c8307" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d73f714f61942ff89321862d15c8307" member_type: VOTER } }
I20260709 12:23:46.338358  4019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:46.337988  4020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d73f714f61942ff89321862d15c8307. Latest consensus state: current_term: 1 leader_uuid: "3d73f714f61942ff89321862d15c8307" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d73f714f61942ff89321862d15c8307" member_type: VOTER } }
I20260709 12:23:46.338940  4020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:46.343457  4025 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:23:46.348585  4025 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:23:46.350772  3431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:23:46.358740  4025 catalog_manager.cc:1383] Generated new cluster ID: e5167c83fbd94b85b74a2aecca654293
I20260709 12:23:46.359093  4025 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:23:46.386991  4025 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:23:46.388204  4025 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:23:46.409920  4025 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307: Generated new TSK 0
I20260709 12:23:46.410550  4025 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:23:46.417227  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:46.423497  4037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:46.424619  4038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:46.425853  4040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:46.426013  3431 server_base.cc:1061] running on GCE node
I20260709 12:23:46.427013  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:46.427187  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:46.427320  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783599826427306 us; error 0 us; skew 500 ppm
I20260709 12:23:46.427783  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:46.430173  3431 webserver.cc:533] Webserver started at http://127.3.89.193:45079/ using document root <none> and password file <none>
I20260709 12:23:46.430606  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:46.430764  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:46.431073  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:46.432142  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-0-root/instance:
uuid: "eb6356fa44624134819afb93643d63da"
format_stamp: "Formatted at 2026-07-09 12:23:46 on dist-test-slave-3hrd"
I20260709 12:23:46.436887  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:23:46.440678  4045 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:46.441552  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:23:46.441896  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-0-root
uuid: "eb6356fa44624134819afb93643d63da"
format_stamp: "Formatted at 2026-07-09 12:23:46 on dist-test-slave-3hrd"
I20260709 12:23:46.442145  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:46.454346  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:46.455672  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:46.457119  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:46.459393  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:46.459573  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:46.459750  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:46.459899  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:46.502162  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:32943
I20260709 12:23:46.502305  4107 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.193:32943 every 8 connection(s)
I20260709 12:23:46.506969  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:46.513935  4112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:46.514436  4113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:46.518477  4115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:46.518525  3431 server_base.cc:1061] running on GCE node
I20260709 12:23:46.520171  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:46.520399  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:46.520601  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783599826520582 us; error 0 us; skew 500 ppm
I20260709 12:23:46.521147  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:46.524295  3431 webserver.cc:533] Webserver started at http://127.3.89.194:37761/ using document root <none> and password file <none>
I20260709 12:23:46.524890  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:46.525082  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:46.525411  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:46.525988  4108 heartbeater.cc:344] Connected to a master server at 127.3.89.254:39031
I20260709 12:23:46.526396  4108 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:46.526985  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-1-root/instance:
uuid: "79d614bf4bc24974baa0c843f2db25c1"
format_stamp: "Formatted at 2026-07-09 12:23:46 on dist-test-slave-3hrd"
I20260709 12:23:46.527248  4108 heartbeater.cc:507] Master 127.3.89.254:39031 requested a full tablet report, sending...
I20260709 12:23:46.529770  3980 ts_manager.cc:194] Registered new tserver with Master: eb6356fa44624134819afb93643d63da (127.3.89.193:32943)
I20260709 12:23:46.532189  3980 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36602
I20260709 12:23:46.532943  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 12:23:46.536963  4120 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:46.537899  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:23:46.538208  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-1-root
uuid: "79d614bf4bc24974baa0c843f2db25c1"
format_stamp: "Formatted at 2026-07-09 12:23:46 on dist-test-slave-3hrd"
I20260709 12:23:46.538527  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:46.558037  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:46.559250  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:46.560734  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:46.563019  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:46.563261  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:46.563503  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:46.563674  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:46.604331  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:43941
I20260709 12:23:46.604487  4182 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.194:43941 every 8 connection(s)
I20260709 12:23:46.609117  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:46.616283  4186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:46.618614  4187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:46.621152  3431 server_base.cc:1061] running on GCE node
W20260709 12:23:46.621009  4189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:46.622786  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:23:46.622961  4183 heartbeater.cc:344] Connected to a master server at 127.3.89.254:39031
W20260709 12:23:46.623093  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:46.623340  4183 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:46.623314  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783599826623296 us; error 0 us; skew 500 ppm
I20260709 12:23:46.624202  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:46.624297  4183 heartbeater.cc:507] Master 127.3.89.254:39031 requested a full tablet report, sending...
I20260709 12:23:46.626545  3980 ts_manager.cc:194] Registered new tserver with Master: 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:46.627274  3431 webserver.cc:533] Webserver started at http://127.3.89.195:34801/ using document root <none> and password file <none>
I20260709 12:23:46.627787  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:46.627996  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:46.627949  3980 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36614
I20260709 12:23:46.628394  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:46.629784  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-2-root/instance:
uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
format_stamp: "Formatted at 2026-07-09 12:23:46 on dist-test-slave-3hrd"
I20260709 12:23:46.634727  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 12:23:46.638417  4194 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:46.639412  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:23:46.639761  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-2-root
uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
format_stamp: "Formatted at 2026-07-09 12:23:46 on dist-test-slave-3hrd"
I20260709 12:23:46.640064  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599794910850-3431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:46.658612  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:46.659832  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:46.661365  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:46.663636  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:46.663882  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:46.664144  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:46.664342  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:46.707166  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:41075
I20260709 12:23:46.707280  4257 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.195:41075 every 8 connection(s)
I20260709 12:23:46.722538  4258 heartbeater.cc:344] Connected to a master server at 127.3.89.254:39031
I20260709 12:23:46.722973  4258 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:46.723812  4258 heartbeater.cc:507] Master 127.3.89.254:39031 requested a full tablet report, sending...
I20260709 12:23:46.726001  3980 ts_manager.cc:194] Registered new tserver with Master: 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:46.726884  3431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016080673s
I20260709 12:23:46.727555  3980 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36618
I20260709 12:23:46.754031  3980 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36626:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:23:46.757074  3980 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:23:46.841367  4073 tablet_service.cc:1511] Processing CreateTablet for tablet f0c9ed3020c344279bcf9b4f9eea78cf (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:23:46.843045  4073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0c9ed3020c344279bcf9b4f9eea78cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.846307  4147 tablet_service.cc:1511] Processing CreateTablet for tablet f2fd419c8941460eb7f76177a854c216 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:23:46.846727  4148 tablet_service.cc:1511] Processing CreateTablet for tablet f0c9ed3020c344279bcf9b4f9eea78cf (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:23:46.846918  4072 tablet_service.cc:1511] Processing CreateTablet for tablet f2fd419c8941460eb7f76177a854c216 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:23:46.847795  4147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2fd419c8941460eb7f76177a854c216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.848464  4148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0c9ed3020c344279bcf9b4f9eea78cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.848500  4072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2fd419c8941460eb7f76177a854c216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.849588  4145 tablet_service.cc:1511] Processing CreateTablet for tablet 91bcf95b3c4a45708d2e7d3e0a4812ad (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:23:46.852319  4146 tablet_service.cc:1511] Processing CreateTablet for tablet e26b66edcf304a63b41d9e775bd86cec (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:23:46.850992  4145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91bcf95b3c4a45708d2e7d3e0a4812ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.853302  4070 tablet_service.cc:1511] Processing CreateTablet for tablet 91bcf95b3c4a45708d2e7d3e0a4812ad (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:23:46.854933  4070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91bcf95b3c4a45708d2e7d3e0a4812ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.856801  4146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e26b66edcf304a63b41d9e775bd86cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.856606  4144 tablet_service.cc:1511] Processing CreateTablet for tablet 974f32885718400cb4c83cf4a8689d3b (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:23:46.850890  4071 tablet_service.cc:1511] Processing CreateTablet for tablet e26b66edcf304a63b41d9e775bd86cec (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:23:46.857964  4144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 974f32885718400cb4c83cf4a8689d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.858844  4071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e26b66edcf304a63b41d9e775bd86cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.865236  4143 tablet_service.cc:1511] Processing CreateTablet for tablet bb76f72e365e4f28a86a3c92ed34d243 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:23:46.867112  4143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb76f72e365e4f28a86a3c92ed34d243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.871376  4068 tablet_service.cc:1511] Processing CreateTablet for tablet bb76f72e365e4f28a86a3c92ed34d243 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:23:46.868984  4069 tablet_service.cc:1511] Processing CreateTablet for tablet 974f32885718400cb4c83cf4a8689d3b (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:23:46.873404  4068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb76f72e365e4f28a86a3c92ed34d243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.875602  4069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 974f32885718400cb4c83cf4a8689d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.880440  4223 tablet_service.cc:1511] Processing CreateTablet for tablet f0c9ed3020c344279bcf9b4f9eea78cf (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:23:46.883109  4223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0c9ed3020c344279bcf9b4f9eea78cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.885875  4222 tablet_service.cc:1511] Processing CreateTablet for tablet f2fd419c8941460eb7f76177a854c216 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:23:46.887645  4222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2fd419c8941460eb7f76177a854c216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.891189  4221 tablet_service.cc:1511] Processing CreateTablet for tablet e26b66edcf304a63b41d9e775bd86cec (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:23:46.893268  4221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e26b66edcf304a63b41d9e775bd86cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.909989  4067 tablet_service.cc:1511] Processing CreateTablet for tablet ea8b49c57549415cba40427c4cb553e7 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:23:46.911574  4067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea8b49c57549415cba40427c4cb553e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.915948  4142 tablet_service.cc:1511] Processing CreateTablet for tablet ea8b49c57549415cba40427c4cb553e7 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:23:46.917443  4142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea8b49c57549415cba40427c4cb553e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.921919  4141 tablet_service.cc:1511] Processing CreateTablet for tablet 272c43daddc341079f8a8af9c2f69e44 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:23:46.923691  4141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 272c43daddc341079f8a8af9c2f69e44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.924059  4066 tablet_service.cc:1511] Processing CreateTablet for tablet 272c43daddc341079f8a8af9c2f69e44 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:23:46.926267  4066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 272c43daddc341079f8a8af9c2f69e44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.945623  4140 tablet_service.cc:1511] Processing CreateTablet for tablet 3e3a70bf60d1483cbfadcdeae5b564a6 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:23:46.947129  4140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3a70bf60d1483cbfadcdeae5b564a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.947504  4065 tablet_service.cc:1511] Processing CreateTablet for tablet 3e3a70bf60d1483cbfadcdeae5b564a6 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:23:46.949003  4065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3a70bf60d1483cbfadcdeae5b564a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.971432  4064 tablet_service.cc:1511] Processing CreateTablet for tablet e26f47e7ee0346538108cf9b1350bd5d (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:23:46.973481  4064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e26f47e7ee0346538108cf9b1350bd5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.978240  4141 tablet_service.cc:1511] Processing CreateTablet for tablet e26f47e7ee0346538108cf9b1350bd5d (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:23:46.978272  4218 tablet_service.cc:1511] Processing CreateTablet for tablet bb76f72e365e4f28a86a3c92ed34d243 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:23:46.979724  4141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e26f47e7ee0346538108cf9b1350bd5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.979933  4218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb76f72e365e4f28a86a3c92ed34d243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.975615  4219 tablet_service.cc:1511] Processing CreateTablet for tablet 974f32885718400cb4c83cf4a8689d3b (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:23:46.982429  4219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 974f32885718400cb4c83cf4a8689d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.987522  4217 tablet_service.cc:1511] Processing CreateTablet for tablet ea8b49c57549415cba40427c4cb553e7 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:23:46.989084  4217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea8b49c57549415cba40427c4cb553e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.975497  4220 tablet_service.cc:1511] Processing CreateTablet for tablet 91bcf95b3c4a45708d2e7d3e0a4812ad (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:23:46.990841  4220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91bcf95b3c4a45708d2e7d3e0a4812ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:46.999547  4216 tablet_service.cc:1511] Processing CreateTablet for tablet 272c43daddc341079f8a8af9c2f69e44 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:23:47.001108  4216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 272c43daddc341079f8a8af9c2f69e44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:47.004554  4215 tablet_service.cc:1511] Processing CreateTablet for tablet 3e3a70bf60d1483cbfadcdeae5b564a6 (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:23:47.006246  4215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3a70bf60d1483cbfadcdeae5b564a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:47.010010  4214 tablet_service.cc:1511] Processing CreateTablet for tablet e26f47e7ee0346538108cf9b1350bd5d (DEFAULT_TABLE table=test-workload [id=8140dbfbd01a4a548fe65d6c4d28634b]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:23:47.011754  4214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e26f47e7ee0346538108cf9b1350bd5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:47.093863  4274 tablet_bootstrap.cc:492] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da: Bootstrap starting.
I20260709 12:23:47.095291  4273 tablet_bootstrap.cc:492] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1: Bootstrap starting.
I20260709 12:23:47.096289  4275 tablet_bootstrap.cc:492] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6: Bootstrap starting.
I20260709 12:23:47.101680  4275 tablet_bootstrap.cc:654] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.102907  4274 tablet_bootstrap.cc:654] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.102926  4273 tablet_bootstrap.cc:654] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.107712  4275 tablet_bootstrap.cc:492] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6: No bootstrap required, opened a new log
I20260709 12:23:47.108234  4275 ts_tablet_manager.cc:1403] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.003s
I20260709 12:23:47.110504  4274 tablet_bootstrap.cc:492] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da: No bootstrap required, opened a new log
I20260709 12:23:47.111029  4274 ts_tablet_manager.cc:1403] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 12:23:47.111306  4273 tablet_bootstrap.cc:492] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1: No bootstrap required, opened a new log
I20260709 12:23:47.111089  4275 raft_consensus.cc:359] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.111876  4273 ts_tablet_manager.cc:1403] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1: Time spent bootstrapping tablet: real 0.017s	user 0.004s	sys 0.010s
I20260709 12:23:47.111944  4275 raft_consensus.cc:385] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.112402  4275 raft_consensus.cc:740] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.113157  4275 consensus_queue.cc:260] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.114308  4274 raft_consensus.cc:359] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.115125  4274 raft_consensus.cc:385] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.115953  4275 ts_tablet_manager.cc:1434] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:47.115723  4258 heartbeater.cc:499] Master 127.3.89.254:39031 was elected leader, sending a full tablet report...
I20260709 12:23:47.115099  4273 raft_consensus.cc:359] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.118211  4274 raft_consensus.cc:740] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.118497  4273 raft_consensus.cc:385] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.118868  4273 raft_consensus.cc:740] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.119073  4275 tablet_bootstrap.cc:492] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6: Bootstrap starting.
I20260709 12:23:47.119194  4274 consensus_queue.cc:260] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.119840  4273 consensus_queue.cc:260] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.124927  4274 ts_tablet_manager.cc:1434] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.007s
I20260709 12:23:47.125133  4108 heartbeater.cc:499] Master 127.3.89.254:39031 was elected leader, sending a full tablet report...
I20260709 12:23:47.125640  4273 ts_tablet_manager.cc:1434] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.007s
I20260709 12:23:47.126024  4274 tablet_bootstrap.cc:492] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da: Bootstrap starting.
I20260709 12:23:47.126709  4273 tablet_bootstrap.cc:492] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1: Bootstrap starting.
I20260709 12:23:47.129897  4183 heartbeater.cc:499] Master 127.3.89.254:39031 was elected leader, sending a full tablet report...
I20260709 12:23:47.133064  4274 tablet_bootstrap.cc:654] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.133302  4273 tablet_bootstrap.cc:654] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.144173  4274 tablet_bootstrap.cc:492] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da: No bootstrap required, opened a new log
I20260709 12:23:47.144665  4273 tablet_bootstrap.cc:492] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1: No bootstrap required, opened a new log
I20260709 12:23:47.144718  4274 ts_tablet_manager.cc:1403] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.001s
I20260709 12:23:47.145453  4273 ts_tablet_manager.cc:1403] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.007s
I20260709 12:23:47.148280  4274 raft_consensus.cc:359] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.149154  4274 raft_consensus.cc:385] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.148708  4273 raft_consensus.cc:359] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.149466  4275 tablet_bootstrap.cc:654] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.149497  4274 raft_consensus.cc:740] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.149711  4273 raft_consensus.cc:385] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.150352  4273 raft_consensus.cc:740] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.150844  4274 consensus_queue.cc:260] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.151356  4273 consensus_queue.cc:260] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.153366  4274 ts_tablet_manager.cc:1434] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:23:47.154490  4274 tablet_bootstrap.cc:492] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da: Bootstrap starting.
I20260709 12:23:47.154660  4273 ts_tablet_manager.cc:1434] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:23:47.155786  4273 tablet_bootstrap.cc:492] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1: Bootstrap starting.
I20260709 12:23:47.162312  4274 tablet_bootstrap.cc:654] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.162881  4273 tablet_bootstrap.cc:654] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.219118  4273 tablet_bootstrap.cc:492] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1: No bootstrap required, opened a new log
I20260709 12:23:47.219244  4275 tablet_bootstrap.cc:492] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6: No bootstrap required, opened a new log
I20260709 12:23:47.219637  4273 ts_tablet_manager.cc:1403] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1: Time spent bootstrapping tablet: real 0.064s	user 0.015s	sys 0.045s
I20260709 12:23:47.219820  4275 ts_tablet_manager.cc:1403] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent bootstrapping tablet: real 0.101s	user 0.027s	sys 0.046s
I20260709 12:23:47.220141  4274 tablet_bootstrap.cc:492] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da: No bootstrap required, opened a new log
I20260709 12:23:47.220634  4274 ts_tablet_manager.cc:1403] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da: Time spent bootstrapping tablet: real 0.066s	user 0.023s	sys 0.040s
I20260709 12:23:47.222200  4273 raft_consensus.cc:359] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.222908  4273 raft_consensus.cc:385] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.223222  4273 raft_consensus.cc:740] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.223043  4275 raft_consensus.cc:359] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.223906  4275 raft_consensus.cc:385] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.223657  4274 raft_consensus.cc:359] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.223973  4273 consensus_queue.cc:260] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.224411  4275 raft_consensus.cc:740] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.224490  4274 raft_consensus.cc:385] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.225050  4274 raft_consensus.cc:740] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.225520  4275 consensus_queue.cc:260] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.226301  4273 ts_tablet_manager.cc:1434] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:23:47.225898  4274 consensus_queue.cc:260] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.227398  4273 tablet_bootstrap.cc:492] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1: Bootstrap starting.
I20260709 12:23:47.228458  4274 ts_tablet_manager.cc:1434] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 12:23:47.229462  4274 tablet_bootstrap.cc:492] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da: Bootstrap starting.
I20260709 12:23:47.234620  4273 tablet_bootstrap.cc:654] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.237999  4279 raft_consensus.cc:493] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.238574  4279 raft_consensus.cc:515] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.240604  4274 tablet_bootstrap.cc:654] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.242264  4273 tablet_bootstrap.cc:492] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1: No bootstrap required, opened a new log
I20260709 12:23:47.242718  4275 ts_tablet_manager.cc:1434] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.002s
I20260709 12:23:47.242921  4273 ts_tablet_manager.cc:1403] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 12:23:47.243939  4275 tablet_bootstrap.cc:492] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6: Bootstrap starting.
I20260709 12:23:47.250068  4273 raft_consensus.cc:359] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.251051  4273 raft_consensus.cc:385] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.251457  4273 raft_consensus.cc:740] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.248175  4279 leader_election.cc:290] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:47.252413  4273 consensus_queue.cc:260] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.255450  4273 ts_tablet_manager.cc:1434] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:23:47.256621  4273 tablet_bootstrap.cc:492] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1: Bootstrap starting.
I20260709 12:23:47.257205  4274 tablet_bootstrap.cc:492] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da: No bootstrap required, opened a new log
I20260709 12:23:47.257766  4274 ts_tablet_manager.cc:1403] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.000s
I20260709 12:23:47.258208  4275 tablet_bootstrap.cc:654] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.261134  4274 raft_consensus.cc:359] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.262127  4274 raft_consensus.cc:385] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.262598  4274 raft_consensus.cc:740] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.263525  4274 consensus_queue.cc:260] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.264642  4273 tablet_bootstrap.cc:654] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.266918  4274 ts_tablet_manager.cc:1434] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:23:47.268129  4274 tablet_bootstrap.cc:492] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da: Bootstrap starting.
I20260709 12:23:47.271858  4279 raft_consensus.cc:493] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.272490  4279 raft_consensus.cc:515] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.275318  4279 leader_election.cc:290] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:47.281396  4274 tablet_bootstrap.cc:654] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.281301  4280 raft_consensus.cc:493] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.282266  4280 raft_consensus.cc:515] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.292987  4275 tablet_bootstrap.cc:492] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6: No bootstrap required, opened a new log
I20260709 12:23:47.293619  4275 ts_tablet_manager.cc:1403] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.004s
I20260709 12:23:47.295557  4274 tablet_bootstrap.cc:492] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da: No bootstrap required, opened a new log
I20260709 12:23:47.296134  4274 ts_tablet_manager.cc:1403] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.007s
I20260709 12:23:47.297199  4275 raft_consensus.cc:359] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.298745  4275 raft_consensus.cc:385] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.299171  4275 raft_consensus.cc:740] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.300236  4275 consensus_queue.cc:260] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.300577  4274 raft_consensus.cc:359] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.296058  4280 leader_election.cc:290] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.302096  4274 raft_consensus.cc:385] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.302594  4274 raft_consensus.cc:740] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.303707  4274 consensus_queue.cc:260] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.306236  4275 ts_tablet_manager.cc:1434] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 12:23:47.307448  4275 tablet_bootstrap.cc:492] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6: Bootstrap starting.
I20260709 12:23:47.308100  4274 ts_tablet_manager.cc:1434] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 12:23:47.309506  4274 tablet_bootstrap.cc:492] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da: Bootstrap starting.
I20260709 12:23:47.315336  4275 tablet_bootstrap.cc:654] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.319605  4274 tablet_bootstrap.cc:654] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.328151  4281 raft_consensus.cc:493] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.329408  4281 raft_consensus.cc:515] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.333340  4274 tablet_bootstrap.cc:492] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da: No bootstrap required, opened a new log
I20260709 12:23:47.334288  4274 ts_tablet_manager.cc:1403] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.013s
I20260709 12:23:47.338465  4274 raft_consensus.cc:359] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.339176  4281 leader_election.cc:290] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.339830  4274 raft_consensus.cc:385] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.340287  4274 raft_consensus.cc:740] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.341799  4275 tablet_bootstrap.cc:492] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6: No bootstrap required, opened a new log
I20260709 12:23:47.341355  4274 consensus_queue.cc:260] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.342684  4275 ts_tablet_manager.cc:1403] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.008s
W20260709 12:23:47.346657  4028 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:47.347662  4275 raft_consensus.cc:359] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.349377  4274 ts_tablet_manager.cc:1434] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.002s
I20260709 12:23:47.349901  4275 raft_consensus.cc:385] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.350483  4275 raft_consensus.cc:740] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.350835  4274 tablet_bootstrap.cc:492] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da: Bootstrap starting.
I20260709 12:23:47.351755  4275 consensus_queue.cc:260] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.356956  4275 ts_tablet_manager.cc:1434] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.004s
I20260709 12:23:47.360718  4274 tablet_bootstrap.cc:654] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.361694  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1" is_pre_election: true
I20260709 12:23:47.362591  4158 raft_consensus.cc:2468] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 0.
I20260709 12:23:47.363039  4275 tablet_bootstrap.cc:492] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6: Bootstrap starting.
I20260709 12:23:47.363867  4281 raft_consensus.cc:493] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.364419  4281 raft_consensus.cc:515] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.366194  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974f32885718400cb4c83cf4a8689d3b" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1" is_pre_election: true
I20260709 12:23:47.366688  4281 leader_election.cc:290] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.367436  4158 raft_consensus.cc:2468] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 0.
I20260709 12:23:47.369174  4198 leader_election.cc:304] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:23:47.370488  4279 raft_consensus.cc:2804] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:47.370935  4279 raft_consensus.cc:493] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.371353  4279 raft_consensus.cc:3060] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.372888  4274 tablet_bootstrap.cc:492] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da: No bootstrap required, opened a new log
I20260709 12:23:47.373443  4274 ts_tablet_manager.cc:1403] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da: Time spent bootstrapping tablet: real 0.023s	user 0.003s	sys 0.014s
I20260709 12:23:47.373991  4198 leader_election.cc:304] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:23:47.379200  4274 raft_consensus.cc:359] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.379982  4274 raft_consensus.cc:385] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.379797  4279 raft_consensus.cc:515] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.380365  4274 raft_consensus.cc:740] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.382056  4279 leader_election.cc:290] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:47.382783  4279 raft_consensus.cc:2804] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:47.383201  4279 raft_consensus.cc:493] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.383658  4279 raft_consensus.cc:3060] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.385685  4274 consensus_queue.cc:260] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.392372  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" is_pre_election: true
I20260709 12:23:47.393137  4233 raft_consensus.cc:2468] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 0.
I20260709 12:23:47.393460  4273 tablet_bootstrap.cc:492] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1: No bootstrap required, opened a new log
I20260709 12:23:47.394060  4273 ts_tablet_manager.cc:1403] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1: Time spent bootstrapping tablet: real 0.138s	user 0.025s	sys 0.052s
I20260709 12:23:47.394758  4047 leader_election.cc:304] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:47.395315  4275 tablet_bootstrap.cc:654] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.395586  4280 raft_consensus.cc:2804] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:47.395952  4280 raft_consensus.cc:493] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.396292  4280 raft_consensus.cc:3060] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.397508  4273 raft_consensus.cc:359] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.398382  4273 raft_consensus.cc:385] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.398459  4304 raft_consensus.cc:493] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.398765  4273 raft_consensus.cc:740] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.399019  4304 raft_consensus.cc:515] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.399703  4273 consensus_queue.cc:260] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.402199  4273 ts_tablet_manager.cc:1434] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:23:47.403272  4273 tablet_bootstrap.cc:492] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1: Bootstrap starting.
I20260709 12:23:47.403663  4280 raft_consensus.cc:515] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.406004  4280 leader_election.cc:290] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 election: Requested vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.406956  4304 leader_election.cc:290] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:47.407650  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:23:47.408257  4233 raft_consensus.cc:3060] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.410321  4273 tablet_bootstrap.cc:654] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.414892  4279 raft_consensus.cc:515] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.419549  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1" is_pre_election: true
I20260709 12:23:47.420311  4158 raft_consensus.cc:2468] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 0.
I20260709 12:23:47.421061  4273 tablet_bootstrap.cc:492] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1: No bootstrap required, opened a new log
I20260709 12:23:47.421224  4233 raft_consensus.cc:2468] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 1.
I20260709 12:23:47.425061  4232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" is_pre_election: true
I20260709 12:23:47.425962  4232 raft_consensus.cc:2393] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79d614bf4bc24974baa0c843f2db25c1 in current term 1: Already voted for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in this term.
I20260709 12:23:47.426023  4047 leader_election.cc:304] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:47.427150  4231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" is_pre_election: true
I20260709 12:23:47.421707  4273 ts_tablet_manager.cc:1403] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.007s
I20260709 12:23:47.421628  4157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:23:47.424731  4280 raft_consensus.cc:493] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:47.427826  4198 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:43941, user_credentials={real_user=slave}} blocked reactor thread for 54002.5us
I20260709 12:23:47.429142  4280 raft_consensus.cc:515] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.430452  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:23:47.428059  4275 tablet_bootstrap.cc:492] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6: No bootstrap required, opened a new log
I20260709 12:23:47.428629  4157 raft_consensus.cc:3060] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.431070  4158 raft_consensus.cc:3060] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.425277  4279 leader_election.cc:290] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:47.432030  4280 leader_election.cc:290] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.432617  4280 raft_consensus.cc:2804] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:47.432287  4273 raft_consensus.cc:359] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.433064  4280 raft_consensus.cc:697] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 1 LEADER]: Becoming Leader. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:23:47.434473  4280 consensus_queue.cc:237] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.433091  4273 raft_consensus.cc:385] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.436939  4273 raft_consensus.cc:740] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.438026  4275 ts_tablet_manager.cc:1403] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent bootstrapping tablet: real 0.080s	user 0.015s	sys 0.001s
I20260709 12:23:47.437934  4273 consensus_queue.cc:260] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.441098  4156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1" is_pre_election: true
I20260709 12:23:47.442874  4273 ts_tablet_manager.cc:1434] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 12:23:47.444038  4273 tablet_bootstrap.cc:492] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1: Bootstrap starting.
I20260709 12:23:47.444778  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da" is_pre_election: true
I20260709 12:23:47.445493  4083 raft_consensus.cc:2468] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 0.
I20260709 12:23:47.447650  4155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974f32885718400cb4c83cf4a8689d3b" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:23:47.450096  4155 raft_consensus.cc:3060] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.449172  4154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3a70bf60d1483cbfadcdeae5b564a6" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1" is_pre_election: true
I20260709 12:23:47.450865  4273 tablet_bootstrap.cc:654] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.451079  4157 raft_consensus.cc:2468] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 1.
I20260709 12:23:47.441392  4275 raft_consensus.cc:359] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.452816  4275 raft_consensus.cc:385] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.452999  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da" is_pre_election: true
I20260709 12:23:47.390957  4274 ts_tablet_manager.cc:1434] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.003s
I20260709 12:23:47.462778  4274 tablet_bootstrap.cc:492] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da: Bootstrap starting.
I20260709 12:23:47.465090  4158 raft_consensus.cc:2468] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 1.
I20260709 12:23:47.465404  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da" is_pre_election: true
I20260709 12:23:47.470124  4083 raft_consensus.cc:2468] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 0.
I20260709 12:23:47.462222  4124 leader_election.cc:304] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 3f0748b6fdcd44dc8601f4ebc846aad6
I20260709 12:23:47.453156  4275 raft_consensus.cc:740] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.467598  4082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974f32885718400cb4c83cf4a8689d3b" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da" is_pre_election: true
W20260709 12:23:47.468456  4122 leader_election.cc:343] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:47.473503  4082 raft_consensus.cc:2468] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 0.
I20260709 12:23:47.474273  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:23:47.474538  4198 leader_election.cc:304] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:23:47.474912  4083 raft_consensus.cc:3060] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:47.473471  4047 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41075, user_credentials={real_user=slave}} blocked reactor thread for 47535.4us
I20260709 12:23:47.475370  4279 raft_consensus.cc:2804] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:47.475920  4279 raft_consensus.cc:697] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Becoming Leader. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:23:47.477180  4155 raft_consensus.cc:2468] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 1.
I20260709 12:23:47.478646  4281 raft_consensus.cc:2764] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:23:47.476855  4279 consensus_queue.cc:237] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.480082  4198 leader_election.cc:304] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:23:47.476858  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3a70bf60d1483cbfadcdeae5b564a6" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" is_pre_election: true
I20260709 12:23:47.481063  4304 raft_consensus.cc:2804] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:47.481977  4082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da" is_pre_election: true
I20260709 12:23:47.490042  4273 tablet_bootstrap.cc:492] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1: No bootstrap required, opened a new log
I20260709 12:23:47.490540  4273 ts_tablet_manager.cc:1403] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.001s
I20260709 12:23:47.493600  4083 raft_consensus.cc:2468] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 1.
I20260709 12:23:47.493468  4273 raft_consensus.cc:359] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.494269  4273 raft_consensus.cc:385] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:23:47.485797  4124 leader_election.cc:343] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb6356fa44624134819afb93643d63da (127.3.89.193:32943): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:47.493503  4047 leader_election.cc:343] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:47.497169  4273 raft_consensus.cc:740] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.499794  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974f32885718400cb4c83cf4a8689d3b" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:23:47.500397  4083 raft_consensus.cc:3060] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.500102  4273 consensus_queue.cc:260] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.502455  4273 ts_tablet_manager.cc:1434] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 12:23:47.503448  4273 tablet_bootstrap.cc:492] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1: Bootstrap starting.
W20260709 12:23:47.506554  4049 leader_election.cc:343] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:47.507083  4049 leader_election.cc:304] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb6356fa44624134819afb93643d63da; no voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1
I20260709 12:23:47.507998  4280 raft_consensus.cc:2749] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:47.508824  3977 catalog_manager.cc:5697] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.510092  4273 tablet_bootstrap.cc:654] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.513356  4083 raft_consensus.cc:2468] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 1.
I20260709 12:23:47.512842  4124 leader_election.cc:304] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1; no voters: 3f0748b6fdcd44dc8601f4ebc846aad6, eb6356fa44624134819afb93643d63da
I20260709 12:23:47.518573  4281 raft_consensus.cc:2749] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:47.487927  4275 consensus_queue.cc:260] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.520560  4198 leader_election.cc:304] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6; no voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da
I20260709 12:23:47.523128  4279 raft_consensus.cc:2749] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:47.523427  4274 tablet_bootstrap.cc:654] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.526193  4281 raft_consensus.cc:493] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.526783  4275 ts_tablet_manager.cc:1434] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent starting tablet: real 0.088s	user 0.004s	sys 0.004s
I20260709 12:23:47.526824  4281 raft_consensus.cc:515] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.527952  4275 tablet_bootstrap.cc:492] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6: Bootstrap starting.
I20260709 12:23:47.527048  3978 catalog_manager.cc:5697] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da reported cstate change: term changed from 0 to 1, leader changed from <none> to eb6356fa44624134819afb93643d63da (127.3.89.193). New cstate: current_term: 1 leader_uuid: "eb6356fa44624134819afb93643d63da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:47.531451  4281 leader_election.cc:290] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.531554  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3a70bf60d1483cbfadcdeae5b564a6" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" is_pre_election: true
I20260709 12:23:47.531670  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3a70bf60d1483cbfadcdeae5b564a6" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da" is_pre_election: true
I20260709 12:23:47.532462  4083 raft_consensus.cc:2468] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 0.
W20260709 12:23:47.533113  4122 leader_election.cc:343] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:47.529688  4304 raft_consensus.cc:697] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Becoming Leader. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:23:47.534711  4304 consensus_queue.cc:237] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.537081  4124 leader_election.cc:304] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 3f0748b6fdcd44dc8601f4ebc846aad6
I20260709 12:23:47.538103  4281 raft_consensus.cc:2804] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:47.538820  4281 raft_consensus.cc:493] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.540016  4281 raft_consensus.cc:3060] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.543473  4273 tablet_bootstrap.cc:492] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1: No bootstrap required, opened a new log
I20260709 12:23:47.543972  4273 ts_tablet_manager.cc:1403] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.002s
I20260709 12:23:47.545534  4275 tablet_bootstrap.cc:654] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.546893  4273 raft_consensus.cc:359] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.547636  4273 raft_consensus.cc:385] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.547904  4273 raft_consensus.cc:740] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.548413  4281 raft_consensus.cc:515] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.548900  4273 consensus_queue.cc:260] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.551898  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3a70bf60d1483cbfadcdeae5b564a6" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:23:47.552476  4083 raft_consensus.cc:3060] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.555538  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3a70bf60d1483cbfadcdeae5b564a6" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
W20260709 12:23:47.557789  4122 leader_election.cc:343] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:47.559913  4281 leader_election.cc:290] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.561662  4083 raft_consensus.cc:2468] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 1.
I20260709 12:23:47.563103  4124 leader_election.cc:304] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 3f0748b6fdcd44dc8601f4ebc846aad6
I20260709 12:23:47.572034  4304 raft_consensus.cc:493] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.572549  4304 raft_consensus.cc:515] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.574980  4274 tablet_bootstrap.cc:492] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da: No bootstrap required, opened a new log
I20260709 12:23:47.575014  4304 leader_election.cc:290] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:47.575801  4281 raft_consensus.cc:2804] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:47.576434  4281 raft_consensus.cc:697] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Becoming Leader. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Running, Role: LEADER
I20260709 12:23:47.576821  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da" is_pre_election: true
I20260709 12:23:47.577270  4155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1" is_pre_election: true
I20260709 12:23:47.577894  4155 raft_consensus.cc:2468] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 0.
W20260709 12:23:47.578181  4198 leader_election.cc:343] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb6356fa44624134819afb93643d63da (127.3.89.193:32943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:47.577605  4281 consensus_queue.cc:237] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.579041  4198 leader_election.cc:304] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: eb6356fa44624134819afb93643d63da
I20260709 12:23:47.580117  4304 raft_consensus.cc:2804] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:47.580283  4274 ts_tablet_manager.cc:1403] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da: Time spent bootstrapping tablet: real 0.118s	user 0.009s	sys 0.006s
I20260709 12:23:47.580513  4304 raft_consensus.cc:493] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.580911  4304 raft_consensus.cc:3060] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.584061  4274 raft_consensus.cc:359] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.584977  4274 raft_consensus.cc:385] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.585314  4274 raft_consensus.cc:740] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.586387  4275 tablet_bootstrap.cc:492] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6: No bootstrap required, opened a new log
I20260709 12:23:47.586711  4274 consensus_queue.cc:260] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.589251  4273 ts_tablet_manager.cc:1434] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1: Time spent starting tablet: real 0.045s	user 0.025s	sys 0.021s
I20260709 12:23:47.589943  4275 ts_tablet_manager.cc:1403] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.005s
I20260709 12:23:47.590483  4273 tablet_bootstrap.cc:492] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1: Bootstrap starting.
I20260709 12:23:47.594460  4274 ts_tablet_manager.cc:1434] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 12:23:47.595368  4275 raft_consensus.cc:359] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.596226  4275 raft_consensus.cc:385] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.596544  4274 tablet_bootstrap.cc:492] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da: Bootstrap starting.
I20260709 12:23:47.596563  4275 raft_consensus.cc:740] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.594717  4304 raft_consensus.cc:515] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.598732  4275 consensus_queue.cc:260] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.601125  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:23:47.601964  4273 tablet_bootstrap.cc:654] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:47.603147  4198 leader_election.cc:343] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb6356fa44624134819afb93643d63da (127.3.89.193:32943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:47.607533  4275 ts_tablet_manager.cc:1434] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.002s
I20260709 12:23:47.608186  4304 leader_election.cc:290] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:47.608775  4275 tablet_bootstrap.cc:492] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6: Bootstrap starting.
I20260709 12:23:47.609581  4304 raft_consensus.cc:493] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.610193  4304 raft_consensus.cc:515] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.612115  3977 catalog_manager.cc:5697] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.614125  4304 leader_election.cc:290] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:47.615958  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c9ed3020c344279bcf9b4f9eea78cf" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da" is_pre_election: true
I20260709 12:23:47.616319  4275 tablet_bootstrap.cc:654] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.616677  4083 raft_consensus.cc:2468] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 0.
I20260709 12:23:47.617959  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c9ed3020c344279bcf9b4f9eea78cf" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1" is_pre_election: true
I20260709 12:23:47.618176  4155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:23:47.619092  4155 raft_consensus.cc:3060] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:47.619740  4198 leader_election.cc:343] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:47.621016  4198 leader_election.cc:304] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, eb6356fa44624134819afb93643d63da; no voters: 79d614bf4bc24974baa0c843f2db25c1
I20260709 12:23:47.622223  4304 raft_consensus.cc:2804] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:47.622500  4273 tablet_bootstrap.cc:492] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1: No bootstrap required, opened a new log
I20260709 12:23:47.622747  4304 raft_consensus.cc:493] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.623014  4273 ts_tablet_manager.cc:1403] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1: Time spent bootstrapping tablet: real 0.033s	user 0.003s	sys 0.008s
I20260709 12:23:47.623142  4304 raft_consensus.cc:3060] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.626579  4273 raft_consensus.cc:359] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.627326  4273 raft_consensus.cc:385] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.627614  4273 raft_consensus.cc:740] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.629925  4274 tablet_bootstrap.cc:654] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.630230  4273 consensus_queue.cc:260] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.631214  4280 raft_consensus.cc:493] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.631888  4280 raft_consensus.cc:515] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.633204  4273 ts_tablet_manager.cc:1434] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:23:47.635012  4273 tablet_bootstrap.cc:492] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1: Bootstrap starting.
I20260709 12:23:47.636173  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26f47e7ee0346538108cf9b1350bd5d" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1" is_pre_election: true
I20260709 12:23:47.636459  4280 leader_election.cc:290] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.638156  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26f47e7ee0346538108cf9b1350bd5d" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" is_pre_election: true
W20260709 12:23:47.639675  4047 leader_election.cc:343] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:47.635043  4304 raft_consensus.cc:515] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
W20260709 12:23:47.642405  4049 leader_election.cc:343] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:47.642743  4049 leader_election.cc:304] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb6356fa44624134819afb93643d63da; no voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1
I20260709 12:23:47.643261  4274 tablet_bootstrap.cc:492] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da: No bootstrap required, opened a new log
I20260709 12:23:47.643560  4280 raft_consensus.cc:2749] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:47.644066  4304 leader_election.cc:290] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:47.645399  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c9ed3020c344279bcf9b4f9eea78cf" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:23:47.646014  4158 raft_consensus.cc:3060] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.646217  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c9ed3020c344279bcf9b4f9eea78cf" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:23:47.647068  4083 raft_consensus.cc:3060] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.650362  4273 tablet_bootstrap.cc:654] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.653043  4158 raft_consensus.cc:2468] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 1.
I20260709 12:23:47.654639  4198 leader_election.cc:304] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:23:47.656199  4274 ts_tablet_manager.cc:1403] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.000s
I20260709 12:23:47.656808  4281 raft_consensus.cc:493] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.655618  4304 raft_consensus.cc:2804] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:47.657476  4281 raft_consensus.cc:515] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.658912  4155 raft_consensus.cc:2468] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 1.
I20260709 12:23:47.659515  4274 raft_consensus.cc:359] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.660024  4281 leader_election.cc:290] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.660257  4274 raft_consensus.cc:385] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.660620  4274 raft_consensus.cc:740] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.660750  4082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da" is_pre_election: true
I20260709 12:23:47.661427  4082 raft_consensus.cc:2468] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 0.
I20260709 12:23:47.662111  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" is_pre_election: true
I20260709 12:23:47.661664  4274 consensus_queue.cc:260] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.662676  4124 leader_election.cc:304] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:47.663571  4281 raft_consensus.cc:2804] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:23:47.663690  4122 leader_election.cc:343] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:47.663973  4281 raft_consensus.cc:493] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.664307  4281 raft_consensus.cc:3060] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.658005  4304 raft_consensus.cc:697] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Becoming Leader. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:23:47.665189  4316 raft_consensus.cc:493] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.665649  4304 consensus_queue.cc:237] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.666091  4316 raft_consensus.cc:515] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.664808  4274 ts_tablet_manager.cc:1434] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:23:47.670873  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272c43daddc341079f8a8af9c2f69e44" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" is_pre_election: true
I20260709 12:23:47.671784  4233 raft_consensus.cc:2468] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 0.
I20260709 12:23:47.671285  4274 tablet_bootstrap.cc:492] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da: Bootstrap starting.
I20260709 12:23:47.673498  4122 leader_election.cc:304] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:23:47.676216  4082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272c43daddc341079f8a8af9c2f69e44" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da" is_pre_election: true
I20260709 12:23:47.676867  4082 raft_consensus.cc:2468] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 0.
I20260709 12:23:47.678344  4316 leader_election.cc:290] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.677351  3980 catalog_manager.cc:5697] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "79d614bf4bc24974baa0c843f2db25c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:47.678906  4316 raft_consensus.cc:2804] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:47.679280  4316 raft_consensus.cc:493] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.679613  4316 raft_consensus.cc:3060] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.665184  4083 raft_consensus.cc:2468] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 1.
I20260709 12:23:47.684473  4281 raft_consensus.cc:515] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.686371  4274 tablet_bootstrap.cc:654] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.697005  4274 tablet_bootstrap.cc:492] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da: No bootstrap required, opened a new log
I20260709 12:23:47.697513  4274 ts_tablet_manager.cc:1403] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.003s
I20260709 12:23:47.698527  4198 leader_election.cc:304] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: eb6356fa44624134819afb93643d63da
I20260709 12:23:47.699563  4310 raft_consensus.cc:2804] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:47.700451  4310 raft_consensus.cc:697] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Becoming Leader. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:23:47.700937  4281 leader_election.cc:290] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.701740  4310 consensus_queue.cc:237] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.710309  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:23:47.712334  4083 raft_consensus.cc:3060] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.742899  4274 raft_consensus.cc:359] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.743860  4274 raft_consensus.cc:385] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.744180  4274 raft_consensus.cc:740] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.745081  4274 consensus_queue.cc:260] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.751055  4274 ts_tablet_manager.cc:1434] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 12:23:47.759130  4083 raft_consensus.cc:2468] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 1.
I20260709 12:23:47.761096  4124 leader_election.cc:304] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:47.746639  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:23:47.767656  4280 raft_consensus.cc:493] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:47.769021  4122 leader_election.cc:343] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:47.769269  4280 raft_consensus.cc:515] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.770979  4316 raft_consensus.cc:515] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.775458  4155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1" is_pre_election: true
I20260709 12:23:47.776124  4155 raft_consensus.cc:2468] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 0.
I20260709 12:23:47.777037  4273 tablet_bootstrap.cc:492] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1: No bootstrap required, opened a new log
I20260709 12:23:47.777534  4273 ts_tablet_manager.cc:1403] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1: Time spent bootstrapping tablet: real 0.144s	user 0.015s	sys 0.053s
I20260709 12:23:47.782502  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" is_pre_election: true
I20260709 12:23:47.782598  4049 leader_election.cc:304] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:47.783967  4307 raft_consensus.cc:2804] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:47.784435  4307 raft_consensus.cc:493] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:47.784578  4047 leader_election.cc:343] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:47.784828  4307 raft_consensus.cc:3060] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.786931  4316 leader_election.cc:290] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.780560  3978 catalog_manager.cc:5697] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.789942  4280 leader_election.cc:290] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.792994  4316 raft_consensus.cc:2804] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:47.796983  4273 raft_consensus.cc:359] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.797737  4316 raft_consensus.cc:697] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Becoming Leader. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Running, Role: LEADER
I20260709 12:23:47.797531  4273 raft_consensus.cc:385] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.798871  4273 raft_consensus.cc:740] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.799471  4273 consensus_queue.cc:260] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.793704  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272c43daddc341079f8a8af9c2f69e44" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:23:47.801268  4233 raft_consensus.cc:3060] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.803318  4273 ts_tablet_manager.cc:1434] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:23:47.804212  4316 consensus_queue.cc:237] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.801128  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272c43daddc341079f8a8af9c2f69e44" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:23:47.808933  4275 tablet_bootstrap.cc:492] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6: No bootstrap required, opened a new log
I20260709 12:23:47.809458  4275 ts_tablet_manager.cc:1403] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent bootstrapping tablet: real 0.201s	user 0.034s	sys 0.063s
I20260709 12:23:47.812441  4307 raft_consensus.cc:515] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.808081  4083 raft_consensus.cc:3060] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.814945  4307 leader_election.cc:290] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 election: Requested vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:47.812537  4275 raft_consensus.cc:359] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.815567  4275 raft_consensus.cc:385] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.815729  4155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:23:47.815943  4275 raft_consensus.cc:740] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.815999  4233 raft_consensus.cc:2468] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 1.
I20260709 12:23:47.816363  4155 raft_consensus.cc:3060] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.816936  4275 consensus_queue.cc:260] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.817485  4122 leader_election.cc:304] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:23:47.818321  4281 raft_consensus.cc:2804] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:47.818847  4281 raft_consensus.cc:697] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Becoming Leader. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Running, Role: LEADER
I20260709 12:23:47.819847  4275 ts_tablet_manager.cc:1434] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:23:47.819766  4281 consensus_queue.cc:237] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.820600  4231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:23:47.820950  4275 tablet_bootstrap.cc:492] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 3f0748b6fdcd44dc8601f4ebc846aad6: Bootstrap starting.
I20260709 12:23:47.821091  4231 raft_consensus.cc:3060] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.827345  4307 consensus_queue.cc:1048] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:47.830292  4275 tablet_bootstrap.cc:654] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 3f0748b6fdcd44dc8601f4ebc846aad6: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.830999  4231 raft_consensus.cc:2468] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 1.
I20260709 12:23:47.837103  4047 leader_election.cc:304] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:47.838330  4307 raft_consensus.cc:2804] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:47.839010  4307 raft_consensus.cc:697] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 1 LEADER]: Becoming Leader. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:23:47.840101  4307 consensus_queue.cc:237] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.841002  4275 tablet_bootstrap.cc:492] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 3f0748b6fdcd44dc8601f4ebc846aad6: No bootstrap required, opened a new log
I20260709 12:23:47.841552  4275 ts_tablet_manager.cc:1403] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.003s
I20260709 12:23:47.842142  4155 raft_consensus.cc:2468] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 1.
I20260709 12:23:47.844746  4275 raft_consensus.cc:359] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.846028  4275 raft_consensus.cc:385] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.846374  4275 raft_consensus.cc:740] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.847108  4275 consensus_queue.cc:260] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.849336  4275 ts_tablet_manager.cc:1434] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:23:47.850366  4275 tablet_bootstrap.cc:492] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6: Bootstrap starting.
I20260709 12:23:47.850713  3978 catalog_manager.cc:5697] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.853917  3980 catalog_manager.cc:5697] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "79d614bf4bc24974baa0c843f2db25c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:47.856678  4275 tablet_bootstrap.cc:654] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.855263  4083 raft_consensus.cc:2468] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 1.
I20260709 12:23:47.872444  4275 tablet_bootstrap.cc:492] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6: No bootstrap required, opened a new log
I20260709 12:23:47.872972  4275 ts_tablet_manager.cc:1403] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.005s
I20260709 12:23:47.873296  3977 catalog_manager.cc:5697] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da reported cstate change: term changed from 0 to 1, leader changed from <none> to eb6356fa44624134819afb93643d63da (127.3.89.193). New cstate: current_term: 1 leader_uuid: "eb6356fa44624134819afb93643d63da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:47.875864  4275 raft_consensus.cc:359] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.876642  4275 raft_consensus.cc:385] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.877046  4275 raft_consensus.cc:740] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.876561  3980 catalog_manager.cc:5697] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "79d614bf4bc24974baa0c843f2db25c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:47.877842  4275 consensus_queue.cc:260] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.879462  4275 ts_tablet_manager.cc:1434] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:23:47.880134  4275 tablet_bootstrap.cc:492] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6: Bootstrap starting.
I20260709 12:23:47.886282  4275 tablet_bootstrap.cc:654] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:47.895993  4275 tablet_bootstrap.cc:492] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6: No bootstrap required, opened a new log
I20260709 12:23:47.896487  4275 ts_tablet_manager.cc:1403] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.003s
I20260709 12:23:47.898604  4307 consensus_queue.cc:1048] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:47.899070  4275 raft_consensus.cc:359] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.899819  4275 raft_consensus.cc:385] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:47.900122  4275 raft_consensus.cc:740] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Initialized, Role: FOLLOWER
I20260709 12:23:47.900979  4275 consensus_queue.cc:260] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:47.903507  4275 ts_tablet_manager.cc:1434] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:47.918848  4304 consensus_queue.cc:1048] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:47.930378  4304 consensus_queue.cc:1048] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:47.981508  4231 raft_consensus.cc:3060] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:47.987483  4316 consensus_queue.cc:1048] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:47.998498  4281 consensus_queue.cc:1048] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.010021  4304 consensus_queue.cc:1048] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.021623  4304 consensus_queue.cc:1048] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.066586  4304 consensus_queue.cc:1048] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.078662  4310 consensus_queue.cc:1048] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.153481  4280 raft_consensus.cc:493] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.154057  4280 raft_consensus.cc:515] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.155997  4280 leader_election.cc:290] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:48.156847  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1" is_pre_election: true
I20260709 12:23:48.157184  4231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" is_pre_election: true
I20260709 12:23:48.157620  4158 raft_consensus.cc:2393] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb6356fa44624134819afb93643d63da in current term 1: Already voted for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in this term.
I20260709 12:23:48.158659  4049 leader_election.cc:304] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb6356fa44624134819afb93643d63da; no voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1
I20260709 12:23:48.159400  4280 raft_consensus.cc:3060] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:48.164160  4280 raft_consensus.cc:2749] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:48.204599  4310 consensus_queue.cc:1048] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.214148  4279 raft_consensus.cc:493] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.214699  4279 raft_consensus.cc:515] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.216643  4304 consensus_queue.cc:1048] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.216956  4279 leader_election.cc:290] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:48.217932  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da" is_pre_election: true
I20260709 12:23:48.218396  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d614bf4bc24974baa0c843f2db25c1" is_pre_election: true
I20260709 12:23:48.218734  4083 raft_consensus.cc:2393] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in current term 1: Already voted for candidate 79d614bf4bc24974baa0c843f2db25c1 in this term.
I20260709 12:23:48.220743  4198 leader_election.cc:304] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6; no voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da
I20260709 12:23:48.221529  4279 raft_consensus.cc:3060] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:48.229454  4279 raft_consensus.cc:2749] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:48.252209  4281 raft_consensus.cc:493] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.252869  4281 raft_consensus.cc:515] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.254772  4281 leader_election.cc:290] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:48.255669  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26f47e7ee0346538108cf9b1350bd5d" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da" is_pre_election: true
I20260709 12:23:48.255806  4231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26f47e7ee0346538108cf9b1350bd5d" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" is_pre_election: true
I20260709 12:23:48.256317  4083 raft_consensus.cc:2468] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 0.
I20260709 12:23:48.256435  4231 raft_consensus.cc:2468] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 0.
I20260709 12:23:48.257433  4124 leader_election.cc:304] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:48.258172  4281 raft_consensus.cc:2804] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:48.258559  4281 raft_consensus.cc:493] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:48.258865  4281 raft_consensus.cc:3060] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:48.263444  4281 raft_consensus.cc:515] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.265270  4281 leader_election.cc:290] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:48.266012  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26f47e7ee0346538108cf9b1350bd5d" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:23:48.266373  4231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26f47e7ee0346538108cf9b1350bd5d" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:23:48.266602  4083 raft_consensus.cc:3060] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:48.266970  4231 raft_consensus.cc:3060] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:48.271306  4083 raft_consensus.cc:2468] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 1.
I20260709 12:23:48.271402  4231 raft_consensus.cc:2468] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 1.
I20260709 12:23:48.272562  4124 leader_election.cc:304] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:48.273320  4281 raft_consensus.cc:2804] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:48.273792  4281 raft_consensus.cc:697] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Becoming Leader. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Running, Role: LEADER
I20260709 12:23:48.274436  4281 consensus_queue.cc:237] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.278908  4281 consensus_queue.cc:1048] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.283344  3979 catalog_manager.cc:5697] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "79d614bf4bc24974baa0c843f2db25c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.291695  4316 consensus_queue.cc:1048] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.338615  4316 consensus_queue.cc:1048] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.350360  4316 consensus_queue.cc:1048] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.402630  4339 consensus_queue.cc:1048] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.413444  4280 consensus_queue.cc:1048] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.631955  4231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f0c9ed3020c344279bcf9b4f9eea78cf"
dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
mode: GRACEFUL
new_leader_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:35142
I20260709 12:23:48.632634  4231 raft_consensus.cc:606] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Received request to transfer leadership to TS eb6356fa44624134819afb93643d63da
I20260709 12:23:48.637856  4231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216"
dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
mode: GRACEFUL
new_leader_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:35142
I20260709 12:23:48.638321  4231 raft_consensus.cc:606] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Received request to transfer leadership to TS eb6356fa44624134819afb93643d63da
I20260709 12:23:48.652316  4158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec"
dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
mode: GRACEFUL
new_leader_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:51458
I20260709 12:23:48.652877  4158 raft_consensus.cc:606] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Received request to transfer leadership to TS eb6356fa44624134819afb93643d63da
I20260709 12:23:48.660917  4231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "974f32885718400cb4c83cf4a8689d3b"
dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
mode: GRACEFUL
new_leader_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:35142
I20260709 12:23:48.661554  4231 raft_consensus.cc:606] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Received request to transfer leadership to TS eb6356fa44624134819afb93643d63da
I20260709 12:23:48.675199  4083 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243"
dest_uuid: "eb6356fa44624134819afb93643d63da"
mode: GRACEFUL
new_leader_uuid: "79d614bf4bc24974baa0c843f2db25c1"
 from {username='slave'} at 127.0.0.1:43578
I20260709 12:23:48.675755  4083 raft_consensus.cc:606] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 1 LEADER]: Received request to transfer leadership to TS 79d614bf4bc24974baa0c843f2db25c1
I20260709 12:23:48.681370  4231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7"
dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
mode: GRACEFUL
new_leader_uuid: "79d614bf4bc24974baa0c843f2db25c1"
 from {username='slave'} at 127.0.0.1:35142
I20260709 12:23:48.682031  4231 raft_consensus.cc:606] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Received request to transfer leadership to TS 79d614bf4bc24974baa0c843f2db25c1
I20260709 12:23:48.692498  4158 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e26f47e7ee0346538108cf9b1350bd5d"
dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
mode: GRACEFUL
new_leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
 from {username='slave'} at 127.0.0.1:51458
I20260709 12:23:48.693058  4158 raft_consensus.cc:606] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Received request to transfer leadership to TS 3f0748b6fdcd44dc8601f4ebc846aad6
I20260709 12:23:48.708062  4281 consensus_queue.cc:1048] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.714931  4327 raft_consensus.cc:993] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1: : Instructing follower 3f0748b6fdcd44dc8601f4ebc846aad6 to start an election
I20260709 12:23:48.715435  4316 raft_consensus.cc:1081] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Signalling peer 3f0748b6fdcd44dc8601f4ebc846aad6 to start an election
I20260709 12:23:48.718380  4233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e26f47e7ee0346538108cf9b1350bd5d"
dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
 from {username='slave'} at 127.0.0.1:35138
I20260709 12:23:48.719022  4233 raft_consensus.cc:493] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:48.719432  4233 raft_consensus.cc:3060] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.721895  4327 consensus_queue.cc:1048] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:48.726866  4233 raft_consensus.cc:515] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.729911  4082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26f47e7ee0346538108cf9b1350bd5d" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:23:48.730547  4082 raft_consensus.cc:3060] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.730769  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26f47e7ee0346538108cf9b1350bd5d" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:23:48.731381  4158 raft_consensus.cc:3055] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:48.731719  4158 raft_consensus.cc:740] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Running, Role: LEADER
I20260709 12:23:48.732005  4233 leader_election.cc:290] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 2 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:48.732561  4158 consensus_queue.cc:260] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.733839  4158 raft_consensus.cc:3060] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.738656  4082 raft_consensus.cc:2468] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 2.
I20260709 12:23:48.741845  4198 leader_election.cc:304] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:48.742787  4304 raft_consensus.cc:2804] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:48.743185  4158 raft_consensus.cc:2468] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 2.
I20260709 12:23:48.743275  4304 raft_consensus.cc:697] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 LEADER]: Becoming Leader. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:23:48.744197  4304 consensus_queue.cc:237] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.751631  3977 catalog_manager.cc:5697] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 reported cstate change: term changed from 1 to 2, leader changed from 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194) to 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:48.764662  4281 raft_consensus.cc:993] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1: : Instructing follower eb6356fa44624134819afb93643d63da to start an election
I20260709 12:23:48.765115  4281 raft_consensus.cc:1081] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Signalling peer eb6356fa44624134819afb93643d63da to start an election
I20260709 12:23:48.766710  4082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec"
dest_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:43568
I20260709 12:23:48.767294  4082 raft_consensus.cc:493] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:48.767689  4082 raft_consensus.cc:3060] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.772786  4082 raft_consensus.cc:515] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.774608  4082 leader_election.cc:290] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 2 election: Requested vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:48.775622  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:23:48.775774  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:23:48.776213  4158 raft_consensus.cc:3055] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:48.776329  4233 raft_consensus.cc:3060] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.776556  4158 raft_consensus.cc:740] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Running, Role: LEADER
I20260709 12:23:48.777400  4158 consensus_queue.cc:260] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.778469  4158 raft_consensus.cc:3060] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.781265  4233 raft_consensus.cc:2468] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 2.
I20260709 12:23:48.782442  4047 leader_election.cc:304] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:48.783309  4280 raft_consensus.cc:2804] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:48.783756  4280 raft_consensus.cc:697] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Becoming Leader. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:23:48.783982  4158 raft_consensus.cc:2468] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 2.
I20260709 12:23:48.784598  4280 consensus_queue.cc:237] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.790206  4310 raft_consensus.cc:993] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6: : Instructing follower eb6356fa44624134819afb93643d63da to start an election
I20260709 12:23:48.790693  4310 raft_consensus.cc:1081] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Signalling peer eb6356fa44624134819afb93643d63da to start an election
I20260709 12:23:48.792412  4082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216"
dest_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:43562
I20260709 12:23:48.792984  4082 raft_consensus.cc:493] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:48.792459  3980 catalog_manager.cc:5697] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da reported cstate change: term changed from 1 to 2, leader changed from 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194) to eb6356fa44624134819afb93643d63da (127.3.89.193). New cstate: current_term: 2 leader_uuid: "eb6356fa44624134819afb93643d63da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.793380  4082 raft_consensus.cc:3060] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.799890  4082 raft_consensus.cc:515] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.801949  4082 leader_election.cc:290] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 2 election: Requested vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:48.802690  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:23:48.802927  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:23:48.803411  4158 raft_consensus.cc:3060] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.803486  4233 raft_consensus.cc:3055] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:48.803855  4233 raft_consensus.cc:740] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:23:48.804512  4233 consensus_queue.cc:260] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.805451  4233 raft_consensus.cc:3060] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.808924  4158 raft_consensus.cc:2468] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 2.
I20260709 12:23:48.810034  4049 leader_election.cc:304] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:48.810420  4233 raft_consensus.cc:2468] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 2.
I20260709 12:23:48.810788  4280 raft_consensus.cc:2804] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:48.811322  4280 raft_consensus.cc:697] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Becoming Leader. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:23:48.812052  4280 consensus_queue.cc:237] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.818122  3980 catalog_manager.cc:5697] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da reported cstate change: term changed from 1 to 2, leader changed from 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195) to eb6356fa44624134819afb93643d63da (127.3.89.193). New cstate: current_term: 2 leader_uuid: "eb6356fa44624134819afb93643d63da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.839948  4304 raft_consensus.cc:993] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6: : Instructing follower 79d614bf4bc24974baa0c843f2db25c1 to start an election
I20260709 12:23:48.840457  4304 raft_consensus.cc:1081] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Signalling peer 79d614bf4bc24974baa0c843f2db25c1 to start an election
I20260709 12:23:48.842042  4158 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7"
dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
 from {username='slave'} at 127.0.0.1:51432
I20260709 12:23:48.842584  4158 raft_consensus.cc:493] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:48.843012  4158 raft_consensus.cc:3060] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.848066  4158 raft_consensus.cc:515] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.850682  4158 leader_election.cc:290] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 2 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:48.850924  4082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:23:48.852453  4231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:23:48.851905  4082 raft_consensus.cc:3060] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.853087  4231 raft_consensus.cc:3055] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:48.853488  4231 raft_consensus.cc:740] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:23:48.854413  4231 consensus_queue.cc:260] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.855729  4231 raft_consensus.cc:3060] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.860518  4082 raft_consensus.cc:2468] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 2.
I20260709 12:23:48.861835  4124 leader_election.cc:304] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:48.862677  4281 raft_consensus.cc:2804] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:48.863111  4281 raft_consensus.cc:697] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 LEADER]: Becoming Leader. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Running, Role: LEADER
I20260709 12:23:48.863180  4231 raft_consensus.cc:2468] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 2.
I20260709 12:23:48.863989  4281 consensus_queue.cc:237] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:48.870404  3980 catalog_manager.cc:5697] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 reported cstate change: term changed from 1 to 2, leader changed from 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195) to 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "79d614bf4bc24974baa0c843f2db25c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:49.004302  4307 raft_consensus.cc:993] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da: : Instructing follower 79d614bf4bc24974baa0c843f2db25c1 to start an election
I20260709 12:23:49.004729  4280 raft_consensus.cc:1081] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 1 LEADER]: Signalling peer 79d614bf4bc24974baa0c843f2db25c1 to start an election
I20260709 12:23:49.006531  4158 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243"
dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
 from {username='slave'} at 127.0.0.1:51448
I20260709 12:23:49.007043  4158 raft_consensus.cc:493] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.007359  4158 raft_consensus.cc:3060] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.010346  4310 raft_consensus.cc:993] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6: : Instructing follower eb6356fa44624134819afb93643d63da to start an election
I20260709 12:23:49.010795  4304 raft_consensus.cc:1081] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Signalling peer eb6356fa44624134819afb93643d63da to start an election
I20260709 12:23:49.012267  4082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f0c9ed3020c344279bcf9b4f9eea78cf"
dest_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:43562
I20260709 12:23:49.012764  4082 raft_consensus.cc:493] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.013088  4082 raft_consensus.cc:3060] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.013293  4158 raft_consensus.cc:515] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:49.014981  4158 leader_election.cc:290] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 2 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:49.015945  4083 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:23:49.016112  4231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:23:49.016528  4083 raft_consensus.cc:3055] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:49.016773  4231 raft_consensus.cc:3060] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.016911  4083 raft_consensus.cc:740] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:23:49.017745  4083 consensus_queue.cc:260] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:49.018823  4083 raft_consensus.cc:3060] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.019624  4082 raft_consensus.cc:515] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:49.023554  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c9ed3020c344279bcf9b4f9eea78cf" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:23:49.023622  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c9ed3020c344279bcf9b4f9eea78cf" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:23:49.024364  4158 raft_consensus.cc:3060] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.024441  4233 raft_consensus.cc:3055] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:49.024503  4231 raft_consensus.cc:2468] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 2.
I20260709 12:23:49.025017  4233 raft_consensus.cc:740] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:23:49.025784  4083 raft_consensus.cc:2468] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 2.
I20260709 12:23:49.026154  4122 leader_election.cc:304] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:23:49.026050  4233 consensus_queue.cc:260] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:49.027156  4281 raft_consensus.cc:2804] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.027225  4082 leader_election.cc:290] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 2 election: Requested vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:49.027402  4233 raft_consensus.cc:3060] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.027683  4281 raft_consensus.cc:697] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 LEADER]: Becoming Leader. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Running, Role: LEADER
I20260709 12:23:49.028548  4281 consensus_queue.cc:237] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:49.031997  4158 raft_consensus.cc:2468] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 2.
I20260709 12:23:49.033221  4049 leader_election.cc:304] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:49.033954  4280 raft_consensus.cc:2804] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.034451  4280 raft_consensus.cc:697] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Becoming Leader. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:23:49.035468  4233 raft_consensus.cc:2468] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 2.
I20260709 12:23:49.036593  4280 consensus_queue.cc:237] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:49.040668  3980 catalog_manager.cc:5697] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 reported cstate change: term changed from 1 to 2, leader changed from eb6356fa44624134819afb93643d63da (127.3.89.193) to 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "79d614bf4bc24974baa0c843f2db25c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:49.043605  3977 catalog_manager.cc:5697] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da reported cstate change: term changed from 1 to 2, leader changed from 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195) to eb6356fa44624134819afb93643d63da (127.3.89.193). New cstate: current_term: 2 leader_uuid: "eb6356fa44624134819afb93643d63da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:49.053057  4310 raft_consensus.cc:993] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6: : Instructing follower eb6356fa44624134819afb93643d63da to start an election
I20260709 12:23:49.053534  4304 raft_consensus.cc:1081] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Signalling peer eb6356fa44624134819afb93643d63da to start an election
I20260709 12:23:49.055121  4082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "974f32885718400cb4c83cf4a8689d3b"
dest_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:43562
I20260709 12:23:49.055668  4082 raft_consensus.cc:493] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.056068  4082 raft_consensus.cc:3060] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.061055  4082 raft_consensus.cc:515] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:49.062988  4082 leader_election.cc:290] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 2 election: Requested vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:23:49.063760  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974f32885718400cb4c83cf4a8689d3b" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:23:49.064288  4158 raft_consensus.cc:3060] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.064273  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974f32885718400cb4c83cf4a8689d3b" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:23:49.065009  4233 raft_consensus.cc:3055] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:49.065241  4233 raft_consensus.cc:740] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:23:49.066001  4233 consensus_queue.cc:260] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:49.066896  4233 raft_consensus.cc:3060] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:49.069458  4158 raft_consensus.cc:2468] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 2.
I20260709 12:23:49.070617  4049 leader_election.cc:304] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:23:49.071439  4280 raft_consensus.cc:2804] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:49.071908  4280 raft_consensus.cc:697] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Becoming Leader. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:23:49.072009  4233 raft_consensus.cc:2468] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 2.
I20260709 12:23:49.072830  4280 consensus_queue.cc:237] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:49.080819  3977 catalog_manager.cc:5697] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da reported cstate change: term changed from 1 to 2, leader changed from 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195) to eb6356fa44624134819afb93643d63da (127.3.89.193). New cstate: current_term: 2 leader_uuid: "eb6356fa44624134819afb93643d63da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:49.140532  4082 raft_consensus.cc:1275] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Refusing update from remote peer 3f0748b6fdcd44dc8601f4ebc846aad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.141990  4304 consensus_queue.cc:1048] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.152384  4158 raft_consensus.cc:1275] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Refusing update from remote peer 3f0748b6fdcd44dc8601f4ebc846aad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.154697  4310 consensus_queue.cc:1048] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.221890  4158 raft_consensus.cc:1275] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.223284  4280 consensus_queue.cc:1048] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.231150  4233 raft_consensus.cc:1275] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.232844  4280 consensus_queue.cc:1048] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.256078  4233 raft_consensus.cc:1275] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Refusing update from remote peer 79d614bf4bc24974baa0c843f2db25c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.257606  4316 consensus_queue.cc:1048] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.266762  4082 raft_consensus.cc:1275] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Refusing update from remote peer 79d614bf4bc24974baa0c843f2db25c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.269251  4281 consensus_queue.cc:1048] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.269812  4158 raft_consensus.cc:1275] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.271538  4280 consensus_queue.cc:1048] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.283212  4233 raft_consensus.cc:1275] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.285068  4367 consensus_queue.cc:1048] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:49.435549  4158 raft_consensus.cc:1275] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.437013  4367 consensus_queue.cc:1048] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.446825  4082 raft_consensus.cc:1275] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Refusing update from remote peer 79d614bf4bc24974baa0c843f2db25c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.448470  4281 consensus_queue.cc:1048] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:49.449178  4233 raft_consensus.cc:1275] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.451642  4280 consensus_queue.cc:1048] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.459782  4231 raft_consensus.cc:1275] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Refusing update from remote peer 79d614bf4bc24974baa0c843f2db25c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.464972  4316 consensus_queue.cc:1048] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:49.580777  4158 raft_consensus.cc:1275] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.582217  4339 consensus_queue.cc:1048] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:49.590356  4233 raft_consensus.cc:1275] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.592299  4339 consensus_queue.cc:1048] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:51.731998  3431 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:23:51.732923  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:51.756065  4082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad"
dest_uuid: "eb6356fa44624134819afb93643d63da"
mode: GRACEFUL
new_leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
 from {username='slave'} at 127.0.0.1:43590
I20260709 12:23:51.756683  4082 raft_consensus.cc:606] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 1 LEADER]: Received request to transfer leadership to TS 3f0748b6fdcd44dc8601f4ebc846aad6
I20260709 12:23:51.759300  4082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216"
dest_uuid: "eb6356fa44624134819afb93643d63da"
mode: GRACEFUL
new_leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
 from {username='slave'} at 127.0.0.1:43590
I20260709 12:23:51.759886  4082 raft_consensus.cc:606] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Received request to transfer leadership to TS 3f0748b6fdcd44dc8601f4ebc846aad6
I20260709 12:23:51.760982  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:23:51.761504  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:52.097996  4378 raft_consensus.cc:993] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da: : Instructing follower 3f0748b6fdcd44dc8601f4ebc846aad6 to start an election
I20260709 12:23:52.098452  4382 raft_consensus.cc:1081] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 1 LEADER]: Signalling peer 3f0748b6fdcd44dc8601f4ebc846aad6 to start an election
I20260709 12:23:52.100123  4233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad"
dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
 from {username='slave'} at 127.0.0.1:35126
I20260709 12:23:52.100625  4233 raft_consensus.cc:493] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.100961  4233 raft_consensus.cc:3060] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.106181  4233 raft_consensus.cc:515] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:52.107942  4233 leader_election.cc:290] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 2 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:52.108932  4082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:23:52.109114  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:23:52.109560  4082 raft_consensus.cc:3055] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:52.109627  4158 raft_consensus.cc:3060] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.110065  4082 raft_consensus.cc:740] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:23:52.110940  4082 consensus_queue.cc:260] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:52.111915  4082 raft_consensus.cc:3060] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:52.115442  4158 raft_consensus.cc:2468] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 2.
I20260709 12:23:52.116940  4198 leader_election.cc:304] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:23:52.117805  4379 raft_consensus.cc:2804] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:52.117898  4082 raft_consensus.cc:2468] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 2.
I20260709 12:23:52.118886  4379 raft_consensus.cc:697] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 LEADER]: Becoming Leader. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:23:52.120072  4379 consensus_queue.cc:237] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:52.130163  3977 catalog_manager.cc:5697] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 reported cstate change: term changed from 1 to 2, leader changed from eb6356fa44624134819afb93643d63da (127.3.89.193) to 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:52.244918  4378 raft_consensus.cc:993] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da: : Instructing follower 3f0748b6fdcd44dc8601f4ebc846aad6 to start an election
I20260709 12:23:52.245368  4382 raft_consensus.cc:1081] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Signalling peer 3f0748b6fdcd44dc8601f4ebc846aad6 to start an election
I20260709 12:23:52.246980  4233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216"
dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
 from {username='slave'} at 127.0.0.1:35126
I20260709 12:23:52.247468  4233 raft_consensus.cc:493] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.247891  4233 raft_consensus.cc:3060] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:52.252063  4233 raft_consensus.cc:515] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:52.253526  4233 leader_election.cc:290] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 3 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:23:52.254810  4082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:23:52.254982  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:23:52.255597  4158 raft_consensus.cc:3060] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:52.255606  4082 raft_consensus.cc:3055] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:52.256283  4082 raft_consensus.cc:740] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:23:52.257747  4082 consensus_queue.cc:260] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:52.259186  4082 raft_consensus.cc:3060] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:52.263718  4158 raft_consensus.cc:2468] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 3.
I20260709 12:23:52.265002  4198 leader_election.cc:304] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:23:52.265923  4379 raft_consensus.cc:2804] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:52.266357  4379 raft_consensus.cc:697] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 LEADER]: Becoming Leader. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:23:52.266753  4082 raft_consensus.cc:2468] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 3.
I20260709 12:23:52.267068  4379 consensus_queue.cc:237] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:23:52.273707  3977 catalog_manager.cc:5697] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 reported cstate change: term changed from 2 to 3, leader changed from eb6356fa44624134819afb93643d63da (127.3.89.193) to 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:52.563174  4082 raft_consensus.cc:1275] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Refusing update from remote peer 3f0748b6fdcd44dc8601f4ebc846aad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:52.564464  4379 consensus_queue.cc:1048] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.573964  4158 raft_consensus.cc:1275] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Refusing update from remote peer 3f0748b6fdcd44dc8601f4ebc846aad6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:52.575608  4379 consensus_queue.cc:1048] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:52.762643  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:52.768628  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:52.769088  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:52.816883  4082 raft_consensus.cc:1275] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Refusing update from remote peer 3f0748b6fdcd44dc8601f4ebc846aad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:52.818185  4379 consensus_queue.cc:1048] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:52.827752  4158 raft_consensus.cc:1275] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Refusing update from remote peer 3f0748b6fdcd44dc8601f4ebc846aad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:52.829274  4379 consensus_queue.cc:1048] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:53.770097  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:53.776734  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:53.777228  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:54.778345  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:54.784466  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:54.784962  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:55.786358  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:55.792198  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:55.792613  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:56.793675  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:56.798822  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:56.799217  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:57.800209  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:57.806331  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:57.806871  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:58.807935  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:58.814178  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:58.814568  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:59.815496  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:59.820686  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:59.821061  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:00.822005  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:00.827772  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:00.828163  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:01.859791  3431 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:24:01.866561  4233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216"
dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
mode: GRACEFUL
new_leader_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:35142
I20260709 12:24:01.867229  4233 raft_consensus.cc:606] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 LEADER]: Received request to transfer leadership to TS eb6356fa44624134819afb93643d63da
I20260709 12:24:01.874670  4233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad"
dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
mode: GRACEFUL
new_leader_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:35142
I20260709 12:24:01.875279  4233 raft_consensus.cc:606] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 LEADER]: Received request to transfer leadership to TS eb6356fa44624134819afb93643d63da
I20260709 12:24:01.883798  4155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243"
dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
mode: GRACEFUL
new_leader_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:51458
I20260709 12:24:01.884245  4155 raft_consensus.cc:606] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 LEADER]: Received request to transfer leadership to TS eb6356fa44624134819afb93643d63da
I20260709 12:24:01.889039  4155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7"
dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
mode: GRACEFUL
new_leader_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:51458
I20260709 12:24:01.889565  4155 raft_consensus.cc:606] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 LEADER]: Received request to transfer leadership to TS eb6356fa44624134819afb93643d63da
I20260709 12:24:01.894485  4155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "272c43daddc341079f8a8af9c2f69e44"
dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
mode: GRACEFUL
new_leader_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:51458
I20260709 12:24:01.894937  4155 raft_consensus.cc:606] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Received request to transfer leadership to TS eb6356fa44624134819afb93643d63da
I20260709 12:24:02.090778  4408 raft_consensus.cc:993] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1: : Instructing follower eb6356fa44624134819afb93643d63da to start an election
I20260709 12:24:02.091423  4409 raft_consensus.cc:1081] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 LEADER]: Signalling peer eb6356fa44624134819afb93643d63da to start an election
I20260709 12:24:02.092923  4082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243"
dest_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:43568
I20260709 12:24:02.093358  4082 raft_consensus.cc:493] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:02.093694  4082 raft_consensus.cc:3060] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.099231  4082 raft_consensus.cc:515] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.100862  4082 leader_election.cc:290] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 3 election: Requested vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:24:02.101953  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:24:02.101929  4155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb76f72e365e4f28a86a3c92ed34d243" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:24:02.102689  4233 raft_consensus.cc:3060] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.102748  4155 raft_consensus.cc:3055] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.103111  4155 raft_consensus.cc:740] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Running, Role: LEADER
I20260709 12:24:02.103909  4155 consensus_queue.cc:260] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.104913  4155 raft_consensus.cc:3060] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.107666  4233 raft_consensus.cc:2468] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 3.
I20260709 12:24:02.108872  4047 leader_election.cc:304] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:24:02.109745  4407 raft_consensus.cc:2804] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:02.110615  4155 raft_consensus.cc:2468] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 3.
I20260709 12:24:02.110752  4407 raft_consensus.cc:697] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 3 LEADER]: Becoming Leader. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:24:02.111840  4407 consensus_queue.cc:237] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.119333  3979 catalog_manager.cc:5697] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da reported cstate change: term changed from 2 to 3, leader changed from 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194) to eb6356fa44624134819afb93643d63da (127.3.89.193). New cstate: current_term: 3 leader_uuid: "eb6356fa44624134819afb93643d63da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:02.160022  4408 raft_consensus.cc:993] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1: : Instructing follower eb6356fa44624134819afb93643d63da to start an election
I20260709 12:24:02.160506  4409 raft_consensus.cc:1081] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Signalling peer eb6356fa44624134819afb93643d63da to start an election
I20260709 12:24:02.162007  4082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "272c43daddc341079f8a8af9c2f69e44"
dest_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:43568
I20260709 12:24:02.162532  4082 raft_consensus.cc:493] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:02.162803  4082 raft_consensus.cc:3060] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.167634  4082 raft_consensus.cc:515] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.169555  4416 raft_consensus.cc:993] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6: : Instructing follower eb6356fa44624134819afb93643d63da to start an election
I20260709 12:24:02.169713  4082 leader_election.cc:290] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 2 election: Requested vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:24:02.170205  4416 raft_consensus.cc:1081] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 LEADER]: Signalling peer eb6356fa44624134819afb93643d63da to start an election
I20260709 12:24:02.170568  4155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272c43daddc341079f8a8af9c2f69e44" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:24:02.170745  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272c43daddc341079f8a8af9c2f69e44" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:24:02.171145  4155 raft_consensus.cc:3055] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.171288  4233 raft_consensus.cc:3060] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.171519  4155 raft_consensus.cc:740] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Running, Role: LEADER
I20260709 12:24:02.171819  4082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216"
dest_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:43562
I20260709 12:24:02.172400  4082 raft_consensus.cc:493] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:02.172797  4082 raft_consensus.cc:3060] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:02.172422  4155 consensus_queue.cc:260] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.173827  4155 raft_consensus.cc:3060] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.177897  4416 raft_consensus.cc:993] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6: : Instructing follower eb6356fa44624134819afb93643d63da to start an election
I20260709 12:24:02.178009  4233 raft_consensus.cc:2468] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 2.
I20260709 12:24:02.178758  4400 raft_consensus.cc:1081] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 LEADER]: Signalling peer eb6356fa44624134819afb93643d63da to start an election
I20260709 12:24:02.179345  4047 leader_election.cc:304] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:24:02.180241  4407 raft_consensus.cc:2804] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:02.180670  4407 raft_consensus.cc:697] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Becoming Leader. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:24:02.180851  4083 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad"
dest_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:43562
I20260709 12:24:02.181452  4083 raft_consensus.cc:493] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:02.181519  4407 consensus_queue.cc:237] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.181876  4083 raft_consensus.cc:3060] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.181874  4082 raft_consensus.cc:515] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.184062  4082 leader_election.cc:290] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 4 election: Requested vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:24:02.184404  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:24:02.184953  4158 raft_consensus.cc:3060] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:02.183571  4155 raft_consensus.cc:2468] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 2.
I20260709 12:24:02.186739  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fd419c8941460eb7f76177a854c216" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:24:02.187297  4233 raft_consensus.cc:3055] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:02.187717  4233 raft_consensus.cc:740] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:24:02.192924  4158 raft_consensus.cc:2468] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 4.
I20260709 12:24:02.193041  4083 raft_consensus.cc:515] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.194659  4049 leader_election.cc:304] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:24:02.195888  4083 leader_election.cc:290] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 3 election: Requested vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:24:02.196714  4410 raft_consensus.cc:2804] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:02.196727  4158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:24:02.197417  4158 raft_consensus.cc:3060] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.199348  4231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:24:02.199965  4231 raft_consensus.cc:3055] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.200636  4231 raft_consensus.cc:740] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:24:02.201992  4231 consensus_queue.cc:260] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.188676  4233 consensus_queue.cc:260] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.203357  4231 raft_consensus.cc:3060] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.203804  4233 raft_consensus.cc:3060] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:02.204629  4158 raft_consensus.cc:2468] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 3.
I20260709 12:24:02.205446  4157 raft_consensus.cc:1240] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Rejecting Update request from peer 3f0748b6fdcd44dc8601f4ebc846aad6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:02.205901  4049 leader_election.cc:304] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79d614bf4bc24974baa0c843f2db25c1, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:24:02.207078  4418 raft_consensus.cc:2804] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:02.210233  4409 raft_consensus.cc:993] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1: : Instructing follower eb6356fa44624134819afb93643d63da to start an election
I20260709 12:24:02.210716  4409 raft_consensus.cc:1081] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 LEADER]: Signalling peer eb6356fa44624134819afb93643d63da to start an election
I20260709 12:24:02.211409  4233 raft_consensus.cc:2468] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 4.
I20260709 12:24:02.212857  4082 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7"
dest_uuid: "eb6356fa44624134819afb93643d63da"
 from {username='slave'} at 127.0.0.1:43568
I20260709 12:24:02.213347  4418 raft_consensus.cc:697] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 3 LEADER]: Becoming Leader. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:24:02.213640  4082 raft_consensus.cc:493] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:02.214066  4082 raft_consensus.cc:3060] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.214514  4418 consensus_queue.cc:237] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.219586  3979 catalog_manager.cc:5697] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da reported cstate change: term changed from 1 to 2, leader changed from 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194) to eb6356fa44624134819afb93643d63da (127.3.89.193). New cstate: current_term: 2 leader_uuid: "eb6356fa44624134819afb93643d63da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:02.221325  4082 raft_consensus.cc:515] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.221935  4410 raft_consensus.cc:697] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 4 LEADER]: Becoming Leader. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:24:02.223624  4410 consensus_queue.cc:237] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.224556  4157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:24:02.225138  4157 raft_consensus.cc:3055] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:02.225492  4157 raft_consensus.cc:740] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Running, Role: LEADER
I20260709 12:24:02.226147  4082 leader_election.cc:290] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 3 election: Requested vote from peers 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:24:02.228195  4157 consensus_queue.cc:260] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.230095  4157 raft_consensus.cc:3060] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.227288  4233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea8b49c57549415cba40427c4cb553e7" candidate_uuid: "eb6356fa44624134819afb93643d63da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:24:02.231338  4233 raft_consensus.cc:3060] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:02.235260  4231 raft_consensus.cc:2468] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 3.
I20260709 12:24:02.240200  4233 raft_consensus.cc:2468] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 3.
I20260709 12:24:02.241560  4047 leader_election.cc:304] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, eb6356fa44624134819afb93643d63da; no voters: 
I20260709 12:24:02.242671  4410 raft_consensus.cc:2804] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:02.243232  4410 raft_consensus.cc:697] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 3 LEADER]: Becoming Leader. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:24:02.242820  3979 catalog_manager.cc:5697] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da reported cstate change: term changed from 3 to 4, leader changed from 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195) to eb6356fa44624134819afb93643d63da (127.3.89.193). New cstate: current_term: 4 leader_uuid: "eb6356fa44624134819afb93643d63da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:02.244108  4157 raft_consensus.cc:2468] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb6356fa44624134819afb93643d63da in term 3.
I20260709 12:24:02.244294  4410 consensus_queue.cc:237] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:02.244354  3979 catalog_manager.cc:5697] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da reported cstate change: term changed from 2 to 3, leader changed from 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195) to eb6356fa44624134819afb93643d63da (127.3.89.193). New cstate: current_term: 3 leader_uuid: "eb6356fa44624134819afb93643d63da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:02.258551  3979 catalog_manager.cc:5697] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da reported cstate change: term changed from 2 to 3, leader changed from 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194) to eb6356fa44624134819afb93643d63da (127.3.89.193). New cstate: current_term: 3 leader_uuid: "eb6356fa44624134819afb93643d63da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:02.591992  4157 raft_consensus.cc:1275] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:02.593515  4410 consensus_queue.cc:1048] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:02.597009  4233 raft_consensus.cc:1275] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:02.598253  4410 consensus_queue.cc:1048] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:02.607024  4155 raft_consensus.cc:1275] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:02.608541  4418 consensus_queue.cc:1048] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:02.621995  4231 raft_consensus.cc:1275] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:02.625372  4410 consensus_queue.cc:1048] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:02.660974  4233 raft_consensus.cc:1275] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 4 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:02.662312  4418 consensus_queue.cc:1048] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:02.671702  4155 raft_consensus.cc:1275] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 4 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:02.673600  4410 consensus_queue.cc:1048] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:02.720773  4233 raft_consensus.cc:1275] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:02.722083  4410 consensus_queue.cc:1048] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:02.730659  4155 raft_consensus.cc:1275] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:02.733045  4407 consensus_queue.cc:1048] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:02.837190  4233 raft_consensus.cc:1275] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:02.838811  4407 consensus_queue.cc:1048] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:02.842247  4155 raft_consensus.cc:1275] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Refusing update from remote peer eb6356fa44624134819afb93643d63da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:02.844285  4407 consensus_queue.cc:1048] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:04.933459  3431 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:24:04.934199  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:04.942960  4082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "272c43daddc341079f8a8af9c2f69e44"
dest_uuid: "eb6356fa44624134819afb93643d63da"
mode: GRACEFUL
new_leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
 from {username='slave'} at 127.0.0.1:43590
I20260709 12:24:04.943608  4082 raft_consensus.cc:606] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Received request to transfer leadership to TS 3f0748b6fdcd44dc8601f4ebc846aad6
I20260709 12:24:04.946570  4082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad"
dest_uuid: "eb6356fa44624134819afb93643d63da"
mode: GRACEFUL
new_leader_uuid: "79d614bf4bc24974baa0c843f2db25c1"
 from {username='slave'} at 127.0.0.1:43590
I20260709 12:24:04.947067  4082 raft_consensus.cc:606] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 3 LEADER]: Received request to transfer leadership to TS 79d614bf4bc24974baa0c843f2db25c1
I20260709 12:24:04.949774  4082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec"
dest_uuid: "eb6356fa44624134819afb93643d63da"
mode: GRACEFUL
new_leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
 from {username='slave'} at 127.0.0.1:43590
I20260709 12:24:04.950309  4082 raft_consensus.cc:606] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Received request to transfer leadership to TS 3f0748b6fdcd44dc8601f4ebc846aad6
I20260709 12:24:04.953293  4082 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f0c9ed3020c344279bcf9b4f9eea78cf"
dest_uuid: "eb6356fa44624134819afb93643d63da"
mode: GRACEFUL
new_leader_uuid: "79d614bf4bc24974baa0c843f2db25c1"
 from {username='slave'} at 127.0.0.1:43590
I20260709 12:24:04.953934  4082 raft_consensus.cc:606] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Received request to transfer leadership to TS 79d614bf4bc24974baa0c843f2db25c1
I20260709 12:24:04.955098  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:24:04.955590  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:05.022868  4418 raft_consensus.cc:993] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da: : Instructing follower 3f0748b6fdcd44dc8601f4ebc846aad6 to start an election
I20260709 12:24:05.023361  4437 raft_consensus.cc:1081] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Signalling peer 3f0748b6fdcd44dc8601f4ebc846aad6 to start an election
I20260709 12:24:05.024709  4231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "272c43daddc341079f8a8af9c2f69e44"
dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
 from {username='slave'} at 127.0.0.1:35126
I20260709 12:24:05.025098  4231 raft_consensus.cc:493] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:05.025368  4231 raft_consensus.cc:3060] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.029740  4231 raft_consensus.cc:515] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:05.031698  4231 leader_election.cc:290] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 3 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:24:05.032446  4082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272c43daddc341079f8a8af9c2f69e44" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:24:05.032675  4155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "272c43daddc341079f8a8af9c2f69e44" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:24:05.033023  4082 raft_consensus.cc:3055] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:05.033214  4155 raft_consensus.cc:3060] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.033341  4082 raft_consensus.cc:740] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:24:05.034205  4082 consensus_queue.cc:260] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:05.035116  4082 raft_consensus.cc:3060] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.038411  4155 raft_consensus.cc:2468] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 3.
I20260709 12:24:05.039469  4082 raft_consensus.cc:2468] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 3.
I20260709 12:24:05.039695  4198 leader_election.cc:304] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:24:05.040586  4400 raft_consensus.cc:2804] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:05.041389  4400 raft_consensus.cc:697] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 LEADER]: Becoming Leader. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:24:05.042208  4400 consensus_queue.cc:237] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:05.045670  4418 raft_consensus.cc:993] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da: : Instructing follower 79d614bf4bc24974baa0c843f2db25c1 to start an election
I20260709 12:24:05.046222  4418 raft_consensus.cc:1081] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Signalling peer 79d614bf4bc24974baa0c843f2db25c1 to start an election
I20260709 12:24:05.047873  4155 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f0c9ed3020c344279bcf9b4f9eea78cf"
dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
 from {username='slave'} at 127.0.0.1:51448
I20260709 12:24:05.048358  4155 raft_consensus.cc:493] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:05.048702  4155 raft_consensus.cc:3060] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.049728  3977 catalog_manager.cc:5697] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 reported cstate change: term changed from 2 to 3, leader changed from eb6356fa44624134819afb93643d63da (127.3.89.193) to 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:05.054406  4155 raft_consensus.cc:515] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:05.057827  4155 leader_election.cc:290] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 3 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:24:05.058418  4231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c9ed3020c344279bcf9b4f9eea78cf" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:24:05.057619  4082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0c9ed3020c344279bcf9b4f9eea78cf" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:24:05.059000  4231 raft_consensus.cc:3060] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.059062  4082 raft_consensus.cc:3055] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:05.059475  4082 raft_consensus.cc:740] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:24:05.061484  4082 consensus_queue.cc:260] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:05.062695  4082 raft_consensus.cc:3060] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.066425  4231 raft_consensus.cc:2468] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 3.
I20260709 12:24:05.067920  4122 leader_election.cc:304] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:24:05.068583  4436 raft_consensus.cc:2804] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:05.069022  4082 raft_consensus.cc:2468] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 3.
I20260709 12:24:05.069883  4436 raft_consensus.cc:697] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 3 LEADER]: Becoming Leader. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Running, Role: LEADER
I20260709 12:24:05.071167  4436 consensus_queue.cc:237] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:05.078989  3977 catalog_manager.cc:5697] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 reported cstate change: term changed from 2 to 3, leader changed from eb6356fa44624134819afb93643d63da (127.3.89.193) to 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194). New cstate: current_term: 3 leader_uuid: "79d614bf4bc24974baa0c843f2db25c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:05.237116  4418 raft_consensus.cc:993] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da: : Instructing follower 79d614bf4bc24974baa0c843f2db25c1 to start an election
I20260709 12:24:05.237547  4418 raft_consensus.cc:1081] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 3 LEADER]: Signalling peer 79d614bf4bc24974baa0c843f2db25c1 to start an election
I20260709 12:24:05.239013  4155 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad"
dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
 from {username='slave'} at 127.0.0.1:51448
I20260709 12:24:05.239544  4155 raft_consensus.cc:493] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:05.239812  4155 raft_consensus.cc:3060] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.244526  4155 raft_consensus.cc:515] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:05.246125  4155 leader_election.cc:290] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 4 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195:41075)
I20260709 12:24:05.246976  4082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:24:05.247386  4231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bcf95b3c4a45708d2e7d3e0a4812ad" candidate_uuid: "79d614bf4bc24974baa0c843f2db25c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
I20260709 12:24:05.247603  4082 raft_consensus.cc:3055] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:05.247938  4082 raft_consensus.cc:740] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:24:05.247938  4231 raft_consensus.cc:3060] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.248878  4082 consensus_queue.cc:260] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:05.250131  4082 raft_consensus.cc:3060] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.255070  4231 raft_consensus.cc:2468] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 4.
I20260709 12:24:05.256162  4122 leader_election.cc:304] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:24:05.256982  4436 raft_consensus.cc:2804] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:05.257128  4082 raft_consensus.cc:2468] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d614bf4bc24974baa0c843f2db25c1 in term 4.
I20260709 12:24:05.257440  4436 raft_consensus.cc:697] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 4 LEADER]: Becoming Leader. State: Replica: 79d614bf4bc24974baa0c843f2db25c1, State: Running, Role: LEADER
I20260709 12:24:05.258718  4436 consensus_queue.cc:237] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:05.284929  3977 catalog_manager.cc:5697] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 reported cstate change: term changed from 3 to 4, leader changed from eb6356fa44624134819afb93643d63da (127.3.89.193) to 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194). New cstate: current_term: 4 leader_uuid: "79d614bf4bc24974baa0c843f2db25c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:05.354801  4437 raft_consensus.cc:993] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da: : Instructing follower 3f0748b6fdcd44dc8601f4ebc846aad6 to start an election
I20260709 12:24:05.355499  4418 raft_consensus.cc:1081] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Signalling peer 3f0748b6fdcd44dc8601f4ebc846aad6 to start an election
I20260709 12:24:05.357175  4231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec"
dest_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6"
 from {username='slave'} at 127.0.0.1:35126
I20260709 12:24:05.357749  4231 raft_consensus.cc:493] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:05.358109  4231 raft_consensus.cc:3060] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.364171  4231 raft_consensus.cc:515] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:05.365829  4231 leader_election.cc:290] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 3 election: Requested vote from peers eb6356fa44624134819afb93643d63da (127.3.89.193:32943), 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941)
I20260709 12:24:05.367501  4082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eb6356fa44624134819afb93643d63da"
I20260709 12:24:05.367472  4155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e26b66edcf304a63b41d9e775bd86cec" candidate_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "79d614bf4bc24974baa0c843f2db25c1"
I20260709 12:24:05.368391  4082 raft_consensus.cc:3055] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:05.368448  4155 raft_consensus.cc:3060] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.368803  4082 raft_consensus.cc:740] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb6356fa44624134819afb93643d63da, State: Running, Role: LEADER
I20260709 12:24:05.369741  4082 consensus_queue.cc:260] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:05.371371  4082 raft_consensus.cc:3060] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:05.377164  4155 raft_consensus.cc:2468] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 3.
I20260709 12:24:05.378777  4198 leader_election.cc:304] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0748b6fdcd44dc8601f4ebc846aad6, 79d614bf4bc24974baa0c843f2db25c1; no voters: 
I20260709 12:24:05.379279  4082 raft_consensus.cc:2468] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0748b6fdcd44dc8601f4ebc846aad6 in term 3.
I20260709 12:24:05.379973  4400 raft_consensus.cc:2804] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:05.380576  4400 raft_consensus.cc:697] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 LEADER]: Becoming Leader. State: Replica: 3f0748b6fdcd44dc8601f4ebc846aad6, State: Running, Role: LEADER
I20260709 12:24:05.381821  4400 consensus_queue.cc:237] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } }
I20260709 12:24:05.390561  3977 catalog_manager.cc:5697] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 reported cstate change: term changed from 2 to 3, leader changed from eb6356fa44624134819afb93643d63da (127.3.89.193) to 3f0748b6fdcd44dc8601f4ebc846aad6 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:05.471014  4231 raft_consensus.cc:1275] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Refusing update from remote peer 79d614bf4bc24974baa0c843f2db25c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:05.472882  4436 consensus_queue.cc:1048] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:05.485999  4082 raft_consensus.cc:1275] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Refusing update from remote peer 79d614bf4bc24974baa0c843f2db25c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:05.487890  4444 consensus_queue.cc:1048] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:05.521435  4082 raft_consensus.cc:1275] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Refusing update from remote peer 3f0748b6fdcd44dc8601f4ebc846aad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:05.522755  4400 consensus_queue.cc:1048] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:05.532384  4155 raft_consensus.cc:1275] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Refusing update from remote peer 3f0748b6fdcd44dc8601f4ebc846aad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:05.533974  4438 consensus_queue.cc:1048] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:05.781996  4082 raft_consensus.cc:1275] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Refusing update from remote peer 3f0748b6fdcd44dc8601f4ebc846aad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:05.783448  4438 consensus_queue.cc:1048] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:05.793479  4155 raft_consensus.cc:1275] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Refusing update from remote peer 3f0748b6fdcd44dc8601f4ebc846aad6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:05.795166  4456 consensus_queue.cc:1048] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d614bf4bc24974baa0c843f2db25c1" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 43941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:05.815507  4082 raft_consensus.cc:1275] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 4 FOLLOWER]: Refusing update from remote peer 79d614bf4bc24974baa0c843f2db25c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:05.816776  4444 consensus_queue.cc:1048] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb6356fa44624134819afb93643d63da" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:05.826354  4231 raft_consensus.cc:1275] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 4 FOLLOWER]: Refusing update from remote peer 79d614bf4bc24974baa0c843f2db25c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:05.828747  4444 consensus_queue.cc:1048] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0748b6fdcd44dc8601f4ebc846aad6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:05.956751  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:05.964738  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:05.965251  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:06.966533  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:06.972371  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:06.972894  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:07.973978  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:07.979839  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:07.980355  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:08.981266  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:08.986516  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:08.986994  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:09.987896  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:09.995517  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:09.996001  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:10.996989  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:11.002466  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:11.002946  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:12.003973  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:12.009970  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:12.010504  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:13.011540  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:13.017658  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:13.018200  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:14.019168  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:14.025189  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:14.025799  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:15.059320  3431 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:24:15.060135  3431 tablet_server.cc:179] TabletServer@127.3.89.193:0 shutting down...
I20260709 12:24:15.091655  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:15.092511  3431 tablet_replica.cc:333] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da: stopping tablet replica
I20260709 12:24:15.093166  3431 raft_consensus.cc:2243] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.093750  3431 raft_consensus.cc:2272] T f0c9ed3020c344279bcf9b4f9eea78cf P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.096483  3431 tablet_replica.cc:333] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da: stopping tablet replica
I20260709 12:24:15.097048  3431 raft_consensus.cc:2243] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:15.097914  3431 raft_consensus.cc:2272] T f2fd419c8941460eb7f76177a854c216 P eb6356fa44624134819afb93643d63da [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.099756  3431 tablet_replica.cc:333] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da: stopping tablet replica
I20260709 12:24:15.100292  3431 raft_consensus.cc:2243] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.100706  3431 raft_consensus.cc:2272] T 272c43daddc341079f8a8af9c2f69e44 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.102501  3431 tablet_replica.cc:333] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da: stopping tablet replica
I20260709 12:24:15.103013  3431 raft_consensus.cc:2243] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:15.103724  3431 raft_consensus.cc:2272] T 974f32885718400cb4c83cf4a8689d3b P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.105422  3431 tablet_replica.cc:333] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da: stopping tablet replica
I20260709 12:24:15.105967  3431 raft_consensus.cc:2243] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:15.106828  3431 raft_consensus.cc:2272] T ea8b49c57549415cba40427c4cb553e7 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.108613  3431 tablet_replica.cc:333] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da: stopping tablet replica
I20260709 12:24:15.109125  3431 raft_consensus.cc:2243] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.109530  3431 raft_consensus.cc:2272] T e26b66edcf304a63b41d9e775bd86cec P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.111119  3431 tablet_replica.cc:333] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da: stopping tablet replica
I20260709 12:24:15.111698  3431 raft_consensus.cc:2243] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.112147  3431 raft_consensus.cc:2272] T 91bcf95b3c4a45708d2e7d3e0a4812ad P eb6356fa44624134819afb93643d63da [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.113884  3431 tablet_replica.cc:333] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da: stopping tablet replica
I20260709 12:24:15.114389  3431 raft_consensus.cc:2243] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.114791  3431 raft_consensus.cc:2272] T e26f47e7ee0346538108cf9b1350bd5d P eb6356fa44624134819afb93643d63da [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.116489  3431 tablet_replica.cc:333] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da: stopping tablet replica
I20260709 12:24:15.116961  3431 raft_consensus.cc:2243] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:15.117789  3431 raft_consensus.cc:2272] T bb76f72e365e4f28a86a3c92ed34d243 P eb6356fa44624134819afb93643d63da [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.119385  3431 tablet_replica.cc:333] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da: stopping tablet replica
I20260709 12:24:15.119825  3431 raft_consensus.cc:2243] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.120280  3431 raft_consensus.cc:2272] T 3e3a70bf60d1483cbfadcdeae5b564a6 P eb6356fa44624134819afb93643d63da [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:15.122627  4124 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
W20260709 12:24:15.126260  4124 consensus_peers.cc:597] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 -> Peer eb6356fa44624134819afb93643d63da (127.3.89.193:32943): Couldn't send request to peer eb6356fa44624134819afb93643d63da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:15.151696  3431 tablet_server.cc:196] TabletServer@127.3.89.193:0 shutdown complete.
I20260709 12:24:15.170887  3431 tablet_server.cc:179] TabletServer@127.3.89.194:0 shutting down...
W20260709 12:24:15.188459  4124 consensus_peers.cc:597] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 -> Peer eb6356fa44624134819afb93643d63da (127.3.89.193:32943): Couldn't send request to peer eb6356fa44624134819afb93643d63da. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:32943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:15.200707  4198 consensus_peers.cc:597] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 -> Peer 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941): Couldn't send request to peer 79d614bf4bc24974baa0c843f2db25c1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:15.207723  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:15.208591  3431 tablet_replica.cc:333] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1: stopping tablet replica
I20260709 12:24:15.209291  3431 raft_consensus.cc:2243] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.209911  3431 raft_consensus.cc:2272] T e26f47e7ee0346538108cf9b1350bd5d P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.212193  3431 tablet_replica.cc:333] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1: stopping tablet replica
I20260709 12:24:15.212900  3431 raft_consensus.cc:2243] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:15.213812  3431 raft_consensus.cc:2272] T f0c9ed3020c344279bcf9b4f9eea78cf P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.216645  3431 tablet_replica.cc:333] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1: stopping tablet replica
I20260709 12:24:15.217530  3431 raft_consensus.cc:2243] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:24:15.218103  4198 consensus_peers.cc:597] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 -> Peer eb6356fa44624134819afb93643d63da (127.3.89.193:32943): Couldn't send request to peer eb6356fa44624134819afb93643d63da. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:32943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:15.218258  3431 raft_consensus.cc:2272] T ea8b49c57549415cba40427c4cb553e7 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.220508  3431 tablet_replica.cc:333] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1: stopping tablet replica
I20260709 12:24:15.221032  3431 raft_consensus.cc:2243] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:15.221843  3431 raft_consensus.cc:2272] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 79d614bf4bc24974baa0c843f2db25c1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.223850  3431 tablet_replica.cc:333] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1: stopping tablet replica
I20260709 12:24:15.224458  3431 raft_consensus.cc:2243] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.224881  3431 raft_consensus.cc:2272] T e26b66edcf304a63b41d9e775bd86cec P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.226771  3431 tablet_replica.cc:333] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1: stopping tablet replica
I20260709 12:24:15.227337  3431 raft_consensus.cc:2243] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.227742  3431 raft_consensus.cc:2272] T f2fd419c8941460eb7f76177a854c216 P 79d614bf4bc24974baa0c843f2db25c1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.229655  3431 tablet_replica.cc:333] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1: stopping tablet replica
I20260709 12:24:15.230147  3431 raft_consensus.cc:2243] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:15.230888  3431 raft_consensus.cc:2272] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 79d614bf4bc24974baa0c843f2db25c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.232820  3431 tablet_replica.cc:333] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1: stopping tablet replica
I20260709 12:24:15.233323  3431 raft_consensus.cc:2243] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.233814  3431 raft_consensus.cc:2272] T bb76f72e365e4f28a86a3c92ed34d243 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.236217  3431 tablet_replica.cc:333] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1: stopping tablet replica
I20260709 12:24:15.236891  3431 raft_consensus.cc:2243] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.237514  3431 raft_consensus.cc:2272] T 272c43daddc341079f8a8af9c2f69e44 P 79d614bf4bc24974baa0c843f2db25c1 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:15.238837  4198 consensus_peers.cc:597] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 -> Peer eb6356fa44624134819afb93643d63da (127.3.89.193:32943): Couldn't send request to peer eb6356fa44624134819afb93643d63da. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:32943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:15.239683  3431 tablet_replica.cc:333] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1: stopping tablet replica
I20260709 12:24:15.240221  3431 raft_consensus.cc:2243] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.240818  3431 raft_consensus.cc:2272] T 974f32885718400cb4c83cf4a8689d3b P 79d614bf4bc24974baa0c843f2db25c1 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:15.270433  4198 consensus_peers.cc:597] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 -> Peer 79d614bf4bc24974baa0c843f2db25c1 (127.3.89.194:43941): Couldn't send request to peer 79d614bf4bc24974baa0c843f2db25c1. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:43941: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:15.273102  3431 tablet_server.cc:196] TabletServer@127.3.89.194:0 shutdown complete.
I20260709 12:24:15.292851  3431 tablet_server.cc:179] TabletServer@127.3.89.195:0 shutting down...
I20260709 12:24:15.346266  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:15.347123  3431 tablet_replica.cc:333] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6: stopping tablet replica
I20260709 12:24:15.348619  3431 raft_consensus.cc:2243] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:15.350199  3431 raft_consensus.cc:2272] T e26f47e7ee0346538108cf9b1350bd5d P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.352733  3431 tablet_replica.cc:333] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 3f0748b6fdcd44dc8601f4ebc846aad6: stopping tablet replica
I20260709 12:24:15.353546  3431 raft_consensus.cc:2243] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.354228  3431 raft_consensus.cc:2272] T 3e3a70bf60d1483cbfadcdeae5b564a6 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.356509  3431 tablet_replica.cc:333] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6: stopping tablet replica
I20260709 12:24:15.357147  3431 raft_consensus.cc:2243] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.357842  3431 raft_consensus.cc:2272] T bb76f72e365e4f28a86a3c92ed34d243 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.359928  3431 tablet_replica.cc:333] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6: stopping tablet replica
I20260709 12:24:15.360525  3431 raft_consensus.cc:2243] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:15.362033  3431 raft_consensus.cc:2272] T e26b66edcf304a63b41d9e775bd86cec P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.364348  3431 tablet_replica.cc:333] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6: stopping tablet replica
I20260709 12:24:15.364981  3431 raft_consensus.cc:2243] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.365691  3431 raft_consensus.cc:2272] T 91bcf95b3c4a45708d2e7d3e0a4812ad P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.367676  3431 tablet_replica.cc:333] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6: stopping tablet replica
I20260709 12:24:15.368223  3431 raft_consensus.cc:2243] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.369046  3431 raft_consensus.cc:2272] T ea8b49c57549415cba40427c4cb553e7 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.371158  3431 tablet_replica.cc:333] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6: stopping tablet replica
I20260709 12:24:15.371730  3431 raft_consensus.cc:2243] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.372251  3431 raft_consensus.cc:2272] T f2fd419c8941460eb7f76177a854c216 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.374241  3431 tablet_replica.cc:333] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6: stopping tablet replica
I20260709 12:24:15.374775  3431 raft_consensus.cc:2243] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.375330  3431 raft_consensus.cc:2272] T f0c9ed3020c344279bcf9b4f9eea78cf P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.377108  3431 tablet_replica.cc:333] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6: stopping tablet replica
I20260709 12:24:15.377673  3431 raft_consensus.cc:2243] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:15.379038  3431 raft_consensus.cc:2272] T 272c43daddc341079f8a8af9c2f69e44 P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.380892  3431 tablet_replica.cc:333] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6: stopping tablet replica
I20260709 12:24:15.381366  3431 raft_consensus.cc:2243] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:15.381865  3431 raft_consensus.cc:2272] T 974f32885718400cb4c83cf4a8689d3b P 3f0748b6fdcd44dc8601f4ebc846aad6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.411679  3431 tablet_server.cc:196] TabletServer@127.3.89.195:0 shutdown complete.
I20260709 12:24:15.435750  3431 master.cc:562] Master@127.3.89.254:39031 shutting down...
I20260709 12:24:15.462024  3431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:15.462713  3431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:15.463061  3431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3d73f714f61942ff89321862d15c8307: stopping tablet replica
I20260709 12:24:15.483228  3431 master.cc:584] Master@127.3.89.254:39031 shutdown complete.
3f0748b6fdcd44dc8601f4ebc846aad6  1
79d614bf4bc24974baa0c843f2db25c1  4
eb6356fa44624134819afb93643d63da  5
3f0748b6fdcd44dc8601f4ebc846aad6  3
79d614bf4bc24974baa0c843f2db25c1  4
eb6356fa44624134819afb93643d63da  3
3f0748b6fdcd44dc8601f4ebc846aad6  1
79d614bf4bc24974baa0c843f2db25c1  1
eb6356fa44624134819afb93643d63da  8
3f0748b6fdcd44dc8601f4ebc846aad6  3
79d614bf4bc24974baa0c843f2db25c1  3
eb6356fa44624134819afb93643d63da  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (29335 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:24:15.527920  3431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.3.89.254:36317
I20260709 12:24:15.529074  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:15.534971  4476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:15.535071  4477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:15.536028  4479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:15.537408  3431 server_base.cc:1061] running on GCE node
I20260709 12:24:15.538542  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:15.538731  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:15.538911  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783599855538896 us; error 0 us; skew 500 ppm
I20260709 12:24:15.539497  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:15.542420  3431 webserver.cc:533] Webserver started at http://127.3.89.254:34569/ using document root <none> and password file <none>
I20260709 12:24:15.542930  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:15.543095  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:15.543438  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:15.544720  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/master-0-root/instance:
uuid: "edba2778b94f4e5aa07d5f710879b375"
format_stamp: "Formatted at 2026-07-09 12:24:15 on dist-test-slave-3hrd"
I20260709 12:24:15.549149  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:24:15.552594  4484 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:15.553356  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:24:15.553732  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/master-0-root
uuid: "edba2778b94f4e5aa07d5f710879b375"
format_stamp: "Formatted at 2026-07-09 12:24:15 on dist-test-slave-3hrd"
I20260709 12:24:15.554037  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:15.579334  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:15.580628  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:15.622255  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:36317
I20260709 12:24:15.622349  4536 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.254:36317 every 8 connection(s)
I20260709 12:24:15.626367  4537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:15.636315  4537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Bootstrap starting.
I20260709 12:24:15.641091  4537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:15.645464  4537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: No bootstrap required, opened a new log
I20260709 12:24:15.647755  4537 raft_consensus.cc:359] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edba2778b94f4e5aa07d5f710879b375" member_type: VOTER }
I20260709 12:24:15.648177  4537 raft_consensus.cc:385] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:15.648468  4537 raft_consensus.cc:740] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edba2778b94f4e5aa07d5f710879b375, State: Initialized, Role: FOLLOWER
I20260709 12:24:15.649024  4537 consensus_queue.cc:260] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edba2778b94f4e5aa07d5f710879b375" member_type: VOTER }
I20260709 12:24:15.649513  4537 raft_consensus.cc:399] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:24:15.649773  4537 raft_consensus.cc:493] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:24:15.650025  4537 raft_consensus.cc:3060] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:15.654273  4537 raft_consensus.cc:515] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edba2778b94f4e5aa07d5f710879b375" member_type: VOTER }
I20260709 12:24:15.654803  4537 leader_election.cc:304] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edba2778b94f4e5aa07d5f710879b375; no voters: 
I20260709 12:24:15.656093  4537 leader_election.cc:290] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:24:15.656544  4540 raft_consensus.cc:2804] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:15.658098  4540 raft_consensus.cc:697] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [term 1 LEADER]: Becoming Leader. State: Replica: edba2778b94f4e5aa07d5f710879b375, State: Running, Role: LEADER
I20260709 12:24:15.659049  4537 sys_catalog.cc:565] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:24:15.658895  4540 consensus_queue.cc:237] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edba2778b94f4e5aa07d5f710879b375" member_type: VOTER }
I20260709 12:24:15.664722  4542 sys_catalog.cc:455] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edba2778b94f4e5aa07d5f710879b375. Latest consensus state: current_term: 1 leader_uuid: "edba2778b94f4e5aa07d5f710879b375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edba2778b94f4e5aa07d5f710879b375" member_type: VOTER } }
I20260709 12:24:15.665588  4542 sys_catalog.cc:458] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [sys.catalog]: This master's current role is: LEADER
I20260709 12:24:15.665952  4541 sys_catalog.cc:455] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edba2778b94f4e5aa07d5f710879b375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edba2778b94f4e5aa07d5f710879b375" member_type: VOTER } }
I20260709 12:24:15.666772  4541 sys_catalog.cc:458] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [sys.catalog]: This master's current role is: LEADER
I20260709 12:24:15.670336  4549 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:24:15.678112  4549 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:24:15.679165  3431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:24:15.687572  4549 catalog_manager.cc:1383] Generated new cluster ID: f830288060894f708e956f5c83f1ea09
I20260709 12:24:15.687873  4549 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:24:15.718168  4549 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:24:15.719445  4549 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:24:15.740996  4549 catalog_manager.cc:6070] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Generated new TSK 0
I20260709 12:24:15.741751  4549 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:24:15.810190  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:15.816893  4559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:15.817970  4560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:15.820523  3431 server_base.cc:1061] running on GCE node
W20260709 12:24:15.820609  4562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:15.821702  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:15.821879  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:15.822103  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783599855822068 us; error 0 us; skew 500 ppm
I20260709 12:24:15.822656  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:15.825029  3431 webserver.cc:533] Webserver started at http://127.3.89.193:37997/ using document root <none> and password file <none>
I20260709 12:24:15.825538  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:15.825796  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:15.826113  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:15.827338  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-0-root/instance:
uuid: "5c3201048f8346009aefca07428828f3"
format_stamp: "Formatted at 2026-07-09 12:24:15 on dist-test-slave-3hrd"
I20260709 12:24:15.831794  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:24:15.835193  4567 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:15.835983  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:24:15.836265  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-0-root
uuid: "5c3201048f8346009aefca07428828f3"
format_stamp: "Formatted at 2026-07-09 12:24:15 on dist-test-slave-3hrd"
I20260709 12:24:15.836570  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:15.854130  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:15.855381  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:15.856921  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:15.859217  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:15.859391  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:15.859680  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:15.859858  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:15.900413  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:45077
I20260709 12:24:15.900514  4630 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.193:45077 every 8 connection(s)
I20260709 12:24:15.905594  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:15.913182  4635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:15.913293  4636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:15.916684  4638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:15.917470  3431 server_base.cc:1061] running on GCE node
I20260709 12:24:15.918927  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:15.919162  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:15.919342  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783599855919319 us; error 0 us; skew 500 ppm
I20260709 12:24:15.919868  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:15.923163  3431 webserver.cc:533] Webserver started at http://127.3.89.194:39201/ using document root <none> and password file <none>
I20260709 12:24:15.923688  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:15.923930  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:15.924173  4631 heartbeater.cc:344] Connected to a master server at 127.3.89.254:36317
I20260709 12:24:15.924238  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:15.924716  4631 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:15.925436  4631 heartbeater.cc:507] Master 127.3.89.254:36317 requested a full tablet report, sending...
I20260709 12:24:15.925963  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root/instance:
uuid: "3a5aeedd11fe49938079fe86cb01232c"
format_stamp: "Formatted at 2026-07-09 12:24:15 on dist-test-slave-3hrd"
I20260709 12:24:15.927546  4501 ts_manager.cc:194] Registered new tserver with Master: 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:24:15.929250  4501 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52808
I20260709 12:24:15.930877  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:24:15.934599  4643 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:15.935385  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 12:24:15.935680  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root
uuid: "3a5aeedd11fe49938079fe86cb01232c"
format_stamp: "Formatted at 2026-07-09 12:24:15 on dist-test-slave-3hrd"
I20260709 12:24:15.935981  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:15.950958  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:15.952172  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:15.953715  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:15.956059  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:15.956314  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:15.956554  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:15.956722  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:15.998888  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:42599
I20260709 12:24:15.998992  4706 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.194:42599 every 8 connection(s)
I20260709 12:24:16.003701  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:16.012180  4710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:16.012393  4711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:16.017262  4713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:16.018153  4707 heartbeater.cc:344] Connected to a master server at 127.3.89.254:36317
I20260709 12:24:16.018702  4707 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:16.019554  4707 heartbeater.cc:507] Master 127.3.89.254:36317 requested a full tablet report, sending...
I20260709 12:24:16.019927  3431 server_base.cc:1061] running on GCE node
I20260709 12:24:16.021054  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:16.021309  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:16.021479  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783599856021467 us; error 0 us; skew 500 ppm
I20260709 12:24:16.022099  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:16.022061  4501 ts_manager.cc:194] Registered new tserver with Master: 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:16.023859  4501 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52812
I20260709 12:24:16.024940  3431 webserver.cc:533] Webserver started at http://127.3.89.195:37553/ using document root <none> and password file <none>
I20260709 12:24:16.025480  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:16.025748  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:16.026031  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:16.027290  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root/instance:
uuid: "49277a7653524787a715232ab908bbc6"
format_stamp: "Formatted at 2026-07-09 12:24:16 on dist-test-slave-3hrd"
I20260709 12:24:16.031755  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:24:16.035086  4718 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:16.035832  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:24:16.036151  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root
uuid: "49277a7653524787a715232ab908bbc6"
format_stamp: "Formatted at 2026-07-09 12:24:16 on dist-test-slave-3hrd"
I20260709 12:24:16.036458  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:16.052587  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:16.053848  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:16.055120  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:16.057431  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:16.057683  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.001s
I20260709 12:24:16.057922  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:16.058101  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:16.098567  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:41049
I20260709 12:24:16.098671  4780 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.195:41049 every 8 connection(s)
I20260709 12:24:16.113195  4781 heartbeater.cc:344] Connected to a master server at 127.3.89.254:36317
I20260709 12:24:16.113653  4781 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:16.114378  4781 heartbeater.cc:507] Master 127.3.89.254:36317 requested a full tablet report, sending...
I20260709 12:24:16.116417  4501 ts_manager.cc:194] Registered new tserver with Master: 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:16.116863  3431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01494374s
I20260709 12:24:16.117846  4501 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52818
I20260709 12:24:16.143543  4501 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52822:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:24:16.150468  4501 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:24:16.362517  4746 tablet_service.cc:1511] Processing CreateTablet for tablet ab695b6fa8e145c7926a540c3ba5b84e (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:24:16.363611  4744 tablet_service.cc:1511] Processing CreateTablet for tablet 23027d24ef1943bdbea76085a97b055c (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:24:16.365612  4595 tablet_service.cc:1511] Processing CreateTablet for tablet ab695b6fa8e145c7926a540c3ba5b84e (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:24:16.366055  4592 tablet_service.cc:1511] Processing CreateTablet for tablet 86297ef907dd4e3590805d3cf15efaca (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:24:16.366447  4745 tablet_service.cc:1511] Processing CreateTablet for tablet 4f682ea56c3b4f2bb6e9413f3d98c6fd (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:24:16.367168  4595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab695b6fa8e145c7926a540c3ba5b84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.367854  4592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86297ef907dd4e3590805d3cf15efaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.369046  4593 tablet_service.cc:1511] Processing CreateTablet for tablet 23027d24ef1943bdbea76085a97b055c (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:24:16.370523  4593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23027d24ef1943bdbea76085a97b055c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.374866  4594 tablet_service.cc:1511] Processing CreateTablet for tablet 4f682ea56c3b4f2bb6e9413f3d98c6fd (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:24:16.376417  4594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f682ea56c3b4f2bb6e9413f3d98c6fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.376549  4743 tablet_service.cc:1511] Processing CreateTablet for tablet 86297ef907dd4e3590805d3cf15efaca (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:24:16.364143  4746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab695b6fa8e145c7926a540c3ba5b84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.377668  4591 tablet_service.cc:1511] Processing CreateTablet for tablet abf423ac214b42acbe1696a11279567c (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:24:16.379161  4591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abf423ac214b42acbe1696a11279567c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.379055  4742 tablet_service.cc:1511] Processing CreateTablet for tablet abf423ac214b42acbe1696a11279567c (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:24:16.380437  4742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abf423ac214b42acbe1696a11279567c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.381369  4744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23027d24ef1943bdbea76085a97b055c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.384624  4741 tablet_service.cc:1511] Processing CreateTablet for tablet 8e4e6edc86e64b8ca5e81a1f1c312757 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:24:16.386056  4741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e4e6edc86e64b8ca5e81a1f1c312757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.386695  4743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86297ef907dd4e3590805d3cf15efaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.388118  4745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f682ea56c3b4f2bb6e9413f3d98c6fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.388906  4740 tablet_service.cc:1511] Processing CreateTablet for tablet dafe783bf87845d6bb64a279ed7d0f75 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:24:16.392557  4739 tablet_service.cc:1511] Processing CreateTablet for tablet 96cc733643644a9ca6f1b2613e987020 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:24:16.402876  4738 tablet_service.cc:1511] Processing CreateTablet for tablet f54c73ae37134549b12188dddc4fa794 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:24:16.404510  4738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54c73ae37134549b12188dddc4fa794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.407948  4590 tablet_service.cc:1511] Processing CreateTablet for tablet 8e4e6edc86e64b8ca5e81a1f1c312757 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:24:16.410825  4737 tablet_service.cc:1511] Processing CreateTablet for tablet 18c2a1115a9a4b4a975dadb63bef992a (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:24:16.412256  4737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18c2a1115a9a4b4a975dadb63bef992a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.409404  4590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e4e6edc86e64b8ca5e81a1f1c312757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.425124  4740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dafe783bf87845d6bb64a279ed7d0f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.429971  4739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96cc733643644a9ca6f1b2613e987020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.439029  4588 tablet_service.cc:1511] Processing CreateTablet for tablet 96cc733643644a9ca6f1b2613e987020 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:24:16.440515  4588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96cc733643644a9ca6f1b2613e987020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.443043  4587 tablet_service.cc:1511] Processing CreateTablet for tablet f54c73ae37134549b12188dddc4fa794 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:24:16.449708  4671 tablet_service.cc:1511] Processing CreateTablet for tablet ab695b6fa8e145c7926a540c3ba5b84e (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:24:16.451241  4671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab695b6fa8e145c7926a540c3ba5b84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.452601  4594 tablet_service.cc:1511] Processing CreateTablet for tablet c3d4ce22fd7947598cc1d1c7732285da (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:24:16.455037  4670 tablet_service.cc:1511] Processing CreateTablet for tablet 4f682ea56c3b4f2bb6e9413f3d98c6fd (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:24:16.456558  4670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f682ea56c3b4f2bb6e9413f3d98c6fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.460336  4668 tablet_service.cc:1511] Processing CreateTablet for tablet 86297ef907dd4e3590805d3cf15efaca (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:24:16.463650  4667 tablet_service.cc:1511] Processing CreateTablet for tablet abf423ac214b42acbe1696a11279567c (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:24:16.466768  4666 tablet_service.cc:1511] Processing CreateTablet for tablet 8e4e6edc86e64b8ca5e81a1f1c312757 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:24:16.469589  4669 tablet_service.cc:1511] Processing CreateTablet for tablet 23027d24ef1943bdbea76085a97b055c (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:24:16.470763  4589 tablet_service.cc:1511] Processing CreateTablet for tablet dafe783bf87845d6bb64a279ed7d0f75 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:24:16.472344  4586 tablet_service.cc:1511] Processing CreateTablet for tablet 18c2a1115a9a4b4a975dadb63bef992a (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:24:16.474778  4668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86297ef907dd4e3590805d3cf15efaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.480440  4667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abf423ac214b42acbe1696a11279567c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.483451  4666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e4e6edc86e64b8ca5e81a1f1c312757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.486400  4669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23027d24ef1943bdbea76085a97b055c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.488952  4589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dafe783bf87845d6bb64a279ed7d0f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.490002  4587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54c73ae37134549b12188dddc4fa794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.490868  4586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18c2a1115a9a4b4a975dadb63bef992a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.496595  4796 tablet_bootstrap.cc:492] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:16.501924  4594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d4ce22fd7947598cc1d1c7732285da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.511586  4665 tablet_service.cc:1511] Processing CreateTablet for tablet dafe783bf87845d6bb64a279ed7d0f75 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:24:16.513206  4665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dafe783bf87845d6bb64a279ed7d0f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.515429  4664 tablet_service.cc:1511] Processing CreateTablet for tablet 96cc733643644a9ca6f1b2613e987020 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:24:16.515682  4746 tablet_service.cc:1511] Processing CreateTablet for tablet c3d4ce22fd7947598cc1d1c7732285da (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:24:16.517019  4746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d4ce22fd7947598cc1d1c7732285da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.519955  4592 tablet_service.cc:1511] Processing CreateTablet for tablet b4a575b75e05458186c89873e1e7622b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:24:16.521147  4664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96cc733643644a9ca6f1b2613e987020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.522661  4591 tablet_service.cc:1511] Processing CreateTablet for tablet 8d9a7637095341feabd700d53579d5d9 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:24:16.523862  4591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d9a7637095341feabd700d53579d5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.525215  4590 tablet_service.cc:1511] Processing CreateTablet for tablet 13ebb8f343c743a1a28db116a4881004 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:24:16.535306  4663 tablet_service.cc:1511] Processing CreateTablet for tablet f54c73ae37134549b12188dddc4fa794 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:24:16.537297  4663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54c73ae37134549b12188dddc4fa794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.546924  4586 tablet_service.cc:1511] Processing CreateTablet for tablet c6f49e7bc416420cb85a6d4319f46c4d (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:24:16.548745  4796 tablet_bootstrap.cc:654] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:16.559065  4745 tablet_service.cc:1511] Processing CreateTablet for tablet 8d9a7637095341feabd700d53579d5d9 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:24:16.577487  4745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d9a7637095341feabd700d53579d5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.561322  4741 tablet_service.cc:1511] Processing CreateTablet for tablet c6f49e7bc416420cb85a6d4319f46c4d (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:24:16.561681  4743 tablet_service.cc:1511] Processing CreateTablet for tablet b4a575b75e05458186c89873e1e7622b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:24:16.562558  4662 tablet_service.cc:1511] Processing CreateTablet for tablet 18c2a1115a9a4b4a975dadb63bef992a (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:24:16.582974  4662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18c2a1115a9a4b4a975dadb63bef992a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.561323  4744 tablet_service.cc:1511] Processing CreateTablet for tablet 13ebb8f343c743a1a28db116a4881004 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:24:16.569785  4590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ebb8f343c743a1a28db116a4881004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.582955  4741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6f49e7bc416420cb85a6d4319f46c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.590029  4586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6f49e7bc416420cb85a6d4319f46c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.590570  4737 tablet_service.cc:1511] Processing CreateTablet for tablet 1d3124fc8af34f378ebd9cff21ee0e6b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:24:16.592391  4737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3124fc8af34f378ebd9cff21ee0e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.593906  4743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4a575b75e05458186c89873e1e7622b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.599501  4744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ebb8f343c743a1a28db116a4881004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.606676  4797 tablet_bootstrap.cc:492] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:16.608590  4738 tablet_service.cc:1511] Processing CreateTablet for tablet 3e20c8bae3534224bb7a82db42cab1a5 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:24:16.612449  4592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4a575b75e05458186c89873e1e7622b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.622962  4740 tablet_service.cc:1511] Processing CreateTablet for tablet 7cc90257afed46488d4574c06b22a1a4 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:24:16.627483  4595 tablet_service.cc:1511] Processing CreateTablet for tablet 1d3124fc8af34f378ebd9cff21ee0e6b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:24:16.628930  4595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3124fc8af34f378ebd9cff21ee0e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.630491  4593 tablet_service.cc:1511] Processing CreateTablet for tablet e8a74137ba104fe0b465cab6399c4940 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:24:16.631726  4593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8a74137ba104fe0b465cab6399c4940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.635193  4738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e20c8bae3534224bb7a82db42cab1a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.642478  4739 tablet_service.cc:1511] Processing CreateTablet for tablet e8a74137ba104fe0b465cab6399c4940 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:24:16.643824  4739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8a74137ba104fe0b465cab6399c4940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.669945  4740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cc90257afed46488d4574c06b22a1a4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:16.672245  4550 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:16.675299  4588 tablet_service.cc:1511] Processing CreateTablet for tablet 70fb561fb8b446969622ee2193d90917 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:24:16.676313  4742 tablet_service.cc:1511] Processing CreateTablet for tablet 70fb561fb8b446969622ee2193d90917 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:24:16.676828  4588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70fb561fb8b446969622ee2193d90917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.685812  4742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70fb561fb8b446969622ee2193d90917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.689646  4662 tablet_service.cc:1511] Processing CreateTablet for tablet c3d4ce22fd7947598cc1d1c7732285da (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:24:16.691059  4662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d4ce22fd7947598cc1d1c7732285da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.691118  4796 tablet_bootstrap.cc:492] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:16.690853  4594 tablet_service.cc:1511] Processing CreateTablet for tablet 3e20c8bae3534224bb7a82db42cab1a5 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:24:16.691713  4796 ts_tablet_manager.cc:1403] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.196s	user 0.007s	sys 0.018s
I20260709 12:24:16.692272  4797 tablet_bootstrap.cc:654] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:16.694701  4796 raft_consensus.cc:359] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } }
I20260709 12:24:16.695429  4796 raft_consensus.cc:385] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:16.695758  4796 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:16.696691  4796 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } }
I20260709 12:24:16.699046  4669 tablet_service.cc:1511] Processing CreateTablet for tablet 8d9a7637095341feabd700d53579d5d9 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:24:16.700428  4669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d9a7637095341feabd700d53579d5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.701856  4587 tablet_service.cc:1511] Processing CreateTablet for tablet 7cc90257afed46488d4574c06b22a1a4 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:24:16.692302  4594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e20c8bae3534224bb7a82db42cab1a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.708333  4799 tablet_bootstrap.cc:492] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:16.708194  4744 tablet_service.cc:1511] Processing CreateTablet for tablet 4d675fbac0f844dc9309a662ba1c96f9 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:24:16.709770  4744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d675fbac0f844dc9309a662ba1c96f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.718812  4738 tablet_service.cc:1511] Processing CreateTablet for tablet 559316f720c74aa1b50405e2ed791f6a (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:24:16.720559  4738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559316f720c74aa1b50405e2ed791f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.723826  4589 tablet_service.cc:1511] Processing CreateTablet for tablet 4d675fbac0f844dc9309a662ba1c96f9 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:24:16.725194  4589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d675fbac0f844dc9309a662ba1c96f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.734845  4671 tablet_service.cc:1511] Processing CreateTablet for tablet c6f49e7bc416420cb85a6d4319f46c4d (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:24:16.740262  4590 tablet_service.cc:1511] Processing CreateTablet for tablet 26f80146cffc4af2ac0af76b3a99c213 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:24:16.729444  4593 tablet_service.cc:1511] Processing CreateTablet for tablet 559316f720c74aa1b50405e2ed791f6a (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:24:16.748917  4668 tablet_service.cc:1511] Processing CreateTablet for tablet 70fb561fb8b446969622ee2193d90917 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:24:16.751850  4746 tablet_service.cc:1511] Processing CreateTablet for tablet 26f80146cffc4af2ac0af76b3a99c213 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:24:16.786985  4746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26f80146cffc4af2ac0af76b3a99c213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.786267  4667 tablet_service.cc:1511] Processing CreateTablet for tablet 1d3124fc8af34f378ebd9cff21ee0e6b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:24:16.762022  4664 tablet_service.cc:1511] Processing CreateTablet for tablet b4a575b75e05458186c89873e1e7622b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:24:16.721102  4666 tablet_service.cc:1511] Processing CreateTablet for tablet 13ebb8f343c743a1a28db116a4881004 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:24:16.766916  4670 tablet_service.cc:1511] Processing CreateTablet for tablet 7cc90257afed46488d4574c06b22a1a4 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:24:16.771874  4663 tablet_service.cc:1511] Processing CreateTablet for tablet 3e20c8bae3534224bb7a82db42cab1a5 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:24:16.772805  4593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559316f720c74aa1b50405e2ed791f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.794734  4590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26f80146cffc4af2ac0af76b3a99c213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.772660  4669 tablet_service.cc:1511] Processing CreateTablet for tablet e8a74137ba104fe0b465cab6399c4940 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:24:16.779774  4797 tablet_bootstrap.cc:492] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:16.781628  4671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6f49e7bc416420cb85a6d4319f46c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.780881  4588 tablet_service.cc:1511] Processing CreateTablet for tablet b8e62ff81eb34853811314530131292b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:24:16.782467  4781 heartbeater.cc:499] Master 127.3.89.254:36317 was elected leader, sending a full tablet report...
I20260709 12:24:16.783749  4796 ts_tablet_manager.cc:1434] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.092s	user 0.016s	sys 0.013s
I20260709 12:24:16.806463  4796 tablet_bootstrap.cc:492] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:16.810002  4588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8e62ff81eb34853811314530131292b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.812280  4587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cc90257afed46488d4574c06b22a1a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.821475  4796 tablet_bootstrap.cc:654] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:16.801445  4662 tablet_service.cc:1511] Processing CreateTablet for tablet 4d675fbac0f844dc9309a662ba1c96f9 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:24:16.822149  4666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ebb8f343c743a1a28db116a4881004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.831264  4667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3124fc8af34f378ebd9cff21ee0e6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.832182  4662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d675fbac0f844dc9309a662ba1c96f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.834497  4664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4a575b75e05458186c89873e1e7622b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.838014  4669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8a74137ba104fe0b465cab6399c4940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.838768  4668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70fb561fb8b446969622ee2193d90917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.846432  4670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cc90257afed46488d4574c06b22a1a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.849251  4797 ts_tablet_manager.cc:1403] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.243s	user 0.014s	sys 0.031s
I20260709 12:24:16.849737  4665 tablet_service.cc:1511] Processing CreateTablet for tablet 559316f720c74aa1b50405e2ed791f6a (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:24:16.851454  4665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559316f720c74aa1b50405e2ed791f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.853699  4663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e20c8bae3534224bb7a82db42cab1a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:16.855710  4782 tablet.cc:2406] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:24:16.785331  4799 tablet_bootstrap.cc:654] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:16.873942  4593 tablet_service.cc:1511] Processing CreateTablet for tablet 231dbfb1b78d491eb8685070c2431d82 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:24:16.876843  4593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 231dbfb1b78d491eb8685070c2431d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.882645  4668 tablet_service.cc:1511] Processing CreateTablet for tablet 26f80146cffc4af2ac0af76b3a99c213 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:24:16.880746  4589 tablet_service.cc:1511] Processing CreateTablet for tablet 718b286f0ba24a90af47c49ec0031b11 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:24:16.883934  4668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26f80146cffc4af2ac0af76b3a99c213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.884057  4589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 718b286f0ba24a90af47c49ec0031b11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.895485  4662 tablet_service.cc:1511] Processing CreateTablet for tablet b8e62ff81eb34853811314530131292b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:24:16.896896  4662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8e62ff81eb34853811314530131292b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.899837  4742 tablet_service.cc:1511] Processing CreateTablet for tablet b8e62ff81eb34853811314530131292b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:24:16.901132  4742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8e62ff81eb34853811314530131292b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.901021  4668 tablet_service.cc:1511] Processing CreateTablet for tablet 231dbfb1b78d491eb8685070c2431d82 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:24:16.902314  4668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 231dbfb1b78d491eb8685070c2431d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.906603  4746 tablet_service.cc:1511] Processing CreateTablet for tablet 231dbfb1b78d491eb8685070c2431d82 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:24:16.907979  4746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 231dbfb1b78d491eb8685070c2431d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.914187  4797 raft_consensus.cc:359] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:16.915225  4797 raft_consensus.cc:385] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:16.915535  4797 raft_consensus.cc:740] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:16.927461  4668 tablet_service.cc:1511] Processing CreateTablet for tablet 718b286f0ba24a90af47c49ec0031b11 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:24:16.927461  4588 tablet_service.cc:1511] Processing CreateTablet for tablet f173b0d5264e4eff8b1704becc555780 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:24:16.928771  4668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 718b286f0ba24a90af47c49ec0031b11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.929023  4588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f173b0d5264e4eff8b1704becc555780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.931332  4741 tablet_service.cc:1511] Processing CreateTablet for tablet 718b286f0ba24a90af47c49ec0031b11 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:24:16.932230  4631 heartbeater.cc:499] Master 127.3.89.254:36317 was elected leader, sending a full tablet report...
I20260709 12:24:16.932587  4741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 718b286f0ba24a90af47c49ec0031b11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.933992  4797 consensus_queue.cc:260] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:16.935592  4740 tablet_service.cc:1511] Processing CreateTablet for tablet 7b27de60f9f44928b0068ab1e562568c (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:24:16.936888  4740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b27de60f9f44928b0068ab1e562568c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.945137  4668 tablet_service.cc:1511] Processing CreateTablet for tablet 7b27de60f9f44928b0068ab1e562568c (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:24:16.946453  4668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b27de60f9f44928b0068ab1e562568c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.952098  4739 tablet_service.cc:1511] Processing CreateTablet for tablet f173b0d5264e4eff8b1704becc555780 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:24:16.924996  4595 tablet_service.cc:1511] Processing CreateTablet for tablet 7b27de60f9f44928b0068ab1e562568c (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:24:16.970618  4595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b27de60f9f44928b0068ab1e562568c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.953435  4739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f173b0d5264e4eff8b1704becc555780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:16.954667  4667 tablet_service.cc:1511] Processing CreateTablet for tablet f173b0d5264e4eff8b1704becc555780 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:24:16.998724  4667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f173b0d5264e4eff8b1704becc555780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.010989  4588 tablet_service.cc:1511] Processing CreateTablet for tablet ea533a99927c40d48b894cf00a75301d (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:24:17.012356  4588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea533a99927c40d48b894cf00a75301d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.017283  4669 tablet_service.cc:1511] Processing CreateTablet for tablet 53908897ce974579947af90acaf0aed7 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:24:17.018730  4669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53908897ce974579947af90acaf0aed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.018599  4670 tablet_service.cc:1511] Processing CreateTablet for tablet 51b7b2074e114d8f872f149fa8acdbcf (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:24:17.019865  4670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51b7b2074e114d8f872f149fa8acdbcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.021629  4595 tablet_service.cc:1511] Processing CreateTablet for tablet 53908897ce974579947af90acaf0aed7 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:24:17.022917  4595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53908897ce974579947af90acaf0aed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.021664  4664 tablet_service.cc:1511] Processing CreateTablet for tablet ea533a99927c40d48b894cf00a75301d (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:24:17.027463  4707 heartbeater.cc:499] Master 127.3.89.254:36317 was elected leader, sending a full tablet report...
I20260709 12:24:17.028292  4589 tablet_service.cc:1511] Processing CreateTablet for tablet 51b7b2074e114d8f872f149fa8acdbcf (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:24:17.029552  4589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51b7b2074e114d8f872f149fa8acdbcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.029881  4664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea533a99927c40d48b894cf00a75301d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.035382  4587 tablet_service.cc:1511] Processing CreateTablet for tablet 23fa1ebbcb7f4a7e9bcf064d4198f48e (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:24:17.036710  4587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23fa1ebbcb7f4a7e9bcf064d4198f48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.046232  4799 tablet_bootstrap.cc:492] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:17.046747  4799 ts_tablet_manager.cc:1403] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.339s	user 0.019s	sys 0.007s
I20260709 12:24:17.049559  4799 raft_consensus.cc:359] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.050271  4799 raft_consensus.cc:385] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:17.050472  4799 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:17.051126  4799 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.054426  4796 tablet_bootstrap.cc:492] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:17.054968  4796 ts_tablet_manager.cc:1403] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.249s	user 0.023s	sys 0.027s
I20260709 12:24:17.056409  4797 ts_tablet_manager.cc:1434] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.207s	user 0.009s	sys 0.013s
I20260709 12:24:17.057461  4797 tablet_bootstrap.cc:492] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:17.058461  4665 tablet_service.cc:1511] Processing CreateTablet for tablet 23fa1ebbcb7f4a7e9bcf064d4198f48e (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:24:17.070262  4738 tablet_service.cc:1511] Processing CreateTablet for tablet 53908897ce974579947af90acaf0aed7 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:24:17.071205  4665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23fa1ebbcb7f4a7e9bcf064d4198f48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.071583  4738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53908897ce974579947af90acaf0aed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.062950  4588 tablet_service.cc:1511] Processing CreateTablet for tablet 23fb4c89626845d8bac3ba87a5346995 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:24:17.074324  4588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23fb4c89626845d8bac3ba87a5346995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.074666  4742 tablet_service.cc:1511] Processing CreateTablet for tablet 51b7b2074e114d8f872f149fa8acdbcf (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:24:17.075934  4742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51b7b2074e114d8f872f149fa8acdbcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.078562  4746 tablet_service.cc:1511] Processing CreateTablet for tablet 23fa1ebbcb7f4a7e9bcf064d4198f48e (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:24:17.078768  4796 raft_consensus.cc:359] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.079519  4796 raft_consensus.cc:385] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:17.079806  4746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23fa1ebbcb7f4a7e9bcf064d4198f48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.067348  4741 tablet_service.cc:1511] Processing CreateTablet for tablet ea533a99927c40d48b894cf00a75301d (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:24:17.082373  4741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea533a99927c40d48b894cf00a75301d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.082461  4797 tablet_bootstrap.cc:654] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:17.084878  4662 tablet_service.cc:1511] Processing CreateTablet for tablet 23fb4c89626845d8bac3ba87a5346995 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:24:17.086205  4662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23fb4c89626845d8bac3ba87a5346995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.079807  4796 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:17.094645  4796 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.100636  4744 tablet_service.cc:1511] Processing CreateTablet for tablet 23fb4c89626845d8bac3ba87a5346995 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:24:17.101994  4744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23fb4c89626845d8bac3ba87a5346995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.103690  4796 ts_tablet_manager.cc:1434] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 12:24:17.104717  4796 tablet_bootstrap.cc:492] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:17.098747  4593 tablet_service.cc:1511] Processing CreateTablet for tablet 09d362006ea24f48a30fa120bbdf8860 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:24:17.106627  4589 tablet_service.cc:1511] Processing CreateTablet for tablet 3be428c36f9242c6a2b8aa853f2c8b2c (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:24:17.107844  4593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09d362006ea24f48a30fa120bbdf8860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.106649  4667 tablet_service.cc:1511] Processing CreateTablet for tablet 09d362006ea24f48a30fa120bbdf8860 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:24:17.110925  4667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09d362006ea24f48a30fa120bbdf8860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.110718  4670 tablet_service.cc:1511] Processing CreateTablet for tablet 3be428c36f9242c6a2b8aa853f2c8b2c (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:24:17.112974  4590 tablet_service.cc:1511] Processing CreateTablet for tablet 3cbe5bef0f514b01ac552ff54d4d13be (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:24:17.116655  4590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cbe5bef0f514b01ac552ff54d4d13be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.122249  4799 ts_tablet_manager.cc:1434] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.075s	user 0.009s	sys 0.011s
I20260709 12:24:17.123382  4799 tablet_bootstrap.cc:492] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:17.124910  4589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3be428c36f9242c6a2b8aa853f2c8b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.115350  4664 tablet_service.cc:1511] Processing CreateTablet for tablet 3cbe5bef0f514b01ac552ff54d4d13be (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:24:17.126308  4664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cbe5bef0f514b01ac552ff54d4d13be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.132848  4670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3be428c36f9242c6a2b8aa853f2c8b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.141561  4588 tablet_service.cc:1511] Processing CreateTablet for tablet e85386ee6a30465188e201d5ee6ae972 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:24:17.142948  4588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e85386ee6a30465188e201d5ee6ae972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.147254  4799 tablet_bootstrap.cc:654] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:17.151144  4671 tablet_service.cc:1511] Processing CreateTablet for tablet e85386ee6a30465188e201d5ee6ae972 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:24:17.161718  4671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e85386ee6a30465188e201d5ee6ae972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.161722  4796 tablet_bootstrap.cc:654] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:17.170307  4663 tablet_service.cc:1511] Processing CreateTablet for tablet 12796fc41f234615a607ba222506e893 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:24:17.171686  4663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12796fc41f234615a607ba222506e893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.171530  4743 tablet_service.cc:1511] Processing CreateTablet for tablet 3cbe5bef0f514b01ac552ff54d4d13be (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:24:17.172756  4743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cbe5bef0f514b01ac552ff54d4d13be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.177263  4737 tablet_service.cc:1511] Processing CreateTablet for tablet e85386ee6a30465188e201d5ee6ae972 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:24:17.178678  4737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e85386ee6a30465188e201d5ee6ae972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.182032  4801 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.182534  4740 tablet_service.cc:1511] Processing CreateTablet for tablet 09d362006ea24f48a30fa120bbdf8860 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:24:17.182559  4801 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } }
I20260709 12:24:17.183941  4740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09d362006ea24f48a30fa120bbdf8860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.184578  4803 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.185076  4803 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.186365  4587 tablet_service.cc:1511] Processing CreateTablet for tablet 12796fc41f234615a607ba222506e893 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:24:17.187795  4587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12796fc41f234615a607ba222506e893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.188819  4801 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:17.189697  4805 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.192809  4805 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.178623  4742 tablet_service.cc:1511] Processing CreateTablet for tablet 3be428c36f9242c6a2b8aa853f2c8b2c (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:24:17.242111  4803 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:17.252059  4805 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:17.272866  4590 tablet_service.cc:1511] Processing CreateTablet for tablet 9702ea78030547e5a2f6fffe805075da (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:24:17.280092  4590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9702ea78030547e5a2f6fffe805075da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.271713  4742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3be428c36f9242c6a2b8aa853f2c8b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.311256  4669 tablet_service.cc:1511] Processing CreateTablet for tablet 9702ea78030547e5a2f6fffe805075da (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:24:17.312117  4799 tablet_bootstrap.cc:492] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:17.312598  4799 ts_tablet_manager.cc:1403] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.189s	user 0.025s	sys 0.022s
I20260709 12:24:17.315738  4799 raft_consensus.cc:359] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.316481  4799 raft_consensus.cc:385] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:17.316788  4799 raft_consensus.cc:740] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:17.310109  4797 tablet_bootstrap.cc:492] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:17.312584  4669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9702ea78030547e5a2f6fffe805075da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.325156  4589 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdfc68cfdd1411cad1b4e974e65067a (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:24:17.326923  4589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdfc68cfdd1411cad1b4e974e65067a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.332635  4797 ts_tablet_manager.cc:1403] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.275s	user 0.015s	sys 0.004s
I20260709 12:24:17.336676  4799 consensus_queue.cc:260] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.340185  4799 ts_tablet_manager.cc:1434] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 12:24:17.341231  4799 tablet_bootstrap.cc:492] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:17.350651  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:17.351624  4681 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:17.360320  4569 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:17.361814  4803 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:17.362212  4803 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.362581  4803 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:17.363178  4664 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdfc68cfdd1411cad1b4e974e65067a (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:24:17.366372  4664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdfc68cfdd1411cad1b4e974e65067a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.348925  4797 raft_consensus.cc:359] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.380678  4801 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.381211  4801 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.383240  4801 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:17.383070  4803 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.383129  4743 tablet_service.cc:1511] Processing CreateTablet for tablet 12796fc41f234615a607ba222506e893 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:24:17.384654  4743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12796fc41f234615a607ba222506e893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.386729  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:17.387317  4681 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:17.370965  4797 raft_consensus.cc:385] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:17.378911  4796 tablet_bootstrap.cc:492] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:17.385329  4803 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:17.396122  4799 tablet_bootstrap.cc:654] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:17.396660  4796 ts_tablet_manager.cc:1403] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.292s	user 0.010s	sys 0.007s
I20260709 12:24:17.400219  4681 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:17.401520  4569 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:17.402438  4803 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:17.412022  4796 raft_consensus.cc:359] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.412820  4796 raft_consensus.cc:385] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:17.413125  4796 raft_consensus.cc:740] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:17.413295  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:17.409747  4667 tablet_service.cc:1511] Processing CreateTablet for tablet ee319a06cc4642fba7ae226abe36e352 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:24:17.414951  4667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee319a06cc4642fba7ae226abe36e352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.420620  4803 raft_consensus.cc:697] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:17.390236  4797 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:17.424429  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:17.424345  4803 consensus_queue.cc:237] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.435105  4797 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.435860  4738 tablet_service.cc:1511] Processing CreateTablet for tablet 9702ea78030547e5a2f6fffe805075da (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:24:17.439283  4738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9702ea78030547e5a2f6fffe805075da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.443222  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:17.436090  4796 consensus_queue.cc:260] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:17.445544  4645 leader_election.cc:343] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:17.446310  4756 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:17.447836  4796 ts_tablet_manager.cc:1434] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.000s
I20260709 12:24:17.452171  4796 tablet_bootstrap.cc:492] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:17.462066  4742 tablet_service.cc:1511] Processing CreateTablet for tablet ee319a06cc4642fba7ae226abe36e352 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:24:17.463614  4742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee319a06cc4642fba7ae226abe36e352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.466315  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:17.466883  4588 tablet_service.cc:1511] Processing CreateTablet for tablet 4163edf182b549dfb724d2e5b5b1ac0e (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:24:17.468083  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:17.470130  4720 leader_election.cc:343] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:17.470288  4737 tablet_service.cc:1511] Processing CreateTablet for tablet 4163edf182b549dfb724d2e5b5b1ac0e (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:24:17.471701  4737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4163edf182b549dfb724d2e5b5b1ac0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.473589  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:17.473920  4587 tablet_service.cc:1511] Processing CreateTablet for tablet ee319a06cc4642fba7ae226abe36e352 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 12:24:17.475111  4647 leader_election.cc:343] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:17.475592  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:17.476145  4744 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdfc68cfdd1411cad1b4e974e65067a (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:24:17.476748  4805 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:17.477698  4720 leader_election.cc:343] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:17.478695  4593 tablet_service.cc:1511] Processing CreateTablet for tablet 8fc78aa9a3bf48afb52246b50ad414eb (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:24:17.480997  4796 tablet_bootstrap.cc:654] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:17.482554  4663 tablet_service.cc:1511] Processing CreateTablet for tablet 4163edf182b549dfb724d2e5b5b1ac0e (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:24:17.487298  4740 tablet_service.cc:1511] Processing CreateTablet for tablet 8fc78aa9a3bf48afb52246b50ad414eb (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:24:17.477455  4744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdfc68cfdd1411cad1b4e974e65067a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.489929  4588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4163edf182b549dfb724d2e5b5b1ac0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.492709  4663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4163edf182b549dfb724d2e5b5b1ac0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.510051  4740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc78aa9a3bf48afb52246b50ad414eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.513715  4593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc78aa9a3bf48afb52246b50ad414eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.515690  4799 tablet_bootstrap.cc:492] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:17.516264  4799 ts_tablet_manager.cc:1403] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.175s	user 0.023s	sys 0.028s
I20260709 12:24:17.520038  4670 tablet_service.cc:1511] Processing CreateTablet for tablet 8fc78aa9a3bf48afb52246b50ad414eb (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:24:17.521525  4670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc78aa9a3bf48afb52246b50ad414eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.529536  4797 ts_tablet_manager.cc:1434] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.184s	user 0.010s	sys 0.001s
I20260709 12:24:17.530768  4797 tablet_bootstrap.cc:492] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:17.532111  4587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee319a06cc4642fba7ae226abe36e352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.537459  4589 tablet_service.cc:1511] Processing CreateTablet for tablet 42d36b6bb6e84733876824d95e25967e (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:24:17.539283  4589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d36b6bb6e84733876824d95e25967e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.543660  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:17.543818  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:17.545441  4720 leader_election.cc:343] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:17.545957  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:24:17.547230  4720 leader_election.cc:343] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:17.547680  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:17.548624  4801 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:17.554883  4801 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:17.564996  4590 tablet_service.cc:1511] Processing CreateTablet for tablet 501b9483d1b74b73876563dd85e610d0 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:24:17.566424  4590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 501b9483d1b74b73876563dd85e610d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.565459  4500 catalog_manager.cc:5697] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:17.570745  4669 tablet_service.cc:1511] Processing CreateTablet for tablet 501b9483d1b74b73876563dd85e610d0 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:24:17.572477  4669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 501b9483d1b74b73876563dd85e610d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.575763  4664 tablet_service.cc:1511] Processing CreateTablet for tablet 42d36b6bb6e84733876824d95e25967e (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:24:17.577169  4664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d36b6bb6e84733876824d95e25967e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.576020  4588 tablet_service.cc:1511] Processing CreateTablet for tablet 8f05b2c44da743e49df87266876b3604 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:24:17.579475  4588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f05b2c44da743e49df87266876b3604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.579879  4671 tablet_service.cc:1511] Processing CreateTablet for tablet 8f05b2c44da743e49df87266876b3604 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:24:17.581198  4671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f05b2c44da743e49df87266876b3604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.586445  4799 raft_consensus.cc:359] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.594648  4796 tablet_bootstrap.cc:492] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:17.595333  4796 ts_tablet_manager.cc:1403] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.143s	user 0.004s	sys 0.016s
I20260709 12:24:17.603015  4796 raft_consensus.cc:359] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.604588  4796 raft_consensus.cc:385] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:17.604326  4595 tablet_service.cc:1511] Processing CreateTablet for tablet a8ee74acbd5c45f0934f6f37d2154bc7 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:24:17.604880  4796 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:17.605690  4595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8ee74acbd5c45f0934f6f37d2154bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.605598  4799 raft_consensus.cc:385] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:17.611476  4662 tablet_service.cc:1511] Processing CreateTablet for tablet a8ee74acbd5c45f0934f6f37d2154bc7 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:24:17.620111  4592 tablet_service.cc:1511] Processing CreateTablet for tablet d410e14541184b6bbaff38737ce4bc02 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:24:17.620916  4662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8ee74acbd5c45f0934f6f37d2154bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.621482  4592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d410e14541184b6bbaff38737ce4bc02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.626550  4797 tablet_bootstrap.cc:654] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:17.630157  4799 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:17.631549  4799 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.638965  4665 tablet_service.cc:1511] Processing CreateTablet for tablet d410e14541184b6bbaff38737ce4bc02 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:24:17.640311  4665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d410e14541184b6bbaff38737ce4bc02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.609416  4796 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.652338  4796 ts_tablet_manager.cc:1434] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 12:24:17.663232  4797 tablet_bootstrap.cc:492] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:17.663875  4797 ts_tablet_manager.cc:1403] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.133s	user 0.016s	sys 0.004s
I20260709 12:24:17.666945  4797 raft_consensus.cc:359] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.667634  4797 raft_consensus.cc:385] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:17.667917  4797 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:17.669085  4797 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.674911  4797 ts_tablet_manager.cc:1434] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.001s
W20260709 12:24:17.675710  4550 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:17.676007  4797 tablet_bootstrap.cc:492] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
W20260709 12:24:17.677731  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 130531us
I20260709 12:24:17.681931  4743 tablet_service.cc:1511] Processing CreateTablet for tablet 42d36b6bb6e84733876824d95e25967e (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:24:17.683064  4797 tablet_bootstrap.cc:654] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:17.683491  4743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42d36b6bb6e84733876824d95e25967e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.702411  4796 tablet_bootstrap.cc:492] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:17.711510  4799 ts_tablet_manager.cc:1434] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.195s	user 0.009s	sys 0.002s
I20260709 12:24:17.712644  4744 tablet_service.cc:1511] Processing CreateTablet for tablet 501b9483d1b74b73876563dd85e610d0 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:24:17.713752  4799 tablet_bootstrap.cc:492] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:17.714392  4744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 501b9483d1b74b73876563dd85e610d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.718493  4797 tablet_bootstrap.cc:492] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:17.719033  4797 ts_tablet_manager.cc:1403] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.021s
I20260709 12:24:17.722973  4737 tablet_service.cc:1511] Processing CreateTablet for tablet 8f05b2c44da743e49df87266876b3604 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:24:17.723551  4738 tablet_service.cc:1511] Processing CreateTablet for tablet a8ee74acbd5c45f0934f6f37d2154bc7 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:24:17.724371  4737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f05b2c44da743e49df87266876b3604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.724997  4738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8ee74acbd5c45f0934f6f37d2154bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.724623  4587 tablet_service.cc:1511] Processing CreateTablet for tablet 2d6e5f40333a47559cd852e4dfa7c86b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:24:17.725961  4587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d6e5f40333a47559cd852e4dfa7c86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.730883  4740 tablet_service.cc:1511] Processing CreateTablet for tablet d410e14541184b6bbaff38737ce4bc02 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:24:17.732761  4740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d410e14541184b6bbaff38737ce4bc02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.737443  4797 raft_consensus.cc:359] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.739605  4797 raft_consensus.cc:385] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:17.739920  4797 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:17.740705  4797 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.742223  4742 tablet_service.cc:1511] Processing CreateTablet for tablet 2d6e5f40333a47559cd852e4dfa7c86b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:24:17.743574  4742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d6e5f40333a47559cd852e4dfa7c86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.752381  4796 tablet_bootstrap.cc:654] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:17.764166  4595 tablet_service.cc:1511] Processing CreateTablet for tablet 1b7b0c6c811e455abe898cb4d21acad4 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:24:17.765633  4595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b7b0c6c811e455abe898cb4d21acad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.751981  4664 tablet_service.cc:1511] Processing CreateTablet for tablet 2d6e5f40333a47559cd852e4dfa7c86b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:24:17.767324  4664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d6e5f40333a47559cd852e4dfa7c86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.753755  4797 ts_tablet_manager.cc:1434] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.000s
I20260709 12:24:17.769488  4797 tablet_bootstrap.cc:492] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:17.754674  4799 tablet_bootstrap.cc:654] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:17.779963  4744 tablet_service.cc:1511] Processing CreateTablet for tablet 9ca7c574e14c4faeac378ee00c0a261b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:24:17.780236  4739 tablet_service.cc:1511] Processing CreateTablet for tablet 485fe0cee4b248f4802d3c15dc97a877 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:24:17.781468  4744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ca7c574e14c4faeac378ee00c0a261b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.783371  4665 tablet_service.cc:1511] Processing CreateTablet for tablet 9ca7c574e14c4faeac378ee00c0a261b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:24:17.784668  4665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ca7c574e14c4faeac378ee00c0a261b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.785977  4670 tablet_service.cc:1511] Processing CreateTablet for tablet 485fe0cee4b248f4802d3c15dc97a877 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:24:17.786589  4589 tablet_service.cc:1511] Processing CreateTablet for tablet 9ca7c574e14c4faeac378ee00c0a261b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:24:17.786635  4592 tablet_service.cc:1511] Processing CreateTablet for tablet 485fe0cee4b248f4802d3c15dc97a877 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:24:17.787370  4670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 485fe0cee4b248f4802d3c15dc97a877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.787941  4589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ca7c574e14c4faeac378ee00c0a261b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.789140  4797 tablet_bootstrap.cc:654] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:17.790203  4592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 485fe0cee4b248f4802d3c15dc97a877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.790258  4588 tablet_service.cc:1511] Processing CreateTablet for tablet b7cd71e05324451f8e6dfcf6c2e76829 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:24:17.797231  4588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cd71e05324451f8e6dfcf6c2e76829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.800844  4662 tablet_service.cc:1511] Processing CreateTablet for tablet 1b7b0c6c811e455abe898cb4d21acad4 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:24:17.801235  4738 tablet_service.cc:1511] Processing CreateTablet for tablet 1b7b0c6c811e455abe898cb4d21acad4 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:24:17.802623  4662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b7b0c6c811e455abe898cb4d21acad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.806093  4739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 485fe0cee4b248f4802d3c15dc97a877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.814360  4803 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.814522  4671 tablet_service.cc:1511] Processing CreateTablet for tablet b7cd71e05324451f8e6dfcf6c2e76829 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:24:17.814965  4803 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.815975  4671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cd71e05324451f8e6dfcf6c2e76829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.817235  4803 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:17.818135  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:17.818948  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:24:17.819633  4569 leader_election.cc:343] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:17.821149  4571 leader_election.cc:343] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:17.821681  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:17.822880  4803 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:17.842531  4799 tablet_bootstrap.cc:492] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:17.843058  4799 ts_tablet_manager.cc:1403] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.130s	user 0.015s	sys 0.015s
I20260709 12:24:17.844619  4669 tablet_service.cc:1511] Processing CreateTablet for tablet ae7ffb2806e441e787796287d56e8b8a (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:24:17.846454  4669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae7ffb2806e441e787796287d56e8b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.845950  4799 raft_consensus.cc:359] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.846904  4799 raft_consensus.cc:385] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:17.847267  4799 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:17.848613  4799 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.851387  4799 ts_tablet_manager.cc:1434] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:24:17.852520  4799 tablet_bootstrap.cc:492] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:17.855547  4738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b7b0c6c811e455abe898cb4d21acad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.866343  4756 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:17.874866  4797 tablet_bootstrap.cc:492] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:17.875428  4797 ts_tablet_manager.cc:1403] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.106s	user 0.008s	sys 0.010s
I20260709 12:24:17.876107  4803 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:17.878230  4825 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.878808  4825 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.881052  4825 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:17.880790  4744 tablet_service.cc:1511] Processing CreateTablet for tablet b7cd71e05324451f8e6dfcf6c2e76829 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:24:17.883589  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:17.884984  4744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cd71e05324451f8e6dfcf6c2e76829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.884855  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:17.885749  4680 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 0.
W20260709 12:24:17.886152  4720 leader_election.cc:343] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:17.886960  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:17.887825  4801 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:17.888193  4801 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.888543  4801 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:17.878238  4588 tablet_service.cc:1511] Processing CreateTablet for tablet ae7ffb2806e441e787796287d56e8b8a (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:24:17.897145  4588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae7ffb2806e441e787796287d56e8b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.906019  4589 tablet_service.cc:1511] Processing CreateTablet for tablet 28f87749718d445ba1615ff33028c4d8 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:24:17.907260  4589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28f87749718d445ba1615ff33028c4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.909363  4797 raft_consensus.cc:359] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.910137  4797 raft_consensus.cc:385] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:17.910444  4797 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:17.911084  4739 tablet_service.cc:1511] Processing CreateTablet for tablet ae7ffb2806e441e787796287d56e8b8a (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:24:17.912608  4739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae7ffb2806e441e787796287d56e8b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.911500  4796 tablet_bootstrap.cc:492] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:17.922319  4796 ts_tablet_manager.cc:1403] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.220s	user 0.015s	sys 0.021s
I20260709 12:24:17.911289  4797 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.934412  4797 ts_tablet_manager.cc:1434] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.059s	user 0.009s	sys 0.000s
I20260709 12:24:17.927465  4801 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.939945  4738 tablet_service.cc:1511] Processing CreateTablet for tablet 28f87749718d445ba1615ff33028c4d8 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:24:17.931176  4796 raft_consensus.cc:359] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.940915  4796 raft_consensus.cc:385] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:17.945395  4796 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:17.946429  4796 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:17.946782  4670 tablet_service.cc:1511] Processing CreateTablet for tablet 9ced730aeaf240b69d68fd0ee470d68d (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:24:17.948259  4670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ced730aeaf240b69d68fd0ee470d68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.949028  4796 ts_tablet_manager.cc:1434] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 12:24:17.944615  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:17.951967  4737 tablet_service.cc:1511] Processing CreateTablet for tablet 9ced730aeaf240b69d68fd0ee470d68d (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:24:17.941747  4797 tablet_bootstrap.cc:492] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:17.951921  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:24:17.955389  4720 leader_election.cc:343] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:17.955788  4680 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:17.941385  4738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28f87749718d445ba1615ff33028c4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.962189  4737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ced730aeaf240b69d68fd0ee470d68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.929436  4799 tablet_bootstrap.cc:654] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:17.953498  4822 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:17.953665  4801 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:17.939929  4671 tablet_service.cc:1511] Processing CreateTablet for tablet 28f87749718d445ba1615ff33028c4d8 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:24:17.967371  4796 tablet_bootstrap.cc:492] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:17.968547  4595 tablet_service.cc:1511] Processing CreateTablet for tablet 9ced730aeaf240b69d68fd0ee470d68d (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:24:17.982636  4595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ced730aeaf240b69d68fd0ee470d68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:17.983357  4797 tablet_bootstrap.cc:654] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.000239  4796 tablet_bootstrap.cc:654] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.000535  4588 tablet_service.cc:1511] Processing CreateTablet for tablet b6e604992a304031ad5cbdc32d90edb4 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:24:17.975107  4680 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:18.009091  4588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e604992a304031ad5cbdc32d90edb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:18.011489  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:18.015328  4739 tablet_service.cc:1511] Processing CreateTablet for tablet b6e604992a304031ad5cbdc32d90edb4 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:24:18.016790  4739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e604992a304031ad5cbdc32d90edb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:18.032173  4797 tablet_bootstrap.cc:492] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:18.032963  4797 ts_tablet_manager.cc:1403] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.091s	user 0.016s	sys 0.005s
I20260709 12:24:18.038527  4671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28f87749718d445ba1615ff33028c4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:18.045728  4797 raft_consensus.cc:359] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.046586  4797 raft_consensus.cc:385] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.046916  4797 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:17.985667  4662 tablet_service.cc:1511] Processing CreateTablet for tablet b6e604992a304031ad5cbdc32d90edb4 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:24:18.047842  4797 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.048316  4836 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.048422  4589 tablet_service.cc:1511] Processing CreateTablet for tablet 34a0b6acf7c6497e8ef8ffd63b4127d6 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:24:18.048913  4836 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.049599  4589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34a0b6acf7c6497e8ef8ffd63b4127d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:18.051369  4836 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.051913  4836 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.037664  4801 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:18.052377  4836 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.060456  4799 tablet_bootstrap.cc:492] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:18.061038  4799 ts_tablet_manager.cc:1403] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.209s	user 0.019s	sys 0.014s
I20260709 12:24:18.068212  4738 tablet_service.cc:1511] Processing CreateTablet for tablet 34a0b6acf7c6497e8ef8ffd63b4127d6 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:24:18.075776  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:18.076318  4836 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.076462  4755 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:18.057446  4801 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:18.079169  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:18.079698  4801 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.079456  4737 tablet_service.cc:1511] Processing CreateTablet for tablet aba6aa625d5a49658718de09523b8a9b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:24:18.083010  4737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aba6aa625d5a49658718de09523b8a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:18.082921  4669 tablet_service.cc:1511] Processing CreateTablet for tablet 34a0b6acf7c6497e8ef8ffd63b4127d6 (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:24:18.085824  4738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34a0b6acf7c6497e8ef8ffd63b4127d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:18.083817  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:18.057303  4822 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.065626  4662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6e604992a304031ad5cbdc32d90edb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:18.091959  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:18.093168  4665 tablet_service.cc:1511] Processing CreateTablet for tablet aba6aa625d5a49658718de09523b8a9b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:24:18.094595  4665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aba6aa625d5a49658718de09523b8a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:18.098153  4669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34a0b6acf7c6497e8ef8ffd63b4127d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:18.098860  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:18.099300  4797 ts_tablet_manager.cc:1434] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.007s
W20260709 12:24:18.100176  4569 leader_election.cc:343] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.100509  4797 tablet_bootstrap.cc:492] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
W20260709 12:24:18.101442  4569 leader_election.cc:343] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.102238  4589 tablet_service.cc:1511] Processing CreateTablet for tablet aba6aa625d5a49658718de09523b8a9b (DEFAULT_TABLE table=test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:24:18.103458  4589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aba6aa625d5a49658718de09523b8a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:18.097791  4822 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.105311  4799 raft_consensus.cc:359] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } }
I20260709 12:24:18.106338  4799 raft_consensus.cc:385] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.106766  4799 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.108078  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:18.107923  4799 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } }
I20260709 12:24:18.108815  4680 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:18.119321  4845 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:18.119827  4845 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.120188  4845 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:18.124841  4571 leader_election.cc:343] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.124933  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:18.125520  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:18.126181  4837 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:18.128173  4837 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.128517  4837 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.128978  4803 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:18.127322  4822 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.132614  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:24:18.134186  4571 leader_election.cc:343] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.148622  4796 tablet_bootstrap.cc:492] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:18.149355  4796 ts_tablet_manager.cc:1403] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.182s	user 0.009s	sys 0.014s
I20260709 12:24:18.155545  4837 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.153354  4796 raft_consensus.cc:359] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.157793  4796 raft_consensus.cc:385] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.158120  4796 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.158279  4837 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.156209  4799 ts_tablet_manager.cc:1434] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.095s	user 0.004s	sys 0.004s
I20260709 12:24:18.158883  4796 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.159998  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:18.160573  4756 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.162386  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:18.163058  4681 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.164610  4799 tablet_bootstrap.cc:492] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:18.166138  4825 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.166682  4825 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.168881  4796 ts_tablet_manager.cc:1434] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:24:18.170207  4796 tablet_bootstrap.cc:492] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:18.170810  4825 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:18.173053  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:18.173785  4605 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 0.
I20260709 12:24:18.174420  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:18.179644  4796 tablet_bootstrap.cc:654] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:18.180251  4720 leader_election.cc:343] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.180475  4797 tablet_bootstrap.cc:654] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.180989  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:18.182178  4825 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:18.182545  4825 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.182858  4825 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.184295  4756 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:18.185860  4571 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:18.186745  4837 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:18.187211  4837 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:18.188048  4837 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.208425  4797 tablet_bootstrap.cc:492] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:18.209298  4797 ts_tablet_manager.cc:1403] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.109s	user 0.013s	sys 0.005s
I20260709 12:24:18.204986  4845 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.214948  4501 catalog_manager.cc:5697] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:18.222551  4797 raft_consensus.cc:359] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } }
I20260709 12:24:18.223371  4797 raft_consensus.cc:385] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.224716  4797 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.229802  4797 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } }
I20260709 12:24:18.223964  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:18.228518  4681 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:18.212908  4825 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:18.235854  4569 leader_election.cc:343] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.241276  4825 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:18.242807  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:18.243445  4796 tablet_bootstrap.cc:492] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:18.243455  4605 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.245400  4796 ts_tablet_manager.cc:1403] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.075s	user 0.028s	sys 0.034s
I20260709 12:24:18.248569  4796 raft_consensus.cc:359] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.249308  4796 raft_consensus.cc:385] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.249625  4796 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.250433  4799 tablet_bootstrap.cc:654] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.250484  4796 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.253182  4796 ts_tablet_manager.cc:1434] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:24:18.254231  4796 tablet_bootstrap.cc:492] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:18.253350  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:18.258780  4605 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:18.259728  4845 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:18.259835  4720 leader_election.cc:343] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.261490  4796 tablet_bootstrap.cc:654] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.261768  4797 ts_tablet_manager.cc:1434] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.000s
I20260709 12:24:18.261865  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:18.263616  4756 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.264597  4797 tablet_bootstrap.cc:492] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:18.269251  4501 catalog_manager.cc:5697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:18.270068  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:18.270881  4825 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:18.273162  4825 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:18.274163  4825 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.282326  4797 tablet_bootstrap.cc:654] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.282536  4756 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:18.289867  4799 tablet_bootstrap.cc:492] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:18.290416  4799 ts_tablet_manager.cc:1403] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.131s	user 0.010s	sys 0.015s
I20260709 12:24:18.292289  4796 tablet_bootstrap.cc:492] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:18.292953  4796 ts_tablet_manager.cc:1403] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.039s	user 0.024s	sys 0.005s
I20260709 12:24:18.293807  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:18.295835  4837 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:18.296296  4837 raft_consensus.cc:697] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:18.297894  4796 raft_consensus.cc:359] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.298375  4796 raft_consensus.cc:385] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.298623  4796 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.298128  4501 catalog_manager.cc:5697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:18.293766  4799 raft_consensus.cc:359] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.297266  4837 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.299463  4799 raft_consensus.cc:385] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.299875  4799 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.300834  4799 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.303843  4796 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.305923  4796 ts_tablet_manager.cc:1434] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:24:18.306819  4796 tablet_bootstrap.cc:492] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:18.307314  4799 ts_tablet_manager.cc:1434] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 12:24:18.308379  4799 tablet_bootstrap.cc:492] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:18.310195  4797 tablet_bootstrap.cc:492] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:18.318015  4797 ts_tablet_manager.cc:1403] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.054s	user 0.017s	sys 0.005s
I20260709 12:24:18.318528  4796 tablet_bootstrap.cc:654] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.320865  4797 raft_consensus.cc:359] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.321911  4797 raft_consensus.cc:385] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.323071  4797 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.324020  4797 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.326318  4797 ts_tablet_manager.cc:1434] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:24:18.327111  4799 tablet_bootstrap.cc:654] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.327335  4797 tablet_bootstrap.cc:492] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:18.330252  4796 tablet_bootstrap.cc:492] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:18.329452  4501 catalog_manager.cc:5697] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:18.330850  4796 ts_tablet_manager.cc:1403] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.009s
I20260709 12:24:18.334493  4796 raft_consensus.cc:359] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.335410  4797 tablet_bootstrap.cc:654] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.335410  4796 raft_consensus.cc:385] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.336043  4796 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.337245  4796 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.345633  4796 ts_tablet_manager.cc:1434] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 12:24:18.347020  4796 tablet_bootstrap.cc:492] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:18.355504  4796 tablet_bootstrap.cc:654] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.363422  4799 tablet_bootstrap.cc:492] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:18.363991  4799 ts_tablet_manager.cc:1403] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.026s
I20260709 12:24:18.365690  4797 tablet_bootstrap.cc:492] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:18.366202  4797 ts_tablet_manager.cc:1403] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.018s
I20260709 12:24:18.366202  4796 tablet_bootstrap.cc:492] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:18.366930  4796 ts_tablet_manager.cc:1403] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.002s
I20260709 12:24:18.367050  4799 raft_consensus.cc:359] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.367779  4799 raft_consensus.cc:385] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.368083  4799 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.368953  4799 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.369146  4797 raft_consensus.cc:359] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.369910  4797 raft_consensus.cc:385] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.369902  4796 raft_consensus.cc:359] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.370461  4797 raft_consensus.cc:740] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.370792  4796 raft_consensus.cc:385] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.371101  4796 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.371392  4797 consensus_queue.cc:260] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.371731  4796 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.373613  4797 ts_tablet_manager.cc:1434] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:24:18.374153  4796 ts_tablet_manager.cc:1434] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:24:18.375140  4796 tablet_bootstrap.cc:492] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:18.376775  4797 tablet_bootstrap.cc:492] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:18.379094  4799 ts_tablet_manager.cc:1434] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:24:18.380133  4799 tablet_bootstrap.cc:492] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:18.381012  4796 tablet_bootstrap.cc:654] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.383445  4797 tablet_bootstrap.cc:654] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.387804  4799 tablet_bootstrap.cc:654] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.389317  4837 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.389871  4837 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.392918  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:18.393115  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:18.393649  4681 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:18.394145  4837 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.395257  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:18.393694  4756 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:18.397287  4837 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:18.397768  4837 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.398156  4837 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.401170  4799 tablet_bootstrap.cc:492] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:18.401796  4797 tablet_bootstrap.cc:492] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:18.401798  4799 ts_tablet_manager.cc:1403] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 12:24:18.402397  4797 ts_tablet_manager.cc:1403] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.001s
I20260709 12:24:18.412253  4797 raft_consensus.cc:359] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.413112  4797 raft_consensus.cc:385] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.413444  4797 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.414322  4797 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.419167  4837 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.420543  4825 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.421617  4797 ts_tablet_manager.cc:1434] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:24:18.421221  4825 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.422683  4797 tablet_bootstrap.cc:492] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:18.425053  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:18.426728  4720 leader_election.cc:343] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.426620  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:18.428457  4720 leader_election.cc:343] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.428973  4720 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:18.429919  4801 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:18.429163  4825 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:18.430644  4797 tablet_bootstrap.cc:654] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.430737  4799 raft_consensus.cc:359] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.431468  4799 raft_consensus.cc:385] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.431743  4799 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.433161  4796 tablet_bootstrap.cc:492] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:18.432768  4799 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.433706  4796 ts_tablet_manager.cc:1403] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.038s
I20260709 12:24:18.436645  4796 raft_consensus.cc:359] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.437378  4796 raft_consensus.cc:385] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.437799  4796 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.438717  4796 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.439291  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:18.440160  4681 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.441378  4796 ts_tablet_manager.cc:1434] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:24:18.442449  4796 tablet_bootstrap.cc:492] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:18.443379  4797 tablet_bootstrap.cc:492] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:18.443576  4799 ts_tablet_manager.cc:1434] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
I20260709 12:24:18.443861  4797 ts_tablet_manager.cc:1403] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.002s
I20260709 12:24:18.447183  4797 raft_consensus.cc:359] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.447907  4797 raft_consensus.cc:385] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.448263  4797 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.448939  4796 tablet_bootstrap.cc:654] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.448963  4799 tablet_bootstrap.cc:492] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:18.449244  4797 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.452037  4797 ts_tablet_manager.cc:1434] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:24:18.453049  4797 tablet_bootstrap.cc:492] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:18.457752  4799 tablet_bootstrap.cc:654] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.459619  4797 tablet_bootstrap.cc:654] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.462280  4845 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.462670  4681 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:18.462940  4796 tablet_bootstrap.cc:492] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:18.462810  4845 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.463429  4796 ts_tablet_manager.cc:1403] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 12:24:18.464035  4837 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.466158  4796 raft_consensus.cc:359] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.465163  4845 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.466878  4796 raft_consensus.cc:385] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.467269  4796 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.467965  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:18.468377  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:18.467957  4796 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.469116  4756 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.469223  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:18.470878  4796 ts_tablet_manager.cc:1434] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:24:18.471841  4796 tablet_bootstrap.cc:492] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
W20260709 12:24:18.471979  4571 leader_election.cc:343] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.474702  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:24:18.476253  4569 leader_election.cc:343] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.476775  4569 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:18.477651  4803 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:18.478413  4803 raft_consensus.cc:697] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:18.478963  4796 tablet_bootstrap.cc:654] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.479274  4803 consensus_queue.cc:237] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.481736  4836 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:18.490854  4756 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:18.508599  4499 catalog_manager.cc:5697] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:18.510778  4805 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.511431  4805 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } }
I20260709 12:24:18.514664  4805 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:24:18.515123  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:18.515769  4756 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:18.515997  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:18.516611  4605 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:18.517025  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:18.517858  4805 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:18.518214  4805 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.518544  4805 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.525878  4805 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } }
I20260709 12:24:18.529285  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:18.530092  4755 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.531917  4805 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:24:18.532665  4805 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.533183  4805 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.536418  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:18.539443  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:18.540143  4755 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:18.540253  4605 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.542264  4805 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.542297  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:18.543309  4604 raft_consensus.cc:2393] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 1: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:18.545298  4645 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:18.546782  4805 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.553121  4799 tablet_bootstrap.cc:492] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:18.553779  4799 ts_tablet_manager.cc:1403] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.105s	user 0.037s	sys 0.039s
I20260709 12:24:18.554633  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:18.556031  4869 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:18.557050  4799 raft_consensus.cc:359] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.557942  4799 raft_consensus.cc:385] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.558279  4799 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.559234  4799 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.560846  4605 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:18.564965  4871 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.565747  4871 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.568573  4797 tablet_bootstrap.cc:492] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:18.562209  4869 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:18.569480  4799 ts_tablet_manager.cc:1434] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.002s
I20260709 12:24:18.569941  4871 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.570542  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:18.570297  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:18.570847  4797 ts_tablet_manager.cc:1403] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.118s	user 0.021s	sys 0.035s
I20260709 12:24:18.571684  4756 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 1: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:18.571750  4799 tablet_bootstrap.cc:492] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:18.573437  4869 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } }
I20260709 12:24:18.574291  4797 raft_consensus.cc:359] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.575111  4797 raft_consensus.cc:385] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.575412  4797 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.575632  4645 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:18.576311  4796 tablet_bootstrap.cc:492] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:18.576156  4797 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.576758  4796 ts_tablet_manager.cc:1403] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.105s	user 0.030s	sys 0.035s
I20260709 12:24:18.577839  4872 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.578867  4797 ts_tablet_manager.cc:1434] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:24:18.579962  4797 tablet_bootstrap.cc:492] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:18.581615  4796 raft_consensus.cc:359] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.582404  4796 raft_consensus.cc:385] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.582710  4796 raft_consensus.cc:740] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.583477  4796 consensus_queue.cc:260] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.584213  4799 tablet_bootstrap.cc:654] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.579574  4805 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:18.587329  4825 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.588286  4825 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.588932  4796 ts_tablet_manager.cc:1434] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 12:24:18.592428  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:18.594144  4825 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:24:18.594321  4720 leader_election.cc:343] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.595304  4796 tablet_bootstrap.cc:492] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:18.595857  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:18.597178  4797 tablet_bootstrap.cc:654] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:18.597339  4720 leader_election.cc:343] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.597805  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:18.599259  4825 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:18.606638  4796 tablet_bootstrap.cc:654] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.606554  4825 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.607141  4825 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.610493  4825 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:18.612830  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:18.613560  4605 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 0.
I20260709 12:24:18.614764  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:18.607668  4799 tablet_bootstrap.cc:492] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:18.617836  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:18.618444  4799 ts_tablet_manager.cc:1403] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.010s
W20260709 12:24:18.618785  4720 leader_election.cc:343] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.619251  4801 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:18.619609  4801 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.619943  4801 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.623260  4825 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:18.625057  4799 raft_consensus.cc:359] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.626030  4799 raft_consensus.cc:385] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.626358  4799 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.626646  4501 catalog_manager.cc:5697] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:18.627267  4799 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.636137  4872 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:18.638897  4799 ts_tablet_manager.cc:1434] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.002s
I20260709 12:24:18.639859  4799 tablet_bootstrap.cc:492] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:18.641428  4796 tablet_bootstrap.cc:492] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:18.641913  4796 ts_tablet_manager.cc:1403] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.047s	user 0.011s	sys 0.031s
I20260709 12:24:18.644600  4796 raft_consensus.cc:359] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.645268  4796 raft_consensus.cc:385] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.645473  4799 tablet_bootstrap.cc:654] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.645622  4796 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.646373  4796 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.648577  4796 ts_tablet_manager.cc:1434] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:24:18.649840  4796 tablet_bootstrap.cc:492] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:18.657941  4796 tablet_bootstrap.cc:654] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.658321  4605 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.668571  4801 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.669171  4825 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.669257  4803 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:18.669802  4797 tablet_bootstrap.cc:492] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:18.670337  4797 ts_tablet_manager.cc:1403] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.091s	user 0.014s	sys 0.001s
I20260709 12:24:18.669715  4825 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.671669  4801 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:18.672274  4825 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:18.678889  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:18.682812  4550 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:18.683072  4801 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:18.673519  4797 raft_consensus.cc:359] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.684260  4797 raft_consensus.cc:385] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.684543  4797 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.683318  4872 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.685364  4872 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.678205  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:18.677110  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:18.688055  4872 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.688584  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:18.689281  4602 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:18.689527  4797 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.691179  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:18.691902  4797 ts_tablet_manager.cc:1434] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.021s	user 0.003s	sys 0.002s
I20260709 12:24:18.691869  4755 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:18.692898  4797 tablet_bootstrap.cc:492] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:18.676770  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:18.694187  4681 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.698406  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:18.687759  4604 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.700237  4797 tablet_bootstrap.cc:654] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:18.702906  4720 leader_election.cc:343] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.704244  4801 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:24:18.704414  4720 leader_election.cc:343] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.705035  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:18.705750  4803 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.706082  4801 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:18.706328  4803 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.709798  4872 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:18.710216  4872 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.710584  4872 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.710595  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:18.713353  4803 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.718089  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:18.677357  4796 tablet_bootstrap.cc:492] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:18.721694  4796 ts_tablet_manager.cc:1403] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.072s	user 0.003s	sys 0.017s
I20260709 12:24:18.725114  4797 tablet_bootstrap.cc:492] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:18.724828  4796 raft_consensus.cc:359] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.725620  4797 ts_tablet_manager.cc:1403] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.000s
I20260709 12:24:18.720498  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:18.729390  4799 tablet_bootstrap.cc:492] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:18.743528  4681 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:18.747816  4799 ts_tablet_manager.cc:1403] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.108s	user 0.016s	sys 0.023s
I20260709 12:24:18.725489  4796 raft_consensus.cc:385] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.748594  4796 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.740922  4604 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:18.752297  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:18.755645  4837 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:18.757762  4797 raft_consensus.cc:359] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.758481  4797 raft_consensus.cc:385] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.758747  4797 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.760466  4799 raft_consensus.cc:359] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.761402  4799 raft_consensus.cc:385] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.761775  4799 raft_consensus.cc:740] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.764027  4797 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.767452  4797 ts_tablet_manager.cc:1434] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 12:24:18.765458  4880 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:18.768757  4797 tablet_bootstrap.cc:492] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:18.769299  4796 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.771201  4799 consensus_queue.cc:260] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.775436  4799 ts_tablet_manager.cc:1434] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:24:18.776387  4799 tablet_bootstrap.cc:492] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:18.779872  4869 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.780465  4869 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.751921  4872 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.783275  4869 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.786190  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:18.786931  4605 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.788165  4880 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:18.789017  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:18.789814  4754 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:18.789433  4880 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.792912  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:18.794245  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:18.794948  4755 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.795506  4605 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:18.796808  4645 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:18.797725  4805 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:18.798105  4805 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.798434  4805 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.797748  4869 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:18.802471  4869 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:18.803004  4755 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:18.803437  4869 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.806223  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:18.806913  4604 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:18.814760  4796 ts_tablet_manager.cc:1434] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.003s
I20260709 12:24:18.814512  4805 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.815696  4796 tablet_bootstrap.cc:492] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:18.817673  4872 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.817353  4501 catalog_manager.cc:5697] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:18.818434  4805 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.819434  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:18.820084  4756 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.821673  4803 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:18.823395  4796 tablet_bootstrap.cc:654] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.827070  4892 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:24:18.827646  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:18.828125  4756 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:18.828459  4604 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.829560  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:18.834954  4796 tablet_bootstrap.cc:492] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:18.835448  4796 ts_tablet_manager.cc:1403] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 12:24:18.837924  4805 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:18.838466  4805 raft_consensus.cc:697] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:18.839139  4796 raft_consensus.cc:359] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.839290  4805 consensus_queue.cc:237] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.839848  4796 raft_consensus.cc:385] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.840119  4796 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.840866  4796 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.843082  4796 ts_tablet_manager.cc:1434] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:24:18.844249  4796 tablet_bootstrap.cc:492] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:18.849074  4796 tablet_bootstrap.cc:654] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.857788  4803 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.858453  4803 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.862623  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:18.863713  4797 tablet_bootstrap.cc:654] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.864380  4803 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:18.865352  4569 leader_election.cc:343] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.865912  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:18.872592  4797 tablet_bootstrap.cc:492] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
W20260709 12:24:18.872920  4571 leader_election.cc:343] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.866317  4796 tablet_bootstrap.cc:492] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:18.867774  4799 tablet_bootstrap.cc:654] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.873458  4571 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:18.873719  4796 ts_tablet_manager.cc:1403] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.012s
I20260709 12:24:18.874670  4803 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:18.873087  4797 ts_tablet_manager.cc:1403] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.105s	user 0.008s	sys 0.002s
I20260709 12:24:18.877060  4796 raft_consensus.cc:359] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.877808  4796 raft_consensus.cc:385] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.878082  4796 raft_consensus.cc:740] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.878307  4797 raft_consensus.cc:359] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.878851  4796 consensus_queue.cc:260] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.879856  4799 tablet_bootstrap.cc:492] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:18.880355  4799 ts_tablet_manager.cc:1403] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.104s	user 0.008s	sys 0.003s
I20260709 12:24:18.882113  4796 ts_tablet_manager.cc:1434] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:24:18.883597  4796 tablet_bootstrap.cc:492] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:18.884325  4499 catalog_manager.cc:5697] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:18.887529  4799 raft_consensus.cc:359] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.888284  4799 raft_consensus.cc:385] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.888701  4799 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.889508  4799 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.866003  4604 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:18.894625  4803 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:18.894991  4799 ts_tablet_manager.cc:1434] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:24:18.896169  4796 tablet_bootstrap.cc:654] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.896241  4799 tablet_bootstrap.cc:492] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:18.903987  4799 tablet_bootstrap.cc:654] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.908342  4501 catalog_manager.cc:5697] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:18.911782  4803 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:18.916190  4845 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.916929  4845 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.920006  4845 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.922719  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:18.923651  4755 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:18.923909  4797 raft_consensus.cc:385] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.924417  4797 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.926262  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:18.926201  4797 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.927250  4679 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:18.930392  4797 ts_tablet_manager.cc:1434] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.054s	user 0.004s	sys 0.003s
I20260709 12:24:18.929164  4571 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:18.931767  4797 tablet_bootstrap.cc:492] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:18.931758  4845 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:18.932842  4845 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.933317  4845 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.938807  4799 tablet_bootstrap.cc:492] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:18.940666  4799 ts_tablet_manager.cc:1403] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.007s
I20260709 12:24:18.942396  4880 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.942993  4880 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.945648  4880 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:18.947698  4845 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.948374  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:18.945807  4799 raft_consensus.cc:359] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.949353  4799 raft_consensus.cc:385] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.950229  4799 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.951138  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:18.951135  4720 leader_election.cc:343] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.951692  4799 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:18.953410  4720 leader_election.cc:343] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:18.953892  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:18.955186  4799 ts_tablet_manager.cc:1434] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:24:18.956331  4799 tablet_bootstrap.cc:492] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:18.956794  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:18.957890  4755 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.957002  4880 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:18.962688  4796 tablet_bootstrap.cc:492] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:18.963181  4796 ts_tablet_manager.cc:1403] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.080s	user 0.023s	sys 0.029s
I20260709 12:24:18.956311  4845 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.963645  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:18.964287  4680 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:18.965868  4796 raft_consensus.cc:359] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.966676  4796 raft_consensus.cc:385] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:18.966909  4796 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:18.967684  4796 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.969005  4799 tablet_bootstrap.cc:654] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.969985  4796 ts_tablet_manager.cc:1434] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:24:18.971052  4796 tablet_bootstrap.cc:492] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:18.975013  4680 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:18.976356  4845 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.976930  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:18.976904  4845 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.978358  4796 tablet_bootstrap.cc:654] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.978739  4755 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:18.979117  4845 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.979641  4845 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.980074  4845 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.982700  4822 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:18.983054  4869 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:18.983340  4822 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:18.984810  4822 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:18.995985  4797 tablet_bootstrap.cc:654] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:18.996151  4845 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:18.997817  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:18.998517  4754 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:18.999614  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:18.999948  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:19.001030  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:19.004647  4845 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:19.006263  4845 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.007050  4845 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.015628  4869 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:19.026177  4569 leader_election.cc:343] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.027294  4499 catalog_manager.cc:5697] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.029141  4822 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:19.031277  4869 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.031793  4869 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.034823  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:19.035991  4869 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.037271  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:19.038043  4680 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:19.038640  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:19.039295  4756 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:19.041383  4647 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:19.048655  4805 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:19.049069  4805 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.049422  4805 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.039299  4755 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:19.052381  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:19.054334  4803 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:19.054834  4803 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.055248  4803 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.064909  4822 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:19.064813  4845 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.073167  4605 raft_consensus.cc:2393] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 1: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:19.073473  4799 tablet_bootstrap.cc:492] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:19.070475  4805 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.075644  4799 ts_tablet_manager.cc:1403] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.120s	user 0.022s	sys 0.001s
I20260709 12:24:19.078109  4796 tablet_bootstrap.cc:492] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:19.078614  4796 ts_tablet_manager.cc:1403] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.108s	user 0.044s	sys 0.046s
I20260709 12:24:19.080415  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:19.081278  4680 raft_consensus.cc:2393] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 1: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:19.082826  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:19.083773  4605 raft_consensus.cc:2393] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 1: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:19.084512  4845 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.083585  4803 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.085561  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:19.086500  4756 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.085616  4797 tablet_bootstrap.cc:492] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:19.089967  4797 ts_tablet_manager.cc:1403] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.158s	user 0.011s	sys 0.015s
I20260709 12:24:19.102209  4799 raft_consensus.cc:359] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.102999  4799 raft_consensus.cc:385] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.103267  4799 raft_consensus.cc:740] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.105155  4799 consensus_queue.cc:260] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.105715  4796 raft_consensus.cc:359] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.106385  4796 raft_consensus.cc:385] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.106632  4796 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.107501  4799 ts_tablet_manager.cc:1434] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 12:24:19.107434  4796 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.108469  4799 tablet_bootstrap.cc:492] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:19.116402  4805 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.118417  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:19.127198  4647 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:19.127429  4880 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.128082  4880 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.128754  4805 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:19.130352  4880 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:19.132380  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:19.133646  4805 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.134145  4805 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.135950  4756 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:19.136270  4805 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.136924  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:19.138556  4720 leader_election.cc:343] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.138447  4797 raft_consensus.cc:359] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.149247  4797 raft_consensus.cc:385] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.149550  4797 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.150183  4797 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.151055  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:19.152277  4797 ts_tablet_manager.cc:1434] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.062s	user 0.001s	sys 0.005s
I20260709 12:24:19.152407  4837 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:19.139597  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:19.152844  4837 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:19.153231  4797 tablet_bootstrap.cc:492] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:19.142174  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:24:19.153728  4720 leader_election.cc:343] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.154024  4754 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:19.154232  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:24:19.154474  4645 leader_election.cc:343] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.155588  4880 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:19.147948  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:19.156514  4755 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.156457  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:19.153720  4837 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.164156  4755 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:19.164978  4805 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:19.165261  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:19.166112  4845 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:19.166646  4845 raft_consensus.cc:697] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:19.167546  4845 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.165721  4805 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.145967  4803 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.146657  4799 tablet_bootstrap.cc:654] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.169204  4805 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.147642  4796 ts_tablet_manager.cc:1434] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.069s	user 0.006s	sys 0.001s
I20260709 12:24:19.147944  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:24:19.175061  4569 leader_election.cc:343] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.177071  4796 tablet_bootstrap.cc:492] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:19.177210  4805 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.180703  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:19.181025  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:19.181427  4755 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.180337  4499 catalog_manager.cc:5697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:19.182750  4645 leader_election.cc:343] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.183465  4799 tablet_bootstrap.cc:492] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:19.183964  4799 ts_tablet_manager.cc:1403] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.076s	user 0.010s	sys 0.000s
I20260709 12:24:19.179581  4805 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.164819  4797 tablet_bootstrap.cc:654] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.189380  4755 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:19.191537  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:19.193908  4799 raft_consensus.cc:359] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.194393  4796 tablet_bootstrap.cc:654] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.192783  4805 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:19.197464  4799 raft_consensus.cc:385] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.198148  4799 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.203419  4797 tablet_bootstrap.cc:492] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:19.203318  4805 raft_consensus.cc:697] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:19.203991  4797 ts_tablet_manager.cc:1403] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.001s
I20260709 12:24:19.204370  4805 consensus_queue.cc:237] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.207547  4797 raft_consensus.cc:359] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.208261  4797 raft_consensus.cc:385] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.208602  4797 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.209245  4797 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.211833  4797 ts_tablet_manager.cc:1434] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:24:19.208323  4799 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.213403  4797 tablet_bootstrap.cc:492] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:19.214706  4799 ts_tablet_manager.cc:1434] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:24:19.215638  4799 tablet_bootstrap.cc:492] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:19.220209  4501 catalog_manager.cc:5697] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.222510  4797 tablet_bootstrap.cc:654] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.229537  4799 tablet_bootstrap.cc:654] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.230149  4499 catalog_manager.cc:5697] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.231694  4805 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:19.246121  4796 tablet_bootstrap.cc:492] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:19.248433  4796 ts_tablet_manager.cc:1403] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.072s	user 0.013s	sys 0.020s
I20260709 12:24:19.251387  4796 raft_consensus.cc:359] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.252238  4796 raft_consensus.cc:385] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.252573  4796 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.253328  4796 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.257156  4797 tablet_bootstrap.cc:492] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:19.257827  4880 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.259891  4880 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.262028  4880 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:19.267834  4805 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:19.269340  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:19.270054  4680 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 0.
I20260709 12:24:19.270931  4845 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.271359  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:19.271404  4845 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.272684  4893 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:19.273058  4893 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.273407  4893 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.273699  4845 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.275523  4797 ts_tablet_manager.cc:1403] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.062s	user 0.018s	sys 0.015s
I20260709 12:24:19.276322  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:19.276968  4754 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:19.277992  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:19.278313  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:19.259415  4796 ts_tablet_manager.cc:1434] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:24:19.279178  4845 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:19.279537  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:19.280855  4569 leader_election.cc:343] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:19.281070  4720 leader_election.cc:343] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.281663  4796 tablet_bootstrap.cc:492] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:19.279593  4845 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.283777  4845 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.288621  4797 raft_consensus.cc:359] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.292496  4797 raft_consensus.cc:385] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.292899  4797 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.293427  4893 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.293905  4797 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.298436  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:19.299086  4799 tablet_bootstrap.cc:492] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:19.299569  4799 ts_tablet_manager.cc:1403] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.084s	user 0.028s	sys 0.031s
I20260709 12:24:19.300828  4797 ts_tablet_manager.cc:1434] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:24:19.301837  4797 tablet_bootstrap.cc:492] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
W20260709 12:24:19.303877  4720 leader_election.cc:343] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.304057  4893 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:19.302879  4880 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:19.305955  4799 raft_consensus.cc:359] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.306725  4799 raft_consensus.cc:385] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.307118  4799 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.305743  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:19.308192  4680 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.309720  4805 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:24:19.312851  4845 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.314985  4799 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.322273  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:19.323388  4845 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.323539  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:19.324268  4756 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:19.324574  4569 leader_election.cc:343] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.327410  4680 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:19.328003  4799 ts_tablet_manager.cc:1434] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.003s
I20260709 12:24:19.335018  4797 tablet_bootstrap.cc:654] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.343478  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:19.345103  4880 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:19.345793  4880 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:19.346982  4880 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.353715  4845 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.354447  4845 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.358278  4799 tablet_bootstrap.cc:492] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:19.362681  4796 tablet_bootstrap.cc:654] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.365000  4845 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.370097  4756 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:19.364154  4805 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.374037  4805 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.372315  4871 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.374994  4501 catalog_manager.cc:5697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:19.375852  4871 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.377084  4805 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.377166  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:19.373385  4872 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:19.377866  4755 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:19.381230  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:19.381918  4753 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:19.377998  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:19.383199  4605 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:19.383440  4647 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:19.378743  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:19.385358  4805 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:19.385882  4805 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.386283  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:19.386598  4805 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.387595  4571 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:19.388492  4845 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.388913  4845 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.389905  4803 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:19.389746  4837 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:19.390944  4837 raft_consensus.cc:697] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:19.390316  4803 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.392803  4803 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:19.385880  4645 leader_election.cc:343] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.373898  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:19.392038  4837 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.401423  4871 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.402076  4845 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.404546  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:19.404616  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:19.405746  4753 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:19.405866  4803 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:19.410887  4569 leader_election.cc:343] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.411240  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:19.416100  4871 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:19.416662  4871 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.418494  4871 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.419935  4893 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:19.423202  4803 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.433792  4797 tablet_bootstrap.cc:492] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:19.434276  4797 ts_tablet_manager.cc:1403] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.133s	user 0.020s	sys 0.010s
W20260709 12:24:19.426527  4927 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:24:19.425799  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:19.446455  4797 raft_consensus.cc:359] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.447189  4797 raft_consensus.cc:385] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.447521  4797 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.448024  4797 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.424721  4805 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.460435  4680 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 1: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:19.460465  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:19.461442  4604 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 1: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:19.451679  4797 ts_tablet_manager.cc:1434] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:24:19.454273  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:19.455425  4501 catalog_manager.cc:5697] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.462235  4753 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.462054  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:19.463990  4797 tablet_bootstrap.cc:492] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:19.462644  4805 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.475997  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:19.477852  4647 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:19.483512  4803 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:19.484619  4871 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.486441  4754 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:19.487393  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:19.487716  4571 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:19.488147  4805 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:19.488831  4803 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:24:19.488955  4645 leader_election.cc:343] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.490556  4753 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:19.497082  4803 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.497772  4803 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.498220  4796 tablet_bootstrap.cc:492] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:19.498667  4796 ts_tablet_manager.cc:1403] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.217s	user 0.017s	sys 0.021s
I20260709 12:24:19.499351  4799 tablet_bootstrap.cc:654] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.501024  4571 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:19.500998  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:19.502574  4754 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.503618  4822 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:19.504180  4822 raft_consensus.cc:697] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:19.504899  4822 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.507393  4871 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.513170  4796 raft_consensus.cc:359] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.513809  4796 raft_consensus.cc:385] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.514043  4796 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.514684  4796 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.522142  4797 tablet_bootstrap.cc:654] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.524894  4796 ts_tablet_manager.cc:1434] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:24:19.526535  4796 tablet_bootstrap.cc:492] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:19.530110  4837 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:19.538563  4803 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.547704  4803 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.547803  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:24:19.554562  4569 leader_election.cc:343] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.559150  4799 tablet_bootstrap.cc:492] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:19.559628  4799 ts_tablet_manager.cc:1403] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.202s	user 0.012s	sys 0.008s
I20260709 12:24:19.547803  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:19.561868  4755 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.562773  4796 tablet_bootstrap.cc:654] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.563333  4754 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:19.565311  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:19.567153  4805 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:19.567673  4805 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:19.568517  4805 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.580960  4755 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:19.585184  4499 catalog_manager.cc:5697] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.589972  4571 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:19.591125  4803 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:19.591632  4803 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:19.592473  4803 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.597249  4799 raft_consensus.cc:359] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.598240  4799 raft_consensus.cc:385] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.598577  4799 raft_consensus.cc:740] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.604449  4799 consensus_queue.cc:260] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.613797  4797 tablet_bootstrap.cc:492] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:19.614331  4797 ts_tablet_manager.cc:1403] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.151s	user 0.015s	sys 0.012s
I20260709 12:24:19.616470  4799 ts_tablet_manager.cc:1434] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.000s
I20260709 12:24:19.617055  4797 raft_consensus.cc:359] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.617841  4797 raft_consensus.cc:385] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.617980  4845 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:19.618201  4797 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.619040  4797 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.621026  4797 ts_tablet_manager.cc:1434] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:24:19.617514  4799 tablet_bootstrap.cc:492] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:19.627668  4796 tablet_bootstrap.cc:492] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:19.628129  4796 ts_tablet_manager.cc:1403] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.103s	user 0.015s	sys 0.011s
I20260709 12:24:19.628587  4799 tablet_bootstrap.cc:654] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.631608  4797 tablet_bootstrap.cc:492] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:19.631225  4499 catalog_manager.cc:5697] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.633131  4796 raft_consensus.cc:359] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.633829  4796 raft_consensus.cc:385] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.634092  4796 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.634733  4796 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.636991  4796 ts_tablet_manager.cc:1434] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:24:19.638144  4796 tablet_bootstrap.cc:492] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:19.639513  4797 tablet_bootstrap.cc:654] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.643601  4501 catalog_manager.cc:5697] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.650121  4931 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.650688  4931 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.653187  4931 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:19.654482  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:19.654713  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:19.656212  4720 leader_election.cc:343] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:19.656755  4720 leader_election.cc:343] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.657097  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:19.657793  4796 tablet_bootstrap.cc:654] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.658618  4931 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:19.660266  4799 tablet_bootstrap.cc:492] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:19.660732  4799 ts_tablet_manager.cc:1403] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.003s
I20260709 12:24:19.663584  4799 raft_consensus.cc:359] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.664300  4799 raft_consensus.cc:385] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:24:19.664392  4569 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:19.664597  4799 raft_consensus.cc:740] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.665318  4799 consensus_queue.cc:260] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.667388  4799 ts_tablet_manager.cc:1434] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:24:19.668466  4799 tablet_bootstrap.cc:492] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:19.677445  4796 tablet_bootstrap.cc:492] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:19.678509  4799 tablet_bootstrap.cc:654] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.678664  4796 ts_tablet_manager.cc:1403] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.041s	user 0.005s	sys 0.010s
I20260709 12:24:19.680675  4796 raft_consensus.cc:359] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.681185  4796 raft_consensus.cc:385] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.681493  4796 raft_consensus.cc:740] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
W20260709 12:24:19.684932  4645 consensus_peers.cc:597] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:19.685534  4797 tablet_bootstrap.cc:492] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:19.686043  4797 ts_tablet_manager.cc:1403] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.055s	user 0.016s	sys 0.002s
I20260709 12:24:19.686105  4796 consensus_queue.cc:260] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.688608  4796 ts_tablet_manager.cc:1434] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:24:19.688642  4797 raft_consensus.cc:359] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.689522  4797 raft_consensus.cc:385] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.689806  4797 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.690503  4796 tablet_bootstrap.cc:492] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:19.690410  4797 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.692400  4797 ts_tablet_manager.cc:1434] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:24:19.685552  4799 tablet_bootstrap.cc:492] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:19.692942  4799 ts_tablet_manager.cc:1403] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.005s
I20260709 12:24:19.696046  4797 tablet_bootstrap.cc:492] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:19.695803  4799 raft_consensus.cc:359] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.696527  4799 raft_consensus.cc:385] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.696815  4799 raft_consensus.cc:740] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.697031  4796 tablet_bootstrap.cc:654] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.697919  4837 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.697624  4799 consensus_queue.cc:260] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.698397  4837 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.700524  4837 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.702306  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:19.703011  4755 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:19.705050  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:19.706228  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:24:19.707930  4569 leader_election.cc:343] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.708102  4931 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.708560  4931 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.710081  4799 ts_tablet_manager.cc:1434] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:24:19.710629  4931 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:19.712146  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:19.714401  4720 leader_election.cc:343] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.715327  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:19.715992  4679 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 0.
I20260709 12:24:19.716185  4799 tablet_bootstrap.cc:492] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:19.717222  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:19.718385  4893 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:19.718748  4893 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.719072  4893 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.723874  4799 tablet_bootstrap.cc:654] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.724320  4845 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:19.724741  4845 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.725952  4845 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.734045  4837 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:19.737051  4796 tablet_bootstrap.cc:492] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:19.739216  4796 ts_tablet_manager.cc:1403] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.020s
I20260709 12:24:19.742739  4931 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:19.742663  4796 raft_consensus.cc:359] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.745729  4799 tablet_bootstrap.cc:492] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:19.746269  4799 ts_tablet_manager.cc:1403] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.030s	user 0.020s	sys 0.000s
I20260709 12:24:19.745865  4796 raft_consensus.cc:385] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.746184  4893 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.746836  4796 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.747761  4796 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.749789  4796 ts_tablet_manager.cc:1434] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:24:19.749414  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:19.749483  4799 raft_consensus.cc:359] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.750448  4799 raft_consensus.cc:385] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.750902  4799 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.751616  4796 tablet_bootstrap.cc:492] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:19.751606  4799 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.754537  4799 ts_tablet_manager.cc:1434] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
W20260709 12:24:19.754940  4720 leader_election.cc:343] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.755427  4799 tablet_bootstrap.cc:492] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
W20260709 12:24:19.756207  4550 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:19.759245  4822 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:19.761656  4799 tablet_bootstrap.cc:654] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.761656  4796 tablet_bootstrap.cc:654] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.781963  4845 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.784384  4893 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:19.789206  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:19.791132  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:19.791802  4681 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.793025  4797 tablet_bootstrap.cc:654] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.796221  4604 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.797351  4799 tablet_bootstrap.cc:492] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:19.797923  4799 ts_tablet_manager.cc:1403] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.043s	user 0.021s	sys 0.018s
W20260709 12:24:19.797780  4569 leader_election.cc:343] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.800714  4799 raft_consensus.cc:359] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.801431  4799 raft_consensus.cc:385] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.801708  4799 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.802277  4796 tablet_bootstrap.cc:492] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:19.802743  4796 ts_tablet_manager.cc:1403] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.010s
I20260709 12:24:19.802534  4799 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.802017  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:19.804013  4754 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.803504  4845 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.806695  4871 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.807193  4871 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.809218  4871 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.813114  4799 ts_tablet_manager.cc:1434] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:24:19.814227  4799 tablet_bootstrap.cc:492] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:19.817436  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:19.815603  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:19.819077  4756 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:19.819885  4805 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:24:19.822404  4645 leader_election.cc:343] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:19.826088  4837 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.826611  4837 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.832979  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:19.833130  4796 raft_consensus.cc:359] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.835605  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:19.837193  4837 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.837291  4805 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:19.837877  4805 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.838253  4805 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.839046  4796 raft_consensus.cc:385] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.839495  4796 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.844561  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:19.845714  4753 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 1: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:19.845656  4796 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.847259  4571 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:19.848383  4837 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.825564  4754 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:19.854696  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:19.860137  4803 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:19.860703  4803 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:19.863524  4796 ts_tablet_manager.cc:1434] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.001s
I20260709 12:24:19.863148  4803 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.864513  4796 tablet_bootstrap.cc:492] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:19.858743  4681 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:19.877352  4845 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.877967  4845 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.881621  4796 tablet_bootstrap.cc:654] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.881891  4797 tablet_bootstrap.cc:492] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:19.882302  4797 ts_tablet_manager.cc:1403] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.186s	user 0.012s	sys 0.007s
I20260709 12:24:19.885911  4501 catalog_manager.cc:5697] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.890409  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:19.893317  4845 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.895929  4929 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:19.896487  4929 raft_consensus.cc:697] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:19.896644  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:19.898058  4797 raft_consensus.cc:359] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.897322  4929 consensus_queue.cc:237] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.898782  4797 raft_consensus.cc:385] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.899127  4797 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.899864  4797 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.901180  4931 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:19.901345  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:19.902280  4797 ts_tablet_manager.cc:1434] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:24:19.903267  4797 tablet_bootstrap.cc:492] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:19.903891  4569 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:19.905951  4845 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:19.894778  4805 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.908245  4805 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:19.909643  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:19.910303  4754 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.921177  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:19.922041  4604 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:19.924108  4803 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:19.935717  4837 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:19.932806  4754 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:19.943078  4799 tablet_bootstrap.cc:654] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.949193  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:19.951916  4796 tablet_bootstrap.cc:492] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:19.953644  4796 ts_tablet_manager.cc:1403] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.089s	user 0.019s	sys 0.003s
I20260709 12:24:19.952761  4805 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:19.954684  4805 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:19.955865  4805 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.968842  4604 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:19.968833  4796 raft_consensus.cc:359] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.970077  4796 raft_consensus.cc:385] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.970429  4796 raft_consensus.cc:740] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:19.987888  4501 catalog_manager.cc:5697] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:19.989349  4837 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.989878  4837 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.989936  4796 consensus_queue.cc:260] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.990931  4797 tablet_bootstrap.cc:654] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:19.992110  4796 ts_tablet_manager.cc:1434] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:24:19.992995  4796 tablet_bootstrap.cc:492] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:19.995582  4799 tablet_bootstrap.cc:492] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:19.996119  4799 ts_tablet_manager.cc:1403] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.182s	user 0.012s	sys 0.017s
I20260709 12:24:19.998901  4799 raft_consensus.cc:359] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:19.999565  4799 raft_consensus.cc:385] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:19.999830  4799 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.000512  4799 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.006436  4872 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.006954  4872 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.008253  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:20.009140  4872 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.009358  4754 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:20.010923  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:20.012336  4645 leader_election.cc:343] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.012795  4799 ts_tablet_manager.cc:1434] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:24:20.013453  4571 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:20.013788  4799 tablet_bootstrap.cc:492] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:20.014768  4845 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:20.015130  4845 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.015663  4845 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.018627  4931 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.019284  4796 tablet_bootstrap.cc:654] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.019112  4931 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.021265  4931 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:20.018211  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:20.022552  4837 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.023602  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:20.024839  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:20.024942  4754 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:20.025206  4602 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 0.
I20260709 12:24:20.025560  4680 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 0.
I20260709 12:24:20.026273  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:20.026859  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:20.027465  4799 tablet_bootstrap.cc:654] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.028286  4869 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:20.028709  4869 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.029048  4869 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.033499  4796 tablet_bootstrap.cc:492] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:20.034029  4796 ts_tablet_manager.cc:1403] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.000s
I20260709 12:24:20.036351  4679 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:20.037273  4569 consensus_peers.cc:597] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:20.040230  4796 raft_consensus.cc:359] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.040982  4796 raft_consensus.cc:385] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.041256  4796 raft_consensus.cc:740] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.042434  4796 consensus_queue.cc:260] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.044077  4929 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:20.044684  4929 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.045090  4796 ts_tablet_manager.cc:1434] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:24:20.045231  4929 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.045406  4872 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:20.046080  4796 tablet_bootstrap.cc:492] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:20.050046  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:20.056389  4869 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.023041  4845 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.059746  4872 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:20.060707  4869 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.062235  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:20.062884  4756 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.063088  4845 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.064226  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:24:20.066334  4645 leader_election.cc:343] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.070039  4845 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:20.073341  4796 tablet_bootstrap.cc:654] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.073907  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:20.074579  4754 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.077829  4872 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.078538  4872 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.082885  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:20.083745  4755 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:20.085422  4647 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:20.086462  4869 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:20.086822  4869 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.087168  4869 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.092561  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:20.094494  4645 leader_election.cc:343] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.094852  4869 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.097656  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:20.098407  4964 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:20.099133  4645 leader_election.cc:343] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.100121  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:24:20.103170  4569 leader_election.cc:343] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:20.106379  4569 leader_election.cc:343] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.108778  4931 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.109311  4931 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.111644  4931 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:20.113631  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:20.115674  4720 leader_election.cc:343] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.116593  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:20.117934  4720 leader_election.cc:343] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.118355  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:20.119593  4931 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:20.120787  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:20.121701  4752 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.124496  4754 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:20.127568  4571 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:20.130218  4803 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:20.130817  4803 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:20.131862  4803 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.136302  4872 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.100127  4869 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.127749  4756 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:20.150367  4837 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:20.154320  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:20.155618  4796 tablet_bootstrap.cc:492] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:20.155480  4869 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:20.156123  4796 ts_tablet_manager.cc:1403] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.110s	user 0.013s	sys 0.027s
I20260709 12:24:20.156582  4869 raft_consensus.cc:697] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:20.157596  4869 consensus_queue.cc:237] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.161880  4796 raft_consensus.cc:359] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.162632  4796 raft_consensus.cc:385] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.162940  4796 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.163936  4796 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.166779  4752 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:20.170810  4647 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:20.172711  4805 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:20.173281  4805 raft_consensus.cc:697] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:20.174268  4805 consensus_queue.cc:237] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.172750  4796 ts_tablet_manager.cc:1434] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:24:20.181591  4796 tablet_bootstrap.cc:492] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:20.191113  4796 tablet_bootstrap.cc:654] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.097429  4496 catalog_manager.cc:5697] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:20.213102  4796 tablet_bootstrap.cc:492] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:20.213636  4796 ts_tablet_manager.cc:1403] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.000s
I20260709 12:24:20.216450  4796 raft_consensus.cc:359] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.217350  4796 raft_consensus.cc:385] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.217748  4796 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.218676  4796 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.221304  4796 ts_tablet_manager.cc:1434] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:24:20.227437  4797 tablet_bootstrap.cc:492] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:20.228025  4797 ts_tablet_manager.cc:1403] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.325s	user 0.056s	sys 0.114s
I20260709 12:24:20.230753  4797 raft_consensus.cc:359] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.231467  4797 raft_consensus.cc:385] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.231851  4797 raft_consensus.cc:740] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.231508  4929 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.232820  4796 tablet_bootstrap.cc:492] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:20.232558  4797 consensus_queue.cc:260] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.235420  4845 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:20.236340  4797 ts_tablet_manager.cc:1434] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:24:20.237350  4797 tablet_bootstrap.cc:492] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:20.240943  4799 tablet_bootstrap.cc:492] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:20.241436  4799 ts_tablet_manager.cc:1403] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.228s	user 0.040s	sys 0.044s
I20260709 12:24:20.244168  4799 raft_consensus.cc:359] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.244908  4799 raft_consensus.cc:385] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.245085  4797 tablet_bootstrap.cc:654] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.245162  4799 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.246140  4799 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.248193  4799 ts_tablet_manager.cc:1434] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:24:20.249056  4799 tablet_bootstrap.cc:492] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:20.256696  4796 tablet_bootstrap.cc:654] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.262550  4799 tablet_bootstrap.cc:654] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.240171  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:20.267637  4796 tablet_bootstrap.cc:492] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:20.267714  4604 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.268191  4796 ts_tablet_manager.cc:1403] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.036s	user 0.007s	sys 0.004s
I20260709 12:24:20.271251  4796 raft_consensus.cc:359] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.272120  4796 raft_consensus.cc:385] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.272430  4796 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.273234  4796 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.275964  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:20.277788  4679 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.275895  4796 ts_tablet_manager.cc:1434] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:24:20.283394  4796 tablet_bootstrap.cc:492] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:20.290119  4929 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:20.332755  4679 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:20.343070  4796 tablet_bootstrap.cc:654] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.346380  4845 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:20.368705  4680 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.358430  4604 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:20.385076  4799 tablet_bootstrap.cc:492] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:20.385469  4799 ts_tablet_manager.cc:1403] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.137s	user 0.022s	sys 0.007s
I20260709 12:24:20.298403  4498 catalog_manager.cc:5697] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:20.386576  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:20.398589  4803 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.398922  4929 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:20.399449  4929 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:20.399128  4803 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.400416  4929 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.401173  4799 raft_consensus.cc:359] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.403452  4799 raft_consensus.cc:385] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.403795  4799 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.405783  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:20.408510  4893 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:20.410923  4797 tablet_bootstrap.cc:492] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:20.411550  4797 ts_tablet_manager.cc:1403] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.174s	user 0.039s	sys 0.074s
I20260709 12:24:20.417088  4799 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.401441  4803 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.419298  4799 ts_tablet_manager.cc:1434] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:24:20.420359  4799 tablet_bootstrap.cc:492] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:20.421420  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:24:20.422629  4720 consensus_peers.cc:597] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:20.427040  4571 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:20.437115  4880 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.437788  4501 catalog_manager.cc:5697] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:20.444552  4880 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.454890  4880 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:20.456166  4837 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:20.457858  4797 raft_consensus.cc:359] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.458572  4797 raft_consensus.cc:385] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.458842  4797 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.459693  4797 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.459473  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:20.447243  4822 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:20.462574  4797 ts_tablet_manager.cc:1434] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:24:20.465755  4872 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:24:20.466504  4720 leader_election.cc:343] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.466365  4872 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.469069  4872 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.471271  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:20.472625  4880 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:20.473707  4720 leader_election.cc:343] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.473560  4880 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.474187  4720 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:20.475907  4893 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:20.477416  4797 tablet_bootstrap.cc:492] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:20.476619  4880 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:20.478732  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:20.481267  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:20.480219  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:20.485705  4799 tablet_bootstrap.cc:654] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:20.492954  4720 leader_election.cc:343] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.493605  4796 tablet_bootstrap.cc:492] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:20.494104  4796 ts_tablet_manager.cc:1403] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.211s	user 0.032s	sys 0.022s
I20260709 12:24:20.497781  4796 raft_consensus.cc:359] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.498458  4796 raft_consensus.cc:385] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.498912  4796 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.499679  4796 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.501492  4796 ts_tablet_manager.cc:1434] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:24:20.502414  4796 tablet_bootstrap.cc:492] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:20.503245  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:20.504583  4720 leader_election.cc:343] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.505964  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:20.506685  4869 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:20.505038  4720 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:20.508872  4797 tablet_bootstrap.cc:654] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.509554  4880 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:20.511277  4872 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.511735  4872 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.517732  4872 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.518380  4796 tablet_bootstrap.cc:654] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.519761  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:20.520402  4756 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:20.521431  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:20.527488  4647 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:20.528430  4872 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:20.528831  4872 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.529215  4872 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.535583  4872 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:20.536513  4645 leader_election.cc:343] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.536410  4976 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:20.543004  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:20.543717  4756 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.546002  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:20.547206  4871 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:20.556205  4498 catalog_manager.cc:5697] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:20.557719  4872 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.559792  4756 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:20.561671  4647 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:20.562260  4799 tablet_bootstrap.cc:492] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:20.562791  4872 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:20.563051  4929 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.563598  4929 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.564590  4872 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:20.565752  4872 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.567196  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:20.568989  4929 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:24:20.569988  4720 leader_election.cc:343] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.570569  4797 tablet_bootstrap.cc:492] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
W20260709 12:24:20.570861  4645 leader_election.cc:343] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.562966  4799 ts_tablet_manager.cc:1403] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.143s	user 0.022s	sys 0.004s
I20260709 12:24:20.573661  4796 tablet_bootstrap.cc:492] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:20.574290  4796 ts_tablet_manager.cc:1403] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.072s	user 0.010s	sys 0.014s
I20260709 12:24:20.575644  4799 raft_consensus.cc:359] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.576380  4799 raft_consensus.cc:385] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.576711  4799 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.571033  4797 ts_tablet_manager.cc:1403] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.096s	user 0.018s	sys 0.005s
I20260709 12:24:20.577497  4799 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.578369  4796 raft_consensus.cc:359] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.579303  4796 raft_consensus.cc:385] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.579738  4796 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.580679  4796 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.583508  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:20.585515  4796 ts_tablet_manager.cc:1434] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:24:20.586990  4796 tablet_bootstrap.cc:492] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:20.589375  4799 ts_tablet_manager.cc:1434] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:24:20.590379  4799 tablet_bootstrap.cc:492] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:20.592239  4797 raft_consensus.cc:359] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:20.594003  4720 leader_election.cc:343] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.594450  4929 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.594904  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:20.595111  4929 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.596225  4893 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:20.597105  4797 raft_consensus.cc:385] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:24:20.599493  4569 consensus_peers.cc:597] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:20.600296  4797 raft_consensus.cc:740] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.600848  4929 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:20.604892  4797 consensus_queue.cc:260] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.603375  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:20.605827  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:20.608402  4720 leader_election.cc:343] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.609673  4797 ts_tablet_manager.cc:1434] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:24:20.610697  4797 tablet_bootstrap.cc:492] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:20.600436  4498 catalog_manager.cc:5697] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:20.615330  4799 tablet_bootstrap.cc:654] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:20.615689  4720 leader_election.cc:343] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.615339  4796 tablet_bootstrap.cc:654] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.616482  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:20.617297  4929 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:20.618263  4977 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:20.627378  4645 consensus_peers.cc:597] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:20.643500  4822 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:20.649093  4797 tablet_bootstrap.cc:654] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.655109  4501 catalog_manager.cc:5697] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:20.666584  4796 tablet_bootstrap.cc:492] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:20.667110  4796 ts_tablet_manager.cc:1403] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.080s	user 0.011s	sys 0.010s
I20260709 12:24:20.674069  4796 raft_consensus.cc:359] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.674722  4796 raft_consensus.cc:385] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.674954  4796 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.675454  4796 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.677760  4796 ts_tablet_manager.cc:1434] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:24:20.678732  4796 tablet_bootstrap.cc:492] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:20.679121  4799 tablet_bootstrap.cc:492] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:20.679526  4799 ts_tablet_manager.cc:1403] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.089s	user 0.017s	sys 0.012s
I20260709 12:24:20.682020  4604 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.685539  4872 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:20.694697  4796 tablet_bootstrap.cc:654] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.694836  4872 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:20.699193  4797 tablet_bootstrap.cc:492] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:20.699914  4797 ts_tablet_manager.cc:1403] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.089s	user 0.015s	sys 0.000s
I20260709 12:24:20.704301  4799 raft_consensus.cc:359] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.709491  4799 raft_consensus.cc:385] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.709852  4799 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.710536  4799 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.712582  4799 ts_tablet_manager.cc:1434] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.000s
I20260709 12:24:20.708747  4797 raft_consensus.cc:359] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.714449  4797 raft_consensus.cc:385] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.714720  4797 raft_consensus.cc:740] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.715379  4797 consensus_queue.cc:260] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.713488  4799 tablet_bootstrap.cc:492] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:20.729022  4929 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.729527  4929 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.732434  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:20.734225  4720 leader_election.cc:343] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.736344  4797 ts_tablet_manager.cc:1434] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.002s
I20260709 12:24:20.737449  4797 tablet_bootstrap.cc:492] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:20.739426  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:20.740903  4796 tablet_bootstrap.cc:492] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:20.741391  4796 ts_tablet_manager.cc:1403] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.006s
W20260709 12:24:20.741278  4720 leader_election.cc:343] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.741923  4720 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:20.742760  4893 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:20.743637  4929 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:20.745270  4799 tablet_bootstrap.cc:654] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.769228  4977 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.769860  4977 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.784258  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:20.786556  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:20.794127  4799 tablet_bootstrap.cc:492] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:20.794680  4799 ts_tablet_manager.cc:1403] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.081s	user 0.014s	sys 0.009s
I20260709 12:24:20.785322  4977 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.798044  4799 raft_consensus.cc:359] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.798738  4799 raft_consensus.cc:385] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.799029  4799 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.799703  4799 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.800307  4647 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:20.800702  4796 raft_consensus.cc:359] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.804744  4796 raft_consensus.cc:385] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.808969  4796 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.809734  4796 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.807164  4605 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.808465  4977 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:20.810330  4799 ts_tablet_manager.cc:1434] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:24:20.812747  4799 tablet_bootstrap.cc:492] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:20.818661  4799 tablet_bootstrap.cc:654] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.804291  4797 tablet_bootstrap.cc:654] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:20.807797  4550 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:20.821779  4796 ts_tablet_manager.cc:1434] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.080s	user 0.008s	sys 0.000s
I20260709 12:24:20.822732  4796 tablet_bootstrap.cc:492] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:20.835489  4799 tablet_bootstrap.cc:492] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:20.836047  4799 ts_tablet_manager.cc:1403] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.009s
I20260709 12:24:20.838888  4799 raft_consensus.cc:359] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.839638  4799 raft_consensus.cc:385] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.839973  4799 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.840757  4799 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.842837  4799 ts_tablet_manager.cc:1434] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:24:20.843992  4799 tablet_bootstrap.cc:492] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:20.849349  4796 tablet_bootstrap.cc:654] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.851418  4799 tablet_bootstrap.cc:654] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.863430  4796 tablet_bootstrap.cc:492] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:20.863909  4796 ts_tablet_manager.cc:1403] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.002s
I20260709 12:24:20.866731  4796 raft_consensus.cc:359] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.867376  4796 raft_consensus.cc:385] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.867619  4796 raft_consensus.cc:740] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.868218  4796 consensus_queue.cc:260] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.871459  4796 ts_tablet_manager.cc:1434] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:24:20.871995  4799 tablet_bootstrap.cc:492] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:20.872437  4799 ts_tablet_manager.cc:1403] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.005s
I20260709 12:24:20.872810  4796 tablet_bootstrap.cc:492] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:20.875089  4799 raft_consensus.cc:359] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.875734  4799 raft_consensus.cc:385] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.876019  4799 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.878626  4797 tablet_bootstrap.cc:492] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:20.879123  4797 ts_tablet_manager.cc:1403] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.142s	user 0.023s	sys 0.006s
I20260709 12:24:20.879505  4796 tablet_bootstrap.cc:654] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.881742  4797 raft_consensus.cc:359] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.882212  4869 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.882490  4797 raft_consensus.cc:385] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.882771  4797 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.882761  4869 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.883698  4797 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.884956  4869 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.886072  4797 ts_tablet_manager.cc:1434] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:24:20.886981  4797 tablet_bootstrap.cc:492] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:20.887079  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:20.889372  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:20.889560  4799 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.891093  4796 tablet_bootstrap.cc:492] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:20.891103  4647 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:20.891687  4796 ts_tablet_manager.cc:1403] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.006s
I20260709 12:24:20.892266  4869 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:20.894325  4796 raft_consensus.cc:359] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.895007  4796 raft_consensus.cc:385] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.895282  4796 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.896401  4799 ts_tablet_manager.cc:1434] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:24:20.896039  4796 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.897425  4799 tablet_bootstrap.cc:492] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:20.897464  4797 tablet_bootstrap.cc:654] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.898947  4796 ts_tablet_manager.cc:1434] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:24:20.899847  4796 tablet_bootstrap.cc:492] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:20.905475  4799 tablet_bootstrap.cc:654] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.909582  4796 tablet_bootstrap.cc:654] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.910280  4837 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.910948  4837 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.914448  4845 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.914951  4845 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.916952  4845 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.917934  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:20.918548  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:20.918866  4679 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:20.921308  4569 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:20.923686  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:20.925794  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:20.926354  4752 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:20.924546  4796 tablet_bootstrap.cc:492] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:20.927642  4796 ts_tablet_manager.cc:1403] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.005s
I20260709 12:24:20.927762  4845 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:20.928128  4845 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.928521  4845 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.928215  4569 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:20.924793  4837 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.929836  4964 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:20.930321  4796 raft_consensus.cc:359] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.931012  4796 raft_consensus.cc:385] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.931273  4796 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.932533  4796 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.934929  4796 ts_tablet_manager.cc:1434] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:24:20.935995  4845 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.938263  4929 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.938750  4929 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.939180  4845 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:20.940255  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:20.940900  4677 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.940994  4929 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:20.942212  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:20.942628  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:20.942869  4752 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.942978  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:20.944208  4720 leader_election.cc:343] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:20.944928  4720 leader_election.cc:343] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:20.945317  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:20.946586  4929 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:20.948487  4677 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:20.949787  4569 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:20.950142  4752 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:20.946599  4797 tablet_bootstrap.cc:492] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:20.950624  4837 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:20.951113  4797 ts_tablet_manager.cc:1403] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.064s	user 0.010s	sys 0.006s
I20260709 12:24:20.951222  4837 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:20.952107  4837 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.953800  4796 tablet_bootstrap.cc:492] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:20.955951  4797 raft_consensus.cc:359] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.956952  4797 raft_consensus.cc:385] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.957311  4797 raft_consensus.cc:740] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.958420  4797 consensus_queue.cc:260] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.962527  4797 ts_tablet_manager.cc:1434] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:24:20.963312  4799 tablet_bootstrap.cc:492] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:20.963644  4797 tablet_bootstrap.cc:492] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:20.964385  4799 ts_tablet_manager.cc:1403] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.067s	user 0.031s	sys 0.016s
I20260709 12:24:20.967298  4799 raft_consensus.cc:359] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.968051  4799 raft_consensus.cc:385] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:20.968341  4799 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:20.968986  4799 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:20.971398  4799 ts_tablet_manager.cc:1434] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:24:20.971638  4605 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:20.973469  4799 tablet_bootstrap.cc:492] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:20.978127  4796 tablet_bootstrap.cc:654] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.980759  4799 tablet_bootstrap.cc:654] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.981433  4797 tablet_bootstrap.cc:654] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:20.991423  4929 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:20.997645  4797 tablet_bootstrap.cc:492] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:21.003296  4797 ts_tablet_manager.cc:1403] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.000s
I20260709 12:24:21.006016  4797 raft_consensus.cc:359] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.006702  4797 raft_consensus.cc:385] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.006968  4797 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.009770  4796 tablet_bootstrap.cc:492] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:21.010777  4796 ts_tablet_manager.cc:1403] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.005s
I20260709 12:24:21.012252  4797 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.013370  4799 tablet_bootstrap.cc:492] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:21.014237  4799 ts_tablet_manager.cc:1403] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.017s
I20260709 12:24:21.015502  4797 ts_tablet_manager.cc:1434] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:24:21.014115  4796 raft_consensus.cc:359] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.016122  4796 raft_consensus.cc:385] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.016387  4797 tablet_bootstrap.cc:492] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:21.016459  4796 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.017331  4799 raft_consensus.cc:359] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.018095  4799 raft_consensus.cc:385] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.017606  4796 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.018374  4799 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.019165  4799 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.021997  4799 ts_tablet_manager.cc:1434] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:24:21.022886  4799 tablet_bootstrap.cc:492] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:21.023100  4797 tablet_bootstrap.cc:654] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.029969  4799 tablet_bootstrap.cc:654] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.030638  4677 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.035233  4797 tablet_bootstrap.cc:492] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:21.035873  4797 ts_tablet_manager.cc:1403] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 12:24:21.038532  4796 ts_tablet_manager.cc:1434] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:24:21.042712  4929 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:21.113193  4893 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.113821  4893 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.115554  4679 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.115928  4893 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:21.119130  4501 catalog_manager.cc:5697] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:21.122439  4822 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.153903  4822 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.156730  4822 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:21.123678  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 81354.1us
I20260709 12:24:21.129801  4845 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.157783  4845 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.159969  4845 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:21.132895  4799 tablet_bootstrap.cc:492] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:21.161052  4799 ts_tablet_manager.cc:1403] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.138s	user 0.014s	sys 0.008s
I20260709 12:24:21.164198  4872 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.164839  4872 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.167007  4604 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.168677  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:21.164848  4799 raft_consensus.cc:359] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.169507  4799 raft_consensus.cc:385] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.134871  4797 raft_consensus.cc:359] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.169904  4799 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.170210  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:21.169615  4752 raft_consensus.cc:2393] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 1: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:21.170318  4797 raft_consensus.cc:385] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.171759  4797 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.168733  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:21.172178  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:21.172725  4753 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:21.173159  4872 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:21.138197  4796 tablet_bootstrap.cc:492] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:21.174118  4799 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.175688  4799 ts_tablet_manager.cc:1434] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:24:21.176553  4799 tablet_bootstrap.cc:492] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:21.172749  4756 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:21.172734  4797 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.179188  4647 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:21.180053  4872 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:21.180171  4797 ts_tablet_manager.cc:1434] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.144s	user 0.004s	sys 0.016s
I20260709 12:24:21.180475  4872 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.181475  4797 tablet_bootstrap.cc:492] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:21.182996  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:21.152276  4977 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:21.181433  4872 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.183411  5023 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.194837  4571 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:24:21.186113  4720 leader_election.cc:343] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:21.189090  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 61745.8us
I20260709 12:24:21.212853  4822 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:21.223743  4822 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.224136  4822 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.212360  4893 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.227996  4893 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.193176  5023 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.232069  4893 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:21.233528  5023 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:21.242679  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:21.243866  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:21.250342  4797 tablet_bootstrap.cc:654] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.251219  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:21.255440  4645 leader_election.cc:343] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:21.256592  4799 tablet_bootstrap.cc:654] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.259691  4977 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.260360  4872 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.260354  4977 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.268157  4796 tablet_bootstrap.cc:654] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.270426  4976 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:21.270843  4720 leader_election.cc:343] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:21.241330  5031 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.278119  4977 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:21.282537  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:21.282356  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:21.287614  4604 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.290251  4822 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.292913  4822 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:21.297946  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:21.305281  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:21.305487  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:21.305878  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:21.286423  4720 leader_election.cc:343] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:21.306425  4678 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 0.
W20260709 12:24:21.307654  4720 leader_election.cc:343] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:21.308080  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:21.309798  5023 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:21.311089  4720 leader_election.cc:343] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:21.311503  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:21.314088  4569 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:21.314792  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:21.318565  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:21.316880  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:21.319826  4753 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.323570  4872 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:21.325016  4647 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:21.327315  5026 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:21.328248  4645 leader_election.cc:343] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:21.335484  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:21.285380  5031 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.337637  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:21.339308  4755 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.333722  4872 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.341738  4893 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.337864  5023 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:21.344799  4872 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.343365  4893 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.356961  4929 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:21.359694  4929 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.360136  4929 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.353698  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:21.372579  4601 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:21.375208  4679 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:21.376293  5031 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:21.378016  4893 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:21.378966  4872 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:21.380586  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:21.380947  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:21.381532  4754 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:21.387027  4647 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:21.391947  4677 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.393131  4753 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:21.396641  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:21.397450  4601 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 1: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:21.400053  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:21.400556  4571 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:21.401273  5031 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:21.401728  5031 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.402313  5025 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:21.402771  5025 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:21.403542  5025 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.405722  4872 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.409986  4755 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:21.411684  5031 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.434770  4647 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:21.441052  4845 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:21.449394  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 79179.1us
I20260709 12:24:21.454166  4799 tablet_bootstrap.cc:492] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:21.454567  4799 ts_tablet_manager.cc:1403] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.278s	user 0.008s	sys 0.009s
I20260709 12:24:21.457963  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:21.472119  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:21.455735  4976 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:21.476569  4796 tablet_bootstrap.cc:492] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:21.477012  4796 ts_tablet_manager.cc:1403] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.339s	user 0.043s	sys 0.016s
I20260709 12:24:21.478803  4797 tablet_bootstrap.cc:492] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:21.479421  4797 ts_tablet_manager.cc:1403] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.298s	user 0.042s	sys 0.059s
I20260709 12:24:21.474200  4976 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:21.475243  4929 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.499677  4799 raft_consensus.cc:359] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.500388  4799 raft_consensus.cc:385] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.500659  4799 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.501830  4799 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:21.503901  4720 leader_election.cc:343] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:21.505422  4720 leader_election.cc:343] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:21.505818  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:21.502692  4976 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.506733  4893 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:21.514680  4799 ts_tablet_manager.cc:1434] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.000s
I20260709 12:24:21.516045  4799 tablet_bootstrap.cc:492] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:21.526279  4797 raft_consensus.cc:359] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.526896  4797 raft_consensus.cc:385] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.527122  4797 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
W20260709 12:24:21.527891  5032 log.cc:927] Time spent T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:24:21.528648  4797 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.531654  5031 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.533710  4796 raft_consensus.cc:359] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.534382  4796 raft_consensus.cc:385] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.534492  5031 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:21.534673  4796 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.535356  4796 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.546553  4796 ts_tablet_manager.cc:1434] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 12:24:21.547335  4796 tablet_bootstrap.cc:492] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:21.547399  4797 ts_tablet_manager.cc:1434] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.002s
I20260709 12:24:21.548231  4797 tablet_bootstrap.cc:492] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:21.552999  4872 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:21.560376  4837 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:21.564378  4845 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.569372  4845 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.573134  4845 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:21.573665  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:21.574433  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:21.564345  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:21.585682  4601 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.589033  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:21.597750  4799 tablet_bootstrap.cc:654] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.594448  4680 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:21.589818  4756 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.601608  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:21.586476  4604 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.590193  4929 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:21.618095  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:21.626633  4845 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:21.636436  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:21.637283  4754 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 1: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:21.629333  4679 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.653038  4796 tablet_bootstrap.cc:654] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.654335  4601 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:21.673468  4837 raft_consensus.cc:2764] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:24:21.679749  4679 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:21.683477  4604 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:21.683780  4756 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:21.685678  4647 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:21.689237  4499 catalog_manager.cc:5697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:21.693526  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:21.694622  4929 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:21.688066  4872 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:21.695117  4929 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:21.699098  4872 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:21.699997  4872 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.700618  4929 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.709209  4799 tablet_bootstrap.cc:492] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:21.709740  4799 ts_tablet_manager.cc:1403] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.194s	user 0.020s	sys 0.003s
I20260709 12:24:21.712983  4976 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.712795  4799 raft_consensus.cc:359] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.713676  4976 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.714542  4799 raft_consensus.cc:385] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.715626  4799 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.718870  4976 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:21.723155  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:21.723887  4753 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:21.725101  4799 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.735891  4799 ts_tablet_manager.cc:1434] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.003s
I20260709 12:24:21.737123  4799 tablet_bootstrap.cc:492] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:21.738448  4797 tablet_bootstrap.cc:654] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.742169  4647 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:21.755389  4869 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:21.756732  4869 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.757084  4869 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.764572  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:21.770773  4869 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:21.773976  4645 leader_election.cc:343] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:21.775664  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:21.775955  4799 tablet_bootstrap.cc:654] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.776331  4755 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.777271  4869 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:21.776368  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:21.778745  4797 tablet_bootstrap.cc:492] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:21.779189  4797 ts_tablet_manager.cc:1403] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.231s	user 0.007s	sys 0.006s
I20260709 12:24:21.783311  4796 tablet_bootstrap.cc:492] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:21.783821  4796 ts_tablet_manager.cc:1403] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.237s	user 0.032s	sys 0.016s
W20260709 12:24:21.784709  4645 leader_election.cc:343] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:21.788182  4797 raft_consensus.cc:359] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.789063  4755 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:21.789423  4797 raft_consensus.cc:385] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.789798  4797 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.790433  4797 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.791188  4647 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:21.792918  4869 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:21.793666  4869 raft_consensus.cc:697] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:21.794518  4797 ts_tablet_manager.cc:1434] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 12:24:21.794570  4869 consensus_queue.cc:237] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.795702  4845 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:21.798137  4796 raft_consensus.cc:359] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.799156  4796 raft_consensus.cc:385] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.799568  4796 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.795742  4797 tablet_bootstrap.cc:492] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:21.808111  4797 tablet_bootstrap.cc:654] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.809842  4796 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.811566  4796 ts_tablet_manager.cc:1434] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.001s
I20260709 12:24:21.812510  4796 tablet_bootstrap.cc:492] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:21.812642  4837 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:21.815217  4500 catalog_manager.cc:5697] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:21.847903  4799 tablet_bootstrap.cc:492] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:21.848762  4799 ts_tablet_manager.cc:1403] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.112s	user 0.010s	sys 0.026s
I20260709 12:24:21.849506  4500 catalog_manager.cc:5697] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:21.851634  4796 tablet_bootstrap.cc:654] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.851470  4799 raft_consensus.cc:359] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.851729  4500 catalog_manager.cc:5697] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:21.852500  4799 raft_consensus.cc:385] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.852950  4799 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.855665  4797 tablet_bootstrap.cc:492] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:21.856173  4797 ts_tablet_manager.cc:1403] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.061s	user 0.012s	sys 0.000s
I20260709 12:24:21.855917  4799 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.859434  4797 raft_consensus.cc:359] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.860092  4797 raft_consensus.cc:385] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.860384  4797 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.861862  4797 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.864120  4797 ts_tablet_manager.cc:1434] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:24:21.865056  4797 tablet_bootstrap.cc:492] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:21.867851  4799 ts_tablet_manager.cc:1434] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.003s
I20260709 12:24:21.868623  4796 tablet_bootstrap.cc:492] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:21.868827  4799 tablet_bootstrap.cc:492] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:21.869233  4796 ts_tablet_manager.cc:1403] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.001s
I20260709 12:24:21.875972  4799 tablet_bootstrap.cc:654] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.877015  4797 tablet_bootstrap.cc:654] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.876581  4796 raft_consensus.cc:359] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.877451  4796 raft_consensus.cc:385] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.877817  4796 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.878890  4796 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.885959  4796 ts_tablet_manager.cc:1434] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:24:21.887414  4796 tablet_bootstrap.cc:492] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:21.889014  4799 tablet_bootstrap.cc:492] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:21.890606  4799 ts_tablet_manager.cc:1403] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.000s
I20260709 12:24:21.893119  5039 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.894001  4796 tablet_bootstrap.cc:654] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.897097  5039 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.893672  4799 raft_consensus.cc:359] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.898061  4799 raft_consensus.cc:385] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.898458  4799 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.901253  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:21.901351  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:21.901839  4797 tablet_bootstrap.cc:492] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:21.901536  4799 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.902354  4797 ts_tablet_manager.cc:1403] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.004s
W20260709 12:24:21.902711  4720 leader_election.cc:343] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:21.903383  4720 leader_election.cc:343] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:21.903738  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:21.904412  4929 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:21.904798  4799 ts_tablet_manager.cc:1434] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:24:21.892421  4499 catalog_manager.cc:5697] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:21.905265  4797 raft_consensus.cc:359] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.906039  4797 raft_consensus.cc:385] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.906432  4797 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.907254  4797 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.899740  5039 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:21.906060  4799 tablet_bootstrap.cc:492] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:21.911896  4797 ts_tablet_manager.cc:1434] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:24:21.913286  4796 tablet_bootstrap.cc:492] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:21.914059  4796 ts_tablet_manager.cc:1403] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.001s
I20260709 12:24:21.913291  4797 tablet_bootstrap.cc:492] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:21.916153  4799 tablet_bootstrap.cc:654] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:21.924749  4837 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.925550  4837 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.926852  4601 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.930646  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:21.931715  4837 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:21.933707  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:21.935143  4755 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 1: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:21.937218  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:21.939069  5024 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:21.949157  5024 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:21.962649  4869 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:21.976575  4797 tablet_bootstrap.cc:654] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:21.964865  4550 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:21.967301  4796 raft_consensus.cc:359] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.981062  4796 raft_consensus.cc:385] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.981405  4796 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.982620  4796 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.985543  4796 ts_tablet_manager.cc:1434] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.001s
I20260709 12:24:21.987104  4796 tablet_bootstrap.cc:492] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:21.991530  4797 tablet_bootstrap.cc:492] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:21.992018  4797 ts_tablet_manager.cc:1403] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.079s	user 0.011s	sys 0.003s
I20260709 12:24:21.994796  4797 raft_consensus.cc:359] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.995571  4797 raft_consensus.cc:385] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:21.995857  4797 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:21.996465  4797 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:21.998606  4797 ts_tablet_manager.cc:1434] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:24:21.999476  4797 tablet_bootstrap.cc:492] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:22.005329  4797 tablet_bootstrap.cc:654] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.005949  5039 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.006696  5039 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.010028  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:22.011837  4720 leader_election.cc:343] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.012428  5039 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:22.012784  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:22.015054  4720 leader_election.cc:343] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.015612  4720 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:22.016719  4929 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:22.017586  4869 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:22.027714  5039 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.028290  5039 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.030383  5039 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:22.030841  4976 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.031745  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:22.033191  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:22.040228  4680 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 0.
I20260709 12:24:22.040890  4799 tablet_bootstrap.cc:492] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:22.041999  4799 ts_tablet_manager.cc:1403] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.136s	user 0.021s	sys 0.046s
W20260709 12:24:22.042855  4720 leader_election.cc:343] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.041903  4976 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.048295  4976 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:22.049374  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:22.050316  4601 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:22.052800  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:22.053510  4756 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:22.055824  4796 tablet_bootstrap.cc:654] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.056761  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:22.057855  4976 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:22.057962  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:22.058316  4976 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.058787  4976 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.059259  5039 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:22.059885  5039 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.060444  5039 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.066394  4799 raft_consensus.cc:359] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.067214  4799 raft_consensus.cc:385] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.067530  4799 raft_consensus.cc:740] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:22.068248  4799 consensus_queue.cc:260] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.075269  4799 ts_tablet_manager.cc:1434] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.002s
I20260709 12:24:22.076299  4799 tablet_bootstrap.cc:492] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:22.084085  4797 tablet_bootstrap.cc:492] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:22.084631  4797 ts_tablet_manager.cc:1403] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.085s	user 0.037s	sys 0.045s
I20260709 12:24:22.084885  4929 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:22.087880  4797 raft_consensus.cc:359] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.088773  4797 raft_consensus.cc:385] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.089103  4797 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:22.089955  4797 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.092869  4797 ts_tablet_manager.cc:1434] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:24:22.094435  4976 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.097807  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:22.098444  4604 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.099821  4799 tablet_bootstrap.cc:654] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.103698  4796 tablet_bootstrap.cc:492] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:22.104158  4796 ts_tablet_manager.cc:1403] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.117s	user 0.012s	sys 0.011s
I20260709 12:24:22.104631  4797 tablet_bootstrap.cc:492] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:22.107124  4796 raft_consensus.cc:359] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.107883  4796 raft_consensus.cc:385] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.108167  4796 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:22.109026  4796 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.109225  4604 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:22.118666  4645 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:22.119750  4872 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:22.120273  4872 raft_consensus.cc:697] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:22.121923  4872 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.127683  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:22.129944  4755 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.122929  4976 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:22.137948  4796 ts_tablet_manager.cc:1434] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.008s
I20260709 12:24:22.139062  4796 tablet_bootstrap.cc:492] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:22.149415  4755 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:22.150523  4498 catalog_manager.cc:5697] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:22.155848  4929 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.157246  4929 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.156008  4799 tablet_bootstrap.cc:492] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:22.159310  4929 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:22.159399  4799 ts_tablet_manager.cc:1403] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.083s	user 0.016s	sys 0.014s
I20260709 12:24:22.164373  4797 tablet_bootstrap.cc:654] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.164855  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:22.166769  4720 leader_election.cc:343] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.160427  5039 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.161902  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:22.170500  4720 leader_election.cc:343] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.170945  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:22.172669  4929 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:22.173377  5045 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:22.174079  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:22.174868  5039 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:22.176792  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:22.177412  4680 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:22.176160  4720 leader_election.cc:343] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.181327  4799 raft_consensus.cc:359] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.182142  4799 raft_consensus.cc:385] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.182432  4799 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:22.183130  4799 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.183692  4680 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:22.185608  4799 ts_tablet_manager.cc:1434] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.000s
I20260709 12:24:22.186733  4799 tablet_bootstrap.cc:492] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:22.187742  4797 tablet_bootstrap.cc:492] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:22.188277  4797 ts_tablet_manager.cc:1403] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.084s	user 0.009s	sys 0.005s
I20260709 12:24:22.191841  4796 tablet_bootstrap.cc:654] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.192058  4797 raft_consensus.cc:359] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.192808  4797 raft_consensus.cc:385] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.193142  4797 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:22.193894  4797 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.196264  4797 ts_tablet_manager.cc:1434] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:24:22.197216  4797 tablet_bootstrap.cc:492] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:22.198383  4872 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:22.203298  4797 tablet_bootstrap.cc:654] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.200937  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:22.217126  4872 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:22.221840  5045 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:22.223199  4796 tablet_bootstrap.cc:492] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:22.223847  4796 ts_tablet_manager.cc:1403] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.085s	user 0.009s	sys 0.004s
I20260709 12:24:22.227013  4872 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.227408  4799 tablet_bootstrap.cc:654] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.227658  4872 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.222396  5045 raft_consensus.cc:697] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:22.235404  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:22.233220  5045 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.265784  4872 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:22.269941  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:22.270776  4755 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:22.262444  4796 raft_consensus.cc:359] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.274988  4796 raft_consensus.cc:385] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.275465  4796 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
W20260709 12:24:22.244994  4645 leader_election.cc:343] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:22.284442  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 46763.6us
I20260709 12:24:22.283612  4872 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.286356  4796 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.285683  5024 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.292889  4796 ts_tablet_manager.cc:1434] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.069s	user 0.008s	sys 0.005s
I20260709 12:24:22.288138  4872 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.304831  4872 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:22.306255  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:22.307756  4645 leader_election.cc:343] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.308005  4796 tablet_bootstrap.cc:492] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:22.292116  5024 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.310781  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:22.311977  5024 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:22.312613  4872 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:22.313064  4872 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.313639  4872 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.312911  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:22.314342  4681 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:22.316834  4799 tablet_bootstrap.cc:492] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:22.316953  4569 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:22.317421  4799 ts_tablet_manager.cc:1403] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.131s	user 0.008s	sys 0.009s
I20260709 12:24:22.319090  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:22.319731  4756 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:22.319684  5024 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:22.320688  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:22.321779  4756 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:22.321122  5024 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.322810  5024 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.325433  4799 raft_consensus.cc:359] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.327680  4799 raft_consensus.cc:385] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.327991  4799 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:22.328100  4797 tablet_bootstrap.cc:492] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:22.328657  4797 ts_tablet_manager.cc:1403] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.132s	user 0.030s	sys 0.031s
I20260709 12:24:22.329797  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:22.331745  4799 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.330967  4976 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:22.333608  4976 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.334010  4976 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.335780  4797 raft_consensus.cc:359] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.339116  4797 raft_consensus.cc:385] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.337678  4872 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.343757  4797 raft_consensus.cc:740] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:22.344106  4499 catalog_manager.cc:5697] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:22.344612  4797 consensus_queue.cc:260] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.346125  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:22.338435  4799 ts_tablet_manager.cc:1434] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:24:22.342404  4869 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:22.347132  4797 ts_tablet_manager.cc:1434] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:24:22.348376  4797 tablet_bootstrap.cc:492] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
W20260709 12:24:22.348232  4645 leader_election.cc:343] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.343417  4796 tablet_bootstrap.cc:654] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.349484  4872 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:22.350636  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:22.351262  4754 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.353679  4799 tablet_bootstrap.cc:492] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:22.355901  4797 tablet_bootstrap.cc:654] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.359669  4754 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:22.336616  5024 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.373147  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:22.373862  4681 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.370755  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:22.365084  4976 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.380261  4796 tablet_bootstrap.cc:492] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:22.390868  4796 ts_tablet_manager.cc:1403] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.083s	user 0.009s	sys 0.003s
I20260709 12:24:22.384902  4799 tablet_bootstrap.cc:654] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.393182  4976 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:22.394232  5024 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:22.394778  4845 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.394711  4796 raft_consensus.cc:359] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.396967  4796 raft_consensus.cc:385] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.397243  4796 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:22.398623  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:22.399219  4604 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.399626  4681 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:22.395371  4845 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.397898  4796 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.394719  4872 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:22.401847  4872 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:22.402642  4872 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.403156  4845 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:22.403604  4845 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.403971  4845 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.407742  5023 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.408246  5023 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.410193  5023 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:22.413244  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:22.414815  4720 leader_election.cc:343] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.417774  4845 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:22.418574  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:22.419329  4680 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 1: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:22.419050  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:22.419945  4677 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:22.421255  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:22.422099  4845 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:22.422416  4845 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.423691  4845 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.430241  4796 ts_tablet_manager.cc:1434] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 12:24:22.431751  4796 tablet_bootstrap.cc:492] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:22.450147  4569 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:22.452188  5024 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:22.453148  5024 raft_consensus.cc:697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:22.454932  5024 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.455438  4797 tablet_bootstrap.cc:492] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:22.458696  4797 ts_tablet_manager.cc:1403] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.111s	user 0.042s	sys 0.037s
I20260709 12:24:22.424937  4604 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:22.460119  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:22.461133  4754 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.461740  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:22.462324  4756 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.464653  4976 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:22.470471  4756 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:22.476665  4845 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.530792  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:22.535346  4845 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:22.536128  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:22.542938  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:22.543598  4681 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.549821  4799 tablet_bootstrap.cc:492] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:22.550467  4799 ts_tablet_manager.cc:1403] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.197s	user 0.031s	sys 0.038s
I20260709 12:24:22.552393  4796 tablet_bootstrap.cc:654] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.553632  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:24:22.554216  4720 leader_election.cc:343] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.554267  4756 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:22.554759  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:22.555665  5023 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:22.556476  4799 raft_consensus.cc:359] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.557163  4799 raft_consensus.cc:385] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.557516  4799 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:22.559177  4799 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.562312  4645 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:22.563877  5026 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:22.564651  5026 raft_consensus.cc:697] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:22.552296  4754 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:22.569350  4799 ts_tablet_manager.cc:1434] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:24:22.582826  4799 tablet_bootstrap.cc:492] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:22.578184  5026 consensus_queue.cc:237] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.576745  4681 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:22.599581  4976 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:22.600167  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:22.600740  4756 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.603310  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:22.605312  5087 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:22.614857  4756 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:22.605898  5087 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:22.641772  4498 catalog_manager.cc:5697] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:22.651242  4797 raft_consensus.cc:359] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.652055  4797 raft_consensus.cc:385] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.652325  4797 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:22.652769  4796 tablet_bootstrap.cc:492] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:22.653287  4796 ts_tablet_manager.cc:1403] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.222s	user 0.012s	sys 0.008s
I20260709 12:24:22.623392  5087 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.672122  4797 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.680497  4499 catalog_manager.cc:5697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:22.682816  4797 ts_tablet_manager.cc:1434] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.224s	user 0.008s	sys 0.004s
W20260709 12:24:22.682168  5088 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 12:24:22.684882  4799 tablet_bootstrap.cc:654] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.684126  4797 tablet_bootstrap.cc:492] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:22.695755  4796 raft_consensus.cc:359] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.696367  4796 raft_consensus.cc:385] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.696615  4796 raft_consensus.cc:740] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:22.698144  4796 consensus_queue.cc:260] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.703641  4796 ts_tablet_manager.cc:1434] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 12:24:22.726279  4796 tablet_bootstrap.cc:492] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:22.732345  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:22.735936  4501 catalog_manager.cc:5697] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:22.736860  5024 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.757512  5024 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:22.775281  4797 tablet_bootstrap.cc:654] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.782833  4799 tablet_bootstrap.cc:492] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:22.783345  4799 ts_tablet_manager.cc:1403] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.201s	user 0.042s	sys 0.024s
I20260709 12:24:22.779774  5045 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.786243  5045 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.778390  4929 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.799755  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:22.794507  5039 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260709 12:24:22.802627  4720 leader_election.cc:343] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.805385  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:22.810508  4720 leader_election.cc:343] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.810730  4799 raft_consensus.cc:359] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.810958  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:22.811426  4799 raft_consensus.cc:385] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.811712  4799 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:22.809676  5045 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:22.798305  4929 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.814605  4799 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.812844  5023 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:22.819623  4929 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:24:22.798363  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 62606.4us
I20260709 12:24:22.825098  4799 ts_tablet_manager.cc:1434] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.041s	user 0.003s	sys 0.007s
I20260709 12:24:22.826254  4799 tablet_bootstrap.cc:492] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:22.829319  4604 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.835652  5045 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:22.846096  4797 tablet_bootstrap.cc:492] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:22.846627  4797 ts_tablet_manager.cc:1403] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.163s	user 0.015s	sys 0.017s
I20260709 12:24:22.848783  5098 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.850028  4872 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.850080  4797 raft_consensus.cc:359] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.851006  4797 raft_consensus.cc:385] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:22.850739  4872 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.851320  4797 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:22.852064  4797 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.854516  4797 ts_tablet_manager.cc:1434] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:24:22.855494  4797 tablet_bootstrap.cc:492] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:22.857306  5026 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:22.857805  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:22.858489  4605 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:22.860242  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:22.861773  4797 tablet_bootstrap.cc:654] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.861619  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:22.865200  4720 leader_election.cc:343] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:22.866461  4720 leader_election.cc:343] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.866945  4720 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:22.869017  5039 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:22.873556  4645 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:22.874799  5026 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:22.875314  5026 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.875744  5026 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.876967  4872 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:22.875998  4501 catalog_manager.cc:5697] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:22.882700  5045 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.849391  5098 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.898756  4796 tablet_bootstrap.cc:654] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.884533  4869 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:22.886861  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:22.902549  4754 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:22.916409  5031 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:22.896766  4799 tablet_bootstrap.cc:654] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:22.926414  5026 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.930418  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:22.931324  4603 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:22.932163  5098 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:22.897477  5045 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:22.933477  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:22.935879  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:22.936278  5045 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:22.936986  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:22.938365  4720 leader_election.cc:343] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.938292  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:22.939239  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:22.940305  5024 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:22.941805  5024 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:22.948580  4720 leader_election.cc:343] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:22.949980  4720 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:22.951323  5039 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:22.998335  4799 tablet_bootstrap.cc:492] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:22.998864  4799 ts_tablet_manager.cc:1403] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.173s	user 0.013s	sys 0.000s
I20260709 12:24:23.001119  5026 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.002048  4872 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.003646  4872 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.014546  4872 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.019537  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:23.026448  4796 tablet_bootstrap.cc:492] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:23.028018  4796 ts_tablet_manager.cc:1403] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.302s	user 0.015s	sys 0.000s
W20260709 12:24:23.029464  4645 leader_election.cc:343] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:23.037616  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:23.038381  4754 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.038914  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:23.039633  4753 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:22.989334  4603 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:23.077206  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:23.078032  4872 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:23.078352  4872 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.078642  4872 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.080947  4796 raft_consensus.cc:359] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.081669  4796 raft_consensus.cc:385] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.081928  4796 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.082577  4796 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.084730  4796 ts_tablet_manager.cc:1434] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.000s
I20260709 12:24:23.087832  4797 tablet_bootstrap.cc:492] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:23.088462  4797 ts_tablet_manager.cc:1403] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.233s	user 0.057s	sys 0.131s
I20260709 12:24:23.091707  4797 raft_consensus.cc:359] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.092482  4797 raft_consensus.cc:385] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.092787  4797 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.093528  4797 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.094168  4872 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.096040  4797 ts_tablet_manager.cc:1434] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:24:23.096747  4872 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.097081  4797 tablet_bootstrap.cc:492] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:23.103665  5093 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:23.104271  4645 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:23.104897  4797 tablet_bootstrap.cc:654] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.121519  4977 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:23.125910  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:23.128973  4977 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:23.129735  4796 tablet_bootstrap.cc:492] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:23.130118  4977 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:23.141731  4645 leader_election.cc:343] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:23.142916  4799 raft_consensus.cc:359] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.143438  4796 tablet_bootstrap.cc:654] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.143725  4799 raft_consensus.cc:385] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.144101  4799 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.144791  4799 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.150257  4754 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:23.150693  4797 tablet_bootstrap.cc:492] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:23.152210  4797 ts_tablet_manager.cc:1403] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.055s	user 0.028s	sys 0.007s
I20260709 12:24:23.157783  4797 raft_consensus.cc:359] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.158593  4797 raft_consensus.cc:385] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.158901  4797 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
W20260709 12:24:23.145942  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 68836.1us
I20260709 12:24:23.159601  4797 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.169358  4799 ts_tablet_manager.cc:1434] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.170s	user 0.009s	sys 0.000s
I20260709 12:24:23.171387  4797 ts_tablet_manager.cc:1434] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.001s
I20260709 12:24:23.171357  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:23.172204  4753 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.173208  4797 tablet_bootstrap.cc:492] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:23.175366  4799 tablet_bootstrap.cc:492] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:23.177371  4796 tablet_bootstrap.cc:492] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:23.177867  4796 ts_tablet_manager.cc:1403] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.048s	user 0.018s	sys 0.003s
I20260709 12:24:23.187988  4753 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:23.184741  4499 catalog_manager.cc:5697] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:23.195729  4869 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:23.197316  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:23.196871  4977 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:23.199005  4799 tablet_bootstrap.cc:654] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.199648  5031 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:23.201166  5031 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:23.202168  5031 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.207360  4796 raft_consensus.cc:359] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.208431  4796 raft_consensus.cc:385] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.208726  4796 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.210707  4796 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.213092  4797 tablet_bootstrap.cc:654] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.223827  4796 ts_tablet_manager.cc:1434] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.001s
I20260709 12:24:23.243039  4796 tablet_bootstrap.cc:492] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:23.262825  4977 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.270437  4977 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.272084  4799 tablet_bootstrap.cc:492] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:23.272706  4799 ts_tablet_manager.cc:1403] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.098s	user 0.016s	sys 0.010s
I20260709 12:24:23.272785  4977 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.275991  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:23.276748  4605 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:23.278445  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:23.279891  5026 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:23.280318  5026 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.280733  5026 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.281924  4799 raft_consensus.cc:359] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.283919  4799 raft_consensus.cc:385] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.284266  4799 raft_consensus.cc:740] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.285951  4799 consensus_queue.cc:260] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.303541  4837 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:23.306376  5045 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.306910  5045 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.311777  4799 ts_tablet_manager.cc:1434] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.039s	user 0.003s	sys 0.004s
I20260709 12:24:23.312994  5045 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:23.314118  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:23.316460  4720 leader_election.cc:343] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:23.316787  4797 tablet_bootstrap.cc:492] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:23.317268  4797 ts_tablet_manager.cc:1403] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.144s	user 0.025s	sys 0.023s
I20260709 12:24:23.320331  4837 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:23.321307  4796 tablet_bootstrap.cc:654] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.324115  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:23.323895  4797 raft_consensus.cc:359] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.333801  4797 raft_consensus.cc:385] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.333930  4753 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:23.334214  4797 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.335016  4797 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.337795  4797 ts_tablet_manager.cc:1434] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:24:23.329268  4799 tablet_bootstrap.cc:492] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:23.329990  5026 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.339224  4797 tablet_bootstrap.cc:492] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:23.353888  4797 tablet_bootstrap.cc:654] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.353703  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:23.357718  4720 leader_election.cc:343] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:23.358206  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:23.359416  5045 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:23.364725  4501 catalog_manager.cc:5697] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:23.379292  4799 tablet_bootstrap.cc:654] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.385277  4796 tablet_bootstrap.cc:492] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:23.386062  4796 ts_tablet_manager.cc:1403] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.146s	user 0.014s	sys 0.002s
I20260709 12:24:23.398049  4796 raft_consensus.cc:359] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.398825  4796 raft_consensus.cc:385] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.399129  4796 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.399425  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:23.400086  4753 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.399806  4796 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.402916  4837 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.403527  4837 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.412343  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:23.413398  4550 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:24:23.414179  4569 leader_election.cc:343] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:23.415746  4753 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:23.416968  4837 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.417665  4647 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:23.419075  4872 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:23.419471  4872 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:23.420086  4797 tablet_bootstrap.cc:492] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:23.420697  4797 ts_tablet_manager.cc:1403] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.082s	user 0.010s	sys 0.003s
I20260709 12:24:23.420126  4872 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.424516  4797 raft_consensus.cc:359] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.425439  4797 raft_consensus.cc:385] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.425760  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:23.425828  4797 raft_consensus.cc:740] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.426388  4755 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:23.427060  4797 consensus_queue.cc:260] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.427697  4571 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:23.428961  5024 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:23.430891  4797 ts_tablet_manager.cc:1434] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.004s
I20260709 12:24:23.429323  5024 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.433626  4799 tablet_bootstrap.cc:492] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:23.433730  5024 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.406487  4796 ts_tablet_manager.cc:1434] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:24:23.434317  4799 ts_tablet_manager.cc:1403] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.105s	user 0.012s	sys 0.014s
I20260709 12:24:23.435303  4796 tablet_bootstrap.cc:492] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:23.439224  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:23.439687  4797 tablet_bootstrap.cc:492] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:23.439870  4603 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.437763  4799 raft_consensus.cc:359] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.441366  4799 raft_consensus.cc:385] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.441663  4799 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.446923  4796 tablet_bootstrap.cc:654] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.450188  4799 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.451124  5093 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.451723  5093 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.452785  4799 ts_tablet_manager.cc:1434] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:24:23.453847  4799 tablet_bootstrap.cc:492] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:23.454216  5093 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.459412  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:23.460929  4603 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:23.462280  4797 tablet_bootstrap.cc:654] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:23.463900  4571 leader_election.cc:343] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:23.401938  5026 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.486690  4796 tablet_bootstrap.cc:492] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:23.487269  4796 ts_tablet_manager.cc:1403] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.052s	user 0.017s	sys 0.000s
I20260709 12:24:23.487416  4799 tablet_bootstrap.cc:654] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.484988  5024 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.493772  5098 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.494553  5098 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.497659  4796 raft_consensus.cc:359] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.501832  4796 raft_consensus.cc:385] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.502741  4796 raft_consensus.cc:740] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.501931  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:23.524123  4797 tablet_bootstrap.cc:492] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:23.523959  4677 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:23.524649  4797 ts_tablet_manager.cc:1403] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.085s	user 0.012s	sys 0.007s
I20260709 12:24:23.525691  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:23.526640  4680 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:23.521346  4796 consensus_queue.cc:260] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.529099  4569 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:23.539898  4569 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:23.541960  4837 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:23.542490  4837 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.542984  4837 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.537175  5093 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:23.549705  5093 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.550254  4797 raft_consensus.cc:359] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.553758  5098 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.551138  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:23.554886  4756 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.554184  4797 raft_consensus.cc:385] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.556119  4797 raft_consensus.cc:740] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.560678  4797 consensus_queue.cc:260] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.562923  4797 ts_tablet_manager.cc:1434] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 12:24:23.562830  5024 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.564170  4797 tablet_bootstrap.cc:492] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:23.569445  5093 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.572201  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:23.580681  5023 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.581521  5023 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.583066  4499 catalog_manager.cc:5697] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:23.584271  5023 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:23.587143  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:23.588850  4720 leader_election.cc:343] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:23.590312  4756 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:23.591580  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:23.592520  4755 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:23.595791  4571 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:23.597963  5094 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:24:23.599023  4569 leader_election.cc:343] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:23.598922  4796 ts_tablet_manager.cc:1434] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.111s	user 0.010s	sys 0.000s
I20260709 12:24:23.598706  4837 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.600883  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:23.601756  5094 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:23.602800  5094 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.604229  4796 tablet_bootstrap.cc:492] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
W20260709 12:24:23.604529  4720 leader_election.cc:343] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:23.613359  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:23.613811  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:23.609764  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:23.613981  4755 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.614686  4799 tablet_bootstrap.cc:492] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:23.616473  4679 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.618916  4799 ts_tablet_manager.cc:1403] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.165s	user 0.012s	sys 0.019s
I20260709 12:24:23.606638  4837 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.615240  5023 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:23.623466  4799 raft_consensus.cc:359] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.624212  4799 raft_consensus.cc:385] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.624512  4799 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.625168  4799 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.628549  4679 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:23.629262  5093 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.629881  4569 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:23.630609  4837 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:23.630998  4755 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:23.632236  4796 tablet_bootstrap.cc:654] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.631186  4837 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:23.633003  4837 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.639114  4797 tablet_bootstrap.cc:654] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.640923  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:24:23.643460  4571 leader_election.cc:343] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:23.643469  4872 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:24:23.641718  4799 ts_tablet_manager.cc:1434] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:24:23.651189  4799 tablet_bootstrap.cc:492] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:23.652942  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:23.653625  4679 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.654492  4797 tablet_bootstrap.cc:492] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:23.655010  4797 ts_tablet_manager.cc:1403] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.091s	user 0.013s	sys 0.004s
I20260709 12:24:23.658017  4799 tablet_bootstrap.cc:654] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.658605  4797 raft_consensus.cc:359] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.659427  4797 raft_consensus.cc:385] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.659780  4797 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.660912  4679 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:23.660785  4797 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:23.645823  4645 consensus_peers.cc:597] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:23.644778  5093 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.663343  4569 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:23.664515  4845 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:23.665014  4845 raft_consensus.cc:697] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:23.665855  4845 consensus_queue.cc:237] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.671168  4797 ts_tablet_manager.cc:1434] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.002s
I20260709 12:24:23.673081  4797 tablet_bootstrap.cc:492] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:23.673899  4799 tablet_bootstrap.cc:492] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:23.674714  5026 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:23.678570  4799 ts_tablet_manager.cc:1403] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.000s
I20260709 12:24:23.681504  4799 raft_consensus.cc:359] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.682217  4799 raft_consensus.cc:385] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.681998  4872 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:23.682505  4799 raft_consensus.cc:740] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.683143  4799 consensus_queue.cc:260] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.684641  4799 ts_tablet_manager.cc:1434] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:24:23.697561  4799 tablet_bootstrap.cc:492] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:23.704456  4796 tablet_bootstrap.cc:492] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:23.705034  4796 ts_tablet_manager.cc:1403] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.101s	user 0.009s	sys 0.008s
I20260709 12:24:23.696476  5023 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.708303  4501 catalog_manager.cc:5697] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:23.711409  5023 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.730639  5023 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:23.743997  4797 tablet_bootstrap.cc:654] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.760272  4799 tablet_bootstrap.cc:654] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.773860  4796 raft_consensus.cc:359] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.780117  4796 raft_consensus.cc:385] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.780519  4796 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.774101  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:23.775969  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:23.781467  4796 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.784413  4796 ts_tablet_manager.cc:1434] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.079s	user 0.009s	sys 0.000s
I20260709 12:24:23.785444  4796 tablet_bootstrap.cc:492] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:23.795964  4797 tablet_bootstrap.cc:492] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:23.796695  4797 ts_tablet_manager.cc:1403] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.124s	user 0.008s	sys 0.004s
W20260709 12:24:23.798040  4720 leader_election.cc:343] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:23.803172  4720 leader_election.cc:343] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:23.803606  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:23.804243  5140 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:23.808156  4797 raft_consensus.cc:359] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.808923  4797 raft_consensus.cc:385] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.811810  4797 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.812645  4797 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.813757  4796 tablet_bootstrap.cc:654] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.814764  4797 ts_tablet_manager.cc:1434] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.006s
I20260709 12:24:23.815804  4797 tablet_bootstrap.cc:492] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:23.819060  4799 tablet_bootstrap.cc:492] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:23.818535  4500 catalog_manager.cc:5697] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:23.819720  4799 ts_tablet_manager.cc:1403] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.128s	user 0.031s	sys 0.010s
I20260709 12:24:23.820317  4500 catalog_manager.cc:5697] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:23.821870  4797 tablet_bootstrap.cc:654] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.823061  4799 raft_consensus.cc:359] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.835117  4799 raft_consensus.cc:385] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.835529  4799 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.836386  4799 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.839140  4799 ts_tablet_manager.cc:1434] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:24:23.840444  4799 tablet_bootstrap.cc:492] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:23.848635  4796 tablet_bootstrap.cc:492] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:23.849152  4796 ts_tablet_manager.cc:1403] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.064s	user 0.014s	sys 0.017s
I20260709 12:24:23.852308  4796 raft_consensus.cc:359] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.852967  4796 raft_consensus.cc:385] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.853209  4796 raft_consensus.cc:740] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.853888  4796 consensus_queue.cc:260] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.855901  4796 ts_tablet_manager.cc:1434] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:24:23.862984  4796 tablet_bootstrap.cc:492] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:23.869164  5140 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.869988  5140 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.877547  5140 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:23.877413  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:23.878125  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:23.879590  4720 leader_election.cc:343] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:23.880239  4720 leader_election.cc:343] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:23.880620  4799 tablet_bootstrap.cc:654] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.880646  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:23.882678  5140 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:23.884760  4797 tablet_bootstrap.cc:492] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:23.885247  4797 ts_tablet_manager.cc:1403] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.070s	user 0.026s	sys 0.022s
I20260709 12:24:23.887785  4797 raft_consensus.cc:359] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.888468  4797 raft_consensus.cc:385] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.888732  4797 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.889202  4796 tablet_bootstrap.cc:654] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.889348  4797 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.889253  5026 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.895798  4797 ts_tablet_manager.cc:1434] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:24:23.895967  5026 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.896760  4797 tablet_bootstrap.cc:492] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:23.900312  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:23.901283  4756 raft_consensus.cc:2393] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 1: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:23.904413  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:23.905308  4797 tablet_bootstrap.cc:654] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.911747  5026 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.915536  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:23.916886  5026 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.919688  4799 tablet_bootstrap.cc:492] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:23.920184  4799 ts_tablet_manager.cc:1403] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.080s	user 0.018s	sys 0.002s
I20260709 12:24:23.922675  4796 tablet_bootstrap.cc:492] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:23.923179  4796 ts_tablet_manager.cc:1403] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.011s
I20260709 12:24:23.924649  5026 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:23.926086  4796 raft_consensus.cc:359] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.926810  4796 raft_consensus.cc:385] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.927121  4796 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.927776  4796 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.928741  4797 tablet_bootstrap.cc:492] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:23.929265  4797 ts_tablet_manager.cc:1403] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.017s
I20260709 12:24:23.929963  4799 raft_consensus.cc:359] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.930653  4799 raft_consensus.cc:385] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.930722  4976 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.930986  4799 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.931099  4796 ts_tablet_manager.cc:1434] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:24:23.931226  4976 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.932628  4796 tablet_bootstrap.cc:492] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:23.932446  4797 raft_consensus.cc:359] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.933225  4797 raft_consensus.cc:385] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.933612  4797 raft_consensus.cc:740] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.934302  4797 consensus_queue.cc:260] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.936690  4797 ts_tablet_manager.cc:1434] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:24:23.937677  4797 tablet_bootstrap.cc:492] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:23.937201  4799 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.938740  4796 tablet_bootstrap.cc:654] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.939415  4799 ts_tablet_manager.cc:1434] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:24:23.940315  4799 tablet_bootstrap.cc:492] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:23.933246  4976 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.947619  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:23.956290  4799 tablet_bootstrap.cc:654] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.956482  4797 tablet_bootstrap.cc:654] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:23.956440  4756 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:23.948791  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:23.949821  4796 tablet_bootstrap.cc:492] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:23.958717  4796 ts_tablet_manager.cc:1403] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.003s
I20260709 12:24:23.949608  4977 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:24:23.961241  5127 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.959877  4647 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:23.962062  5026 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:24:23.961750  5127 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.968113  4796 raft_consensus.cc:359] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.969190  4796 raft_consensus.cc:385] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:23.970031  5127 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:23.970795  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:23.971473  4679 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:23.971863  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:23.967198  4977 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:23.972482  4754 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:23.958078  4605 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:23.972765  4569 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:23.969508  4796 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:23.973822  5127 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:23.974217  4796 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:23.976187  4796 ts_tablet_manager.cc:1434] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.017s	user 0.002s	sys 0.002s
I20260709 12:24:23.977097  4796 tablet_bootstrap.cc:492] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:23.974238  5127 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.980501  5127 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:23.972793  4977 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.982087  4977 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.006273  5026 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.006891  5026 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.021173  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:24.021204  5045 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.022079  4754 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:24.022114  5045 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.011876  5026 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.032600  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:24.036175  4601 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:24.036252  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:24.034294  5045 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:24.037122  4679 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 1: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:24.037307  4977 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.038769  5127 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.039063  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:24.040745  4977 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.041872  4796 tablet_bootstrap.cc:654] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:24.042064  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:24.042672  4679 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.040863  5127 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.045020  4645 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:24.047693  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:24.048337  4756 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.049799  5026 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:24.050323  5026 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.050753  5026 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.057108  4796 tablet_bootstrap.cc:492] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:24.057631  4796 ts_tablet_manager.cc:1403] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.081s	user 0.008s	sys 0.005s
I20260709 12:24:24.061882  4756 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:24.062928  5094 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.063411  5094 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.064301  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:24.065234  5140 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.065851  5094 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.066776  4796 raft_consensus.cc:359] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.067663  4796 raft_consensus.cc:385] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.067984  4796 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.068608  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:24.070369  4569 leader_election.cc:343] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:24.072952  5140 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:24.068823  4796 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.074721  4679 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:24.076099  4569 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:24.076982  5127 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:24.077201  4796 ts_tablet_manager.cc:1434] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:24:24.077454  5127 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:24.078136  4796 tablet_bootstrap.cc:492] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:24.078295  5127 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.088833  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:24.089462  4756 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:24.090869  4796 tablet_bootstrap.cc:654] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:24.090670  4571 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:24.091969  5024 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:24.092343  5024 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.092718  5024 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.096604  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:24.097249  4755 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.097347  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:24.098031  4604 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.100682  5026 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.102809  5026 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.104297  4604 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:24.103780  5024 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.107103  4797 tablet_bootstrap.cc:492] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:24.107667  4797 ts_tablet_manager.cc:1403] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.170s	user 0.025s	sys 0.031s
I20260709 12:24:24.108378  4755 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:24.110854  4797 raft_consensus.cc:359] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.111231  4837 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:24.116637  4645 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:24.113598  4799 tablet_bootstrap.cc:492] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:24.119903  4799 ts_tablet_manager.cc:1403] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.180s	user 0.028s	sys 0.031s
I20260709 12:24:24.120092  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:24.120733  4754 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.120973  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:24.123180  4797 raft_consensus.cc:385] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:24:24.123070  4569 leader_election.cc:343] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:24.123620  4797 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.119323  4872 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:24.124284  4872 raft_consensus.cc:697] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:24.125029  4872 consensus_queue.cc:237] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.128834  5045 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.129474  5045 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.123090  4799 raft_consensus.cc:359] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.131436  4799 raft_consensus.cc:385] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.131557  5024 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.132426  4797 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.133106  5140 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.135671  5140 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.137048  4754 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:24.134368  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:24.137892  5140 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:24.139149  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:24.139633  4720 leader_election.cc:343] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:24.139825  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:24.140300  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:24.131702  5045 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:24.140466  4571 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:24.141598  4837 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:24.142537  4837 raft_consensus.cc:697] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:24.131748  4799 raft_consensus.cc:740] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.143327  4837 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.143597  4799 consensus_queue.cc:260] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.144181  4797 ts_tablet_manager.cc:1434] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.001s
I20260709 12:24:24.145112  4797 tablet_bootstrap.cc:492] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
W20260709 12:24:24.147140  4720 leader_election.cc:343] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:24.147600  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:24:24.148885  4720 leader_election.cc:343] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:24.149760  5045 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:24.150338  4799 ts_tablet_manager.cc:1434] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.001s
I20260709 12:24:24.155157  4796 tablet_bootstrap.cc:492] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:24.155203  4799 tablet_bootstrap.cc:492] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:24.155716  4796 ts_tablet_manager.cc:1403] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.078s	user 0.009s	sys 0.004s
W20260709 12:24:24.149544  4720 leader_election.cc:343] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:24.163347  5024 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:24.165737  4796 raft_consensus.cc:359] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.166445  4796 raft_consensus.cc:385] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.165988  4500 catalog_manager.cc:5697] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:24.163825  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:24.168643  5045 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:24.166754  4796 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.169763  4796 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.171561  4797 tablet_bootstrap.cc:654] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:24.171905  4796 ts_tablet_manager.cc:1434] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:24:24.172873  4796 tablet_bootstrap.cc:492] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:24.174283  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:24.175945  4756 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.183758  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:24.184422  4602 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.192973  4602 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:24.194262  4645 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:24.195333  4872 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:24.196271  4872 raft_consensus.cc:697] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:24.197103  4872 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.205003  4756 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:24.206254  4837 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.206746  4837 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.207383  4799 tablet_bootstrap.cc:654] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:24.208528  4837 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.210228  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:24.220247  4604 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.221096  4797 tablet_bootstrap.cc:492] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:24.221657  4797 ts_tablet_manager.cc:1403] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.077s	user 0.008s	sys 0.013s
I20260709 12:24:24.224284  4797 raft_consensus.cc:359] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.224942  4797 raft_consensus.cc:385] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.225217  4797 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.225862  4797 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.228621  4797 ts_tablet_manager.cc:1434] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:24:24.229537  4797 tablet_bootstrap.cc:492] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:24.233520  4837 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:24.235410  4797 tablet_bootstrap.cc:654] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:24.237601  4498 catalog_manager.cc:5697] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:24.247291  4796 tablet_bootstrap.cc:654] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:24.249351  5127 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:24.250166  4797 tablet_bootstrap.cc:492] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:24.257203  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:24.267712  4500 catalog_manager.cc:5697] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:24.270836  4571 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:24.271922  4837 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:24.250638  4797 ts_tablet_manager.cc:1403] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.006s
I20260709 12:24:24.273300  4500 catalog_manager.cc:5697] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:24.278007  5127 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:24.279145  5094 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:24.294010  4797 raft_consensus.cc:359] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.294643  4797 raft_consensus.cc:385] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.294818  4797 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.295374  4797 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.298485  4796 tablet_bootstrap.cc:492] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:24.298982  4796 ts_tablet_manager.cc:1403] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.126s	user 0.015s	sys 0.004s
I20260709 12:24:24.301671  4796 raft_consensus.cc:359] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.302347  4796 raft_consensus.cc:385] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.302620  4796 raft_consensus.cc:740] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.306017  4796 consensus_queue.cc:260] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.308877  4796 ts_tablet_manager.cc:1434] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:24:24.309916  4796 tablet_bootstrap.cc:492] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:24.310285  4797 ts_tablet_manager.cc:1434] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.001s
I20260709 12:24:24.318113  4797 tablet_bootstrap.cc:492] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:24.319555  4796 tablet_bootstrap.cc:654] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:24.328884  5140 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.329391  5140 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.331094  4799 tablet_bootstrap.cc:492] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:24.331575  4799 ts_tablet_manager.cc:1403] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.177s	user 0.038s	sys 0.041s
I20260709 12:24:24.331470  5140 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:24.334108  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:24.336539  4797 tablet_bootstrap.cc:654] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:24.338709  4720 leader_election.cc:343] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:24.339006  4799 raft_consensus.cc:359] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.339514  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:24.339704  4799 raft_consensus.cc:385] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.341665  4799 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
W20260709 12:24:24.341696  4720 leader_election.cc:343] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:24.342720  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:24.342471  4799 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.344085  5158 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:24.346154  4799 ts_tablet_manager.cc:1434] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 12:24:24.347821  4799 tablet_bootstrap.cc:492] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:24.353629  4799 tablet_bootstrap.cc:654] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:24.363265  4797 tablet_bootstrap.cc:492] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:24.372771  4797 ts_tablet_manager.cc:1403] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.055s	user 0.016s	sys 0.013s
I20260709 12:24:24.379473  4797 raft_consensus.cc:359] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.380262  4797 raft_consensus.cc:385] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.380553  4797 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.381399  4797 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.383478  4799 tablet_bootstrap.cc:492] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:24.384012  4799 ts_tablet_manager.cc:1403] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.036s	user 0.020s	sys 0.007s
I20260709 12:24:24.386857  4799 raft_consensus.cc:359] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.387365  4797 ts_tablet_manager.cc:1434] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:24:24.387620  4799 raft_consensus.cc:385] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.387876  4799 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.388307  4797 tablet_bootstrap.cc:492] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:24.388512  4799 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.394145  4796 tablet_bootstrap.cc:492] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:24.394640  4796 ts_tablet_manager.cc:1403] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.085s	user 0.013s	sys 0.003s
I20260709 12:24:24.395442  4797 tablet_bootstrap.cc:654] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:24.395521  5140 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.396049  5140 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.397434  4796 raft_consensus.cc:359] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.398172  4796 raft_consensus.cc:385] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.398233  5140 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:24.398486  4796 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.399619  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:24.399452  4796 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:24.402153  4720 leader_election.cc:343] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:24.403069  4799 ts_tablet_manager.cc:1434] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:24:24.403793  5026 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.404333  4799 tablet_bootstrap.cc:492] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:24.404281  5026 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.406498  5026 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.408593  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:24.403299  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:24.412079  4799 tablet_bootstrap.cc:654] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:24.414027  4796 ts_tablet_manager.cc:1434] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:24:24.422761  5158 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.423348  5158 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.434810  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:24.435635  4755 raft_consensus.cc:2393] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 1: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:24.437918  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:24.439086  5026 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.444078  5158 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:24:24.445530  4720 leader_election.cc:343] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:24.446036  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:24.446120  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:24.446866  4604 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 0.
I20260709 12:24:24.447033  5158 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:24.448025  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:24.449637  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:24.452109  5158 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:24.452548  5158 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.452919  5158 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:24.456271  4720 leader_election.cc:343] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:24.456799  4799 tablet_bootstrap.cc:492] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:24.457160  4799 ts_tablet_manager.cc:1403] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.053s	user 0.032s	sys 0.012s
I20260709 12:24:24.458017  4796 tablet_bootstrap.cc:492] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Bootstrap starting.
I20260709 12:24:24.459738  4799 raft_consensus.cc:359] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.462284  4799 raft_consensus.cc:385] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.463189  4799 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.463886  4799 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.465939  4799 ts_tablet_manager.cc:1434] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:24:24.467247  4799 tablet_bootstrap.cc:492] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:24.463742  5158 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.469512  5158 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:24.470147  5024 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.470660  5024 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.470939  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:24.471570  4605 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.472708  5024 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.475427  4799 tablet_bootstrap.cc:654] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:24.475960  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:24.476636  4678 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:24.477836  4569 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:24.479003  4837 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:24.479346  4837 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.479662  4837 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.481348  4605 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:24.481818  5094 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.482376  5094 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.470508  4796 tablet_bootstrap.cc:654] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:24.483791  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:24.484390  4755 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:24.484493  5094 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.486646  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:24.487442  4679 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:24.487931  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:24.488552  4755 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:24.488848  4569 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:24.489682  5024 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:24.489948  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:24.493101  5140 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:24.493655  5140 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:24.494009  5094 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:24.494470  5140 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.490051  5024 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.497364  5024 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.498991  5026 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:24.505007  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:24.508957  4837 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.513299  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:24.514103  4755 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.514142  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:24.515041  4678 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:24.522469  4720 leader_election.cc:343] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:24.533612  4837 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.537308  5031 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.505523  5024 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.538182  5031 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.541546  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:24.542299  4756 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.537766  4755 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:24.547662  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:24.554214  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:24.554766  4677 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.558562  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:24.559464  4605 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:24.568393  4756 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:24.570111  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:24.577443  4837 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:24.578133  4837 raft_consensus.cc:697] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:24.579198  4837 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.548029  5031 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.561101  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:24.582863  4872 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:24.583271  4872 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.583586  4872 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.584072  4797 tablet_bootstrap.cc:492] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:24.584736  4797 ts_tablet_manager.cc:1403] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.197s	user 0.021s	sys 0.021s
I20260709 12:24:24.555154  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:24.559193  5127 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:24.560037  5024 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.595603  4754 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:24.598383  5127 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:24.617650  4872 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.615576  4869 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:24:24.623199  4797 raft_consensus.cc:359] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.623911  4797 raft_consensus.cc:385] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.624179  4797 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.624809  4797 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.632619  5127 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.633560  4799 tablet_bootstrap.cc:492] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:24.634078  4799 ts_tablet_manager.cc:1403] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.167s	user 0.044s	sys 0.069s
I20260709 12:24:24.642798  4677 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
W20260709 12:24:24.646879  4550 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:24:24.649417  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 79475.8us
I20260709 12:24:24.647599  4501 catalog_manager.cc:5697] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:24.684195  4797 ts_tablet_manager.cc:1434] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.099s	user 0.004s	sys 0.003s
I20260709 12:24:24.686493  4797 tablet_bootstrap.cc:492] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:24.695005  4799 raft_consensus.cc:359] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.696192  4799 raft_consensus.cc:385] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.696527  4799 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.697391  4799 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.728653  4796 tablet_bootstrap.cc:492] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: No bootstrap required, opened a new log
I20260709 12:24:24.729264  4796 ts_tablet_manager.cc:1403] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Time spent bootstrapping tablet: real 0.271s	user 0.019s	sys 0.017s
I20260709 12:24:24.742360  4799 ts_tablet_manager.cc:1434] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.108s	user 0.010s	sys 0.003s
I20260709 12:24:24.743765  4799 tablet_bootstrap.cc:492] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:24.728070  5158 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.744545  5158 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.746795  5158 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:24.674027  5031 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.060s
W20260709 12:24:24.760162  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 122588us
I20260709 12:24:24.765930  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:24.766685  4605 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:24.770210  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 119168us
I20260709 12:24:24.772962  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:24.773592  4755 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:24.774708  4797 tablet_bootstrap.cc:654] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:24.780303  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:24.781841  4872 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:24.784178  4755 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
W20260709 12:24:24.788120  4720 leader_election.cc:343] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:24.802830  4605 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:24.804970  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:24.824474  4977 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:24.823782  5045 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.805174  5188 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.049s
I20260709 12:24:24.881858  4977 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:24.894306  4678 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:24.881166  5045 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.918011  4498 catalog_manager.cc:5697] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:24.928107  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 90434.9us
I20260709 12:24:24.948140  4796 raft_consensus.cc:359] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.949013  4796 raft_consensus.cc:385] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:24.949270  4796 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Initialized, Role: FOLLOWER
I20260709 12:24:24.946367  5045 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:24.957841  4796 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.964973  4796 ts_tablet_manager.cc:1434] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Time spent starting tablet: real 0.235s	user 0.006s	sys 0.002s
I20260709 12:24:24.940503  4977 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.971540  5184 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:24:24.973542  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:24.987135  4837 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:24.989773  5024 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.010892  5024 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:24.991931  5203 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.054s
I20260709 12:24:25.040527  5024 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:25.062813  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:25.063601  4756 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:25.064471  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:25.068437  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:25.090596  4720 leader_election.cc:343] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:25.090901  4799 tablet_bootstrap.cc:654] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:25.091096  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:25.097321  5140 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:25.062294  4977 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.098716  4977 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:25.145906  4571 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:24:25.148278  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57705.3us
I20260709 12:24:25.154201  5200 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:25.154678  5200 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.154995  5200 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:25.108372  4676 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:25.172030  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:25.334870  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 182411us
I20260709 12:24:25.338923  4797 tablet_bootstrap.cc:492] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:25.339428  4797 ts_tablet_manager.cc:1403] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.654s	user 0.017s	sys 0.017s
W20260709 12:24:25.342429  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 115826us
W20260709 12:24:25.344614  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 192290us
W20260709 12:24:25.345407  4720 leader_election.cc:343] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:25.272889  5180 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.241s
W20260709 12:24:25.351790  4720 leader_election.cc:343] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:25.352365  4720 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:25.379230  5211 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:25.380515  5140 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:25.384745  5178 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.014s
I20260709 12:24:25.398602  5140 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:25.415711  5208 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:24:25.417124  5200 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:25.462471  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 62075.1us
I20260709 12:24:25.474210  4797 raft_consensus.cc:359] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:25.475932  4797 raft_consensus.cc:385] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:25.476356  4797 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:25.477732  4797 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:25.482647  4799 tablet_bootstrap.cc:492] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:25.491719  4799 ts_tablet_manager.cc:1403] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.748s	user 0.024s	sys 0.017s
I20260709 12:24:25.412801  5199 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.064s
I20260709 12:24:25.507926  4498 catalog_manager.cc:5697] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:25.523425  4797 ts_tablet_manager.cc:1434] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.184s	user 0.013s	sys 0.000s
I20260709 12:24:25.497102  4799 raft_consensus.cc:359] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:25.528009  4799 raft_consensus.cc:385] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:25.528429  4799 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:25.534441  4799 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:25.534323  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 43687.9us
I20260709 12:24:25.541873  5140 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:25.542686  5200 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:25.555480  4797 tablet_bootstrap.cc:492] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:25.525002  5216 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
I20260709 12:24:25.567005  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:25.574047  4799 ts_tablet_manager.cc:1434] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.082s	user 0.010s	sys 0.001s
I20260709 12:24:25.580134  4799 tablet_bootstrap.cc:492] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
W20260709 12:24:25.603147  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 462249us
I20260709 12:24:25.621441  4797 tablet_bootstrap.cc:654] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:25.634940  4720 leader_election.cc:343] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:25.636165  4977 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:25.645509  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:25.646867  4598 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 1: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:25.658020  5189 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.658905  5189 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:25.664362  5189 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:25.664700  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:25.665398  4679 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:25.692911  4799 tablet_bootstrap.cc:654] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:25.701505  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 53067.3us
W20260709 12:24:25.821954  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 69639.7us
I20260709 12:24:25.769989  4500 catalog_manager.cc:5697] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:25.825258  4679 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:25.826479  5186 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:25.831501  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:25.832559  4755 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:25.830202  5192 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:25.840767  5192 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:25.843318  5192 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:25.849047  5192 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.850329  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:25.850198  5192 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:25.852463  5192 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:25.849627  4755 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:25.854277  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:25.857409  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:25.862057  5094 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:25.862669  5094 raft_consensus.cc:697] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:25.868722  5094 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:25.871682  4755 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:25.885002  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:25.886029  4678 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:25.888706  4797 tablet_bootstrap.cc:492] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:25.889218  4797 ts_tablet_manager.cc:1403] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.334s	user 0.015s	sys 0.016s
I20260709 12:24:25.892241  5180 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:25.905227  4720 consensus_peers.cc:597] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.914863  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:25.921452  4720 leader_election.cc:343] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:25.922205  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:25.926353  5193 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:25.928028  4569 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:25.924858  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:25.935973  4599 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:25.939055  4797 raft_consensus.cc:359] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:25.940104  4797 raft_consensus.cc:385] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:25.940454  4797 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:25.941331  4797 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:25.954134  5228 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:25.954586  5228 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.954926  5228 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:25.965037  4797 ts_tablet_manager.cc:1434] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.075s	user 0.010s	sys 0.001s
I20260709 12:24:25.976107  4977 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:25.979637  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:25.980396  4751 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:25.984606  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:25.985420  4755 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 1: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:25.989123  5228 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:25.991436  5228 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:25.996275  4797 tablet_bootstrap.cc:492] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:25.999924  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:26.015883  4837 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:26.021461  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:26.022281  4748 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:26.024809  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:26.032816  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:26.033588  4681 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:26.034536  4799 tablet_bootstrap.cc:492] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:26.035187  4799 ts_tablet_manager.cc:1403] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.456s	user 0.035s	sys 0.017s
I20260709 12:24:26.040334  4797 tablet_bootstrap.cc:654] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:26.041690  4681 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:26.042245  4977 raft_consensus.cc:2764] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:24:26.042451  5185 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:26.129029  4647 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:26.150224  4977 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:26.268642  5094 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.273206  5094 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:26.366770  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 306208us
I20260709 12:24:26.371296  5094 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:26.371963  4799 raft_consensus.cc:359] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:26.372920  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 223354us
I20260709 12:24:26.373936  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:26.375546  4869 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:26.372946  4799 raft_consensus.cc:385] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:26.379169  4799 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:26.382632  4799 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:26.392714  4500 catalog_manager.cc:5697] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:26.404104  4550 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:26.382091  4748 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:26.420750  4799 ts_tablet_manager.cc:1434] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.385s	user 0.015s	sys 0.005s
W20260709 12:24:26.450573  5251 log.cc:927] Time spent T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:24:26.451849  4799 tablet_bootstrap.cc:492] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:26.463522  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:26.464226  4755 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:26.485065  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:26.487458  5225 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:26.487962  5225 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:26.488804  5225 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:26.504005  4571 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:26.511406  4869 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.511966  4869 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:26.514931  5180 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:26.530011  4869 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:26.534832  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:26.535585  4675 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:26.541118  4799 tablet_bootstrap.cc:654] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:26.529279  5180 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.549285  5180 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:26.560726  4797 tablet_bootstrap.cc:492] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:26.561455  4797 ts_tablet_manager.cc:1403] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.565s	user 0.007s	sys 0.014s
I20260709 12:24:26.568192  5195 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:26.563469  5198 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:26.619359  5262 log.cc:927] Time spent T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:24:26.621794  4797 raft_consensus.cc:359] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:26.622808  4797 raft_consensus.cc:385] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:26.623137  4797 raft_consensus.cc:740] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:26.589148  5198 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:26.634945  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:26.629923  4797 consensus_queue.cc:260] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:26.635047  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:26.659263  4753 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:26.644574  5225 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
W20260709 12:24:26.692775  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 101698us
I20260709 12:24:26.745388  4799 tablet_bootstrap.cc:492] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:26.746155  4799 ts_tablet_manager.cc:1403] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.295s	user 0.015s	sys 0.033s
W20260709 12:24:26.754282  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 58786.4us
W20260709 12:24:26.755021  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 58267.1us
I20260709 12:24:26.756673  4647 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:26.769924  4797 ts_tablet_manager.cc:1434] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.208s	user 0.004s	sys 0.008s
I20260709 12:24:26.764348  4799 raft_consensus.cc:359] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:26.776940  4799 raft_consensus.cc:385] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:26.777758  4799 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:26.783886  4799 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:26.751179  5198 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:26.803551  5188 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:26.804049  5188 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.804395  5188 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:26.808902  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:26.769418  5180 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:26.811954  4720 leader_election.cc:343] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:26.820710  4596 raft_consensus.cc:2393] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 1: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:26.821743  5180 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:26.823220  4799 ts_tablet_manager.cc:1434] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.002s
I20260709 12:24:26.824357  4799 tablet_bootstrap.cc:492] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:26.824388  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:26.826650  4752 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:26.833389  4799 tablet_bootstrap.cc:654] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:26.818931  4672 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:26.845160  5188 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:26.848966  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:26.850658  5188 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:26.851480  4720 leader_election.cc:343] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:26.851912  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:26.861430  5270 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:26.862970  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:26.863984  4674 raft_consensus.cc:2393] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 1: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:26.855993  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:26.865401  4602 raft_consensus.cc:2393] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 1: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:26.871829  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:26.886508  4647 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:26.888006  5026 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:26.901968  4797 tablet_bootstrap.cc:492] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:26.905237  4752 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:26.937028  4571 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:26.941035  5186 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:26.957150  5186 raft_consensus.cc:697] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:26.958161  5186 consensus_queue.cc:237] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:26.976081  4498 catalog_manager.cc:5697] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:26.985592  4797 tablet_bootstrap.cc:654] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:26.990608  5127 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:27.040671  4799 tablet_bootstrap.cc:492] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:27.041221  4799 ts_tablet_manager.cc:1403] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.217s	user 0.009s	sys 0.014s
I20260709 12:24:27.058070  4799 raft_consensus.cc:359] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.058797  4799 raft_consensus.cc:385] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.059094  4799 raft_consensus.cc:740] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.059739  5226 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.060220  5226 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.062319  5226 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:27.066614  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:27.064510  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:27.068753  4747 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:27.070122  4571 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:27.071023  5195 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:27.071412  5195 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.071779  5195 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:27.076512  4799 consensus_queue.cc:260] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.077648  4679 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:27.078964  4799 ts_tablet_manager.cc:1434] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 12:24:27.087951  4498 catalog_manager.cc:5697] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:27.100040  4799 tablet_bootstrap.cc:492] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:27.101177  4797 tablet_bootstrap.cc:492] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:27.101663  4797 ts_tablet_manager.cc:1403] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.200s	user 0.010s	sys 0.006s
I20260709 12:24:27.104866  4797 raft_consensus.cc:359] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.109519  5186 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:27.105517  4797 raft_consensus.cc:385] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.114714  4797 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.118005  4797 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.122637  5195 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.125417  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:27.126030  4675 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:27.132993  4675 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:27.134251  4797 ts_tablet_manager.cc:1434] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:24:27.135356  4797 tablet_bootstrap.cc:492] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Bootstrap starting.
I20260709 12:24:27.138074  4569 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:27.139024  5186 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:27.139469  5186 raft_consensus.cc:697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:27.140250  5186 consensus_queue.cc:237] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.148415  4799 tablet_bootstrap.cc:654] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:27.168869  5195 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:27.187364  4797 tablet_bootstrap.cc:654] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:27.199648  4501 catalog_manager.cc:5697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:27.202441  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:27.203159  4752 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:27.203948  4797 tablet_bootstrap.cc:492] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: No bootstrap required, opened a new log
I20260709 12:24:27.204434  4797 ts_tablet_manager.cc:1403] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Time spent bootstrapping tablet: real 0.069s	user 0.009s	sys 0.000s
I20260709 12:24:27.208464  4797 raft_consensus.cc:359] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.209126  4797 raft_consensus.cc:385] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.209380  4797 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.211545  4752 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:27.212846  4797 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.214602  4797 ts_tablet_manager.cc:1434] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:24:27.233862  4799 tablet_bootstrap.cc:492] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:27.233814  5186 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.234606  4799 ts_tablet_manager.cc:1403] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.140s	user 0.013s	sys 0.002s
I20260709 12:24:27.234550  5186 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.237524  5186 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:27.237738  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:27.237540  4799 raft_consensus.cc:359] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.238504  4799 raft_consensus.cc:385] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.238757  4799 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.239287  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:27.239440  4799 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.240312  4672 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:27.241650  4799 ts_tablet_manager.cc:1434] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:24:27.242254  4569 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:27.242964  5186 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:27.243299  4799 tablet_bootstrap.cc:492] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Bootstrap starting.
I20260709 12:24:27.243304  5186 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.244570  5186 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:27.239977  4752 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 0.
I20260709 12:24:27.253067  5186 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.256846  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:27.257445  4672 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:27.266907  4799 tablet_bootstrap.cc:654] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:27.267966  4672 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:27.269171  4569 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:27.270062  5195 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:27.270527  5195 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:27.271130  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:27.271723  4752 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:27.271358  5195 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.278136  5186 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 1 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:27.282934  4799 tablet_bootstrap.cc:492] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: No bootstrap required, opened a new log
I20260709 12:24:27.284479  4799 ts_tablet_manager.cc:1403] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.004s
I20260709 12:24:27.287236  4799 raft_consensus.cc:359] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.287905  4799 raft_consensus.cc:385] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.288172  4799 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.288743  4799 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.290230  4799 ts_tablet_manager.cc:1434] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:24:27.283744  4752 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:27.290112  4501 catalog_manager.cc:5697] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:27.366231  5195 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:27.370455  5255 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.379740  5127 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:27.370993  5255 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.384747  5255 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:27.386328  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:27.387019  4602 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:27.388039  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:27.388703  4749 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 0.
I20260709 12:24:27.395143  4645 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:27.397220  4976 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:27.397632  4976 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.397972  4976 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:27.425958  4976 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.428169  4976 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:27.429113  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:27.429826  4602 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:27.436940  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:27.437649  4751 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:27.437531  4602 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:27.440735  4645 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:27.441459  4976 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:27.442008  4976 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:27.442898  4976 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:27.459095  4751 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:27.467639  4498 catalog_manager.cc:5697] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:27.634171  5127 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:27.644315  4837 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:27.759666  5195 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:27.775699  5186 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:27.954942  5255 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:27.976059  5255 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:28.780100  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:28.811882  5300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:28.814517  5299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:28.821843  5302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:28.826750  3431 server_base.cc:1061] running on GCE node
I20260709 12:24:28.829854  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:28.830427  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:28.830703  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783599868830679 us; error 0 us; skew 500 ppm
I20260709 12:24:28.831501  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:28.834558  3431 webserver.cc:533] Webserver started at http://127.3.89.196:33171/ using document root <none> and password file <none>
I20260709 12:24:28.835196  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:28.835467  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:28.835868  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:28.837435  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-3-root/instance:
uuid: "dbe960c35c84451faf4e5b5c9233fa39"
format_stamp: "Formatted at 2026-07-09 12:24:28 on dist-test-slave-3hrd"
I20260709 12:24:28.842697  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:24:28.846149  5310 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:28.847134  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:24:28.847515  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-3-root
uuid: "dbe960c35c84451faf4e5b5c9233fa39"
format_stamp: "Formatted at 2026-07-09 12:24:28 on dist-test-slave-3hrd"
I20260709 12:24:28.847895  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:28.866051  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:28.867636  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:28.869685  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:28.873711  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:28.874032  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:28.874361  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:28.874572  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:28.935719  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.196:39067
I20260709 12:24:28.935863  5373 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.196:39067 every 8 connection(s)
I20260709 12:24:28.987095  3431 auto_leader_rebalancer-test.cc:439] add a tserver: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:29.011569  5374 heartbeater.cc:344] Connected to a master server at 127.3.89.254:36317
I20260709 12:24:29.012097  5374 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:29.013025  5374 heartbeater.cc:507] Master 127.3.89.254:36317 requested a full tablet report, sending...
I20260709 12:24:29.015470  4501 ts_manager.cc:194] Registered new tserver with Master: dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:29.016739  4501 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60024
I20260709 12:24:29.628775  4672 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:29.636220  4747 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:29.636499  4602 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:29.637848  5188 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:29.640111  5255 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:29.646943  5188 raft_consensus.cc:2955] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:29.649399  4747 raft_consensus.cc:2955] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:29.654551  4602 raft_consensus.cc:2955] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
W20260709 12:24:29.669588  4644 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a8ee74acbd5c45f0934f6f37d2154bc7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:29.669071  4672 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:29.671393  4501 catalog_manager.cc:5697] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
W20260709 12:24:29.687458  4644 consensus_peers.cc:597] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab695b6fa8e145c7926a540c3ba5b84e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:29.687575  4747 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:29.690500  5188 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:29.692080  4602 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:29.693951  5188 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:29.711758  5255 raft_consensus.cc:2955] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:29.714088  4602 raft_consensus.cc:2955] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:29.714578  4747 raft_consensus.cc:2955] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:29.757145  4498 catalog_manager.cc:5697] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:29.787492  4602 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:29.796159  4679 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:29.797822  5195 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:29.798164  4747 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:29.800369  5195 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:24:29.814502  4568 consensus_peers.cc:597] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9ca7c574e14c4faeac378ee00c0a261b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:29.822979  5186 raft_consensus.cc:2955] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:29.826191  4749 raft_consensus.cc:2955] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:29.836670  4672 consensus_queue.cc:237] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:29.857692  4679 raft_consensus.cc:2955] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
W20260709 12:24:29.862576  4644 consensus_peers.cc:597] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c3d4ce22fd7947598cc1d1c7732285da. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:29.870759  4749 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:29.871173  4602 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:29.870931  4500 catalog_manager.cc:5697] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:29.872817  5255 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:29.875114  5255 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:29.887846  5188 raft_consensus.cc:2955] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:29.896548  4752 raft_consensus.cc:2955] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:29.918059  4501 catalog_manager.cc:5697] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:29.919806  4602 raft_consensus.cc:2955] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:30.019359  5374 heartbeater.cc:499] Master 127.3.89.254:36317 was elected leader, sending a full tablet report...
I20260709 12:24:30.142387  5406 ts_tablet_manager.cc:933] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:30.147243  5406 tablet_copy_client.cc:323] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.194:42599
I20260709 12:24:30.181460  4691 tablet_copy_service.cc:140] P 3a5aeedd11fe49938079fe86cb01232c: Received BeginTabletCopySession request for tablet a8ee74acbd5c45f0934f6f37d2154bc7 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:48198)
I20260709 12:24:30.182111  4691 tablet_copy_service.cc:161] P 3a5aeedd11fe49938079fe86cb01232c: Beginning new tablet copy session on tablet a8ee74acbd5c45f0934f6f37d2154bc7 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:48198: session id = dbe960c35c84451faf4e5b5c9233fa39-a8ee74acbd5c45f0934f6f37d2154bc7
I20260709 12:24:30.187939  4691 tablet_copy_source_session.cc:215] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:30.191457  5406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8ee74acbd5c45f0934f6f37d2154bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:30.207644  5410 ts_tablet_manager.cc:933] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:30.211763  5406 tablet_copy_client.cc:806] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:30.212502  5406 tablet_copy_client.cc:670] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:30.219764  5406 tablet_copy_client.cc:538] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:30.234570  5406 tablet_bootstrap.cc:492] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:30.271000  5412 ts_tablet_manager.cc:933] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:24:30.280544  5410 tablet_copy_client.cc:323] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.194:42599
I20260709 12:24:30.282558  4691 tablet_copy_service.cc:140] P 3a5aeedd11fe49938079fe86cb01232c: Received BeginTabletCopySession request for tablet ab695b6fa8e145c7926a540c3ba5b84e from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:48198)
I20260709 12:24:30.283124  4691 tablet_copy_service.cc:161] P 3a5aeedd11fe49938079fe86cb01232c: Beginning new tablet copy session on tablet ab695b6fa8e145c7926a540c3ba5b84e from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:48198: session id = dbe960c35c84451faf4e5b5c9233fa39-ab695b6fa8e145c7926a540c3ba5b84e
I20260709 12:24:30.285439  5406 tablet_bootstrap.cc:492] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:30.286830  5406 tablet_bootstrap.cc:492] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:30.287539  5406 ts_tablet_manager.cc:1403] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.053s	user 0.025s	sys 0.012s
I20260709 12:24:30.288455  5412 tablet_copy_client.cc:323] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.193:45077
I20260709 12:24:30.290972  5406 raft_consensus.cc:359] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:30.291919  5406 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:30.292624  5406 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:30.297358  5417 ts_tablet_manager.cc:933] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:30.298418  4691 tablet_copy_source_session.cc:215] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:30.311293  5410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab695b6fa8e145c7926a540c3ba5b84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:30.318598  5417 tablet_copy_client.cc:323] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.194:42599
I20260709 12:24:30.321463  4691 tablet_copy_service.cc:140] P 3a5aeedd11fe49938079fe86cb01232c: Received BeginTabletCopySession request for tablet c3d4ce22fd7947598cc1d1c7732285da from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:48198)
I20260709 12:24:30.322257  4691 tablet_copy_service.cc:161] P 3a5aeedd11fe49938079fe86cb01232c: Beginning new tablet copy session on tablet c3d4ce22fd7947598cc1d1c7732285da from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:48198: session id = dbe960c35c84451faf4e5b5c9233fa39-c3d4ce22fd7947598cc1d1c7732285da
I20260709 12:24:30.339527  5406 ts_tablet_manager.cc:1434] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.026s
I20260709 12:24:30.342840  4690 tablet_copy_service.cc:342] P 3a5aeedd11fe49938079fe86cb01232c: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-a8ee74acbd5c45f0934f6f37d2154bc7 received from {username='slave'} at 127.0.0.1:48198
I20260709 12:24:30.343412  4690 tablet_copy_service.cc:434] P 3a5aeedd11fe49938079fe86cb01232c: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-a8ee74acbd5c45f0934f6f37d2154bc7 on tablet a8ee74acbd5c45f0934f6f37d2154bc7 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:30.356766  4691 tablet_copy_source_session.cc:215] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:30.362818  5417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d4ce22fd7947598cc1d1c7732285da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:30.366581  5410 tablet_copy_client.cc:806] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:30.367295  5410 tablet_copy_client.cc:670] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:30.374625  5410 tablet_copy_client.cc:538] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:30.388558  5410 tablet_bootstrap.cc:492] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:30.390631  5417 tablet_copy_client.cc:806] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:30.391592  5417 tablet_copy_client.cc:670] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:30.391500  4615 tablet_copy_service.cc:140] P 5c3201048f8346009aefca07428828f3: Received BeginTabletCopySession request for tablet 9ca7c574e14c4faeac378ee00c0a261b from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:53448)
I20260709 12:24:30.392622  4615 tablet_copy_service.cc:161] P 5c3201048f8346009aefca07428828f3: Beginning new tablet copy session on tablet 9ca7c574e14c4faeac378ee00c0a261b from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:53448: session id = dbe960c35c84451faf4e5b5c9233fa39-9ca7c574e14c4faeac378ee00c0a261b
I20260709 12:24:30.404281  5417 tablet_copy_client.cc:538] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:30.418134  5417 tablet_bootstrap.cc:492] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:30.442337  5410 tablet_bootstrap.cc:492] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:30.444960  5410 tablet_bootstrap.cc:492] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:30.445276  4615 tablet_copy_source_session.cc:215] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:30.445837  5410 ts_tablet_manager.cc:1403] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.057s	user 0.033s	sys 0.012s
I20260709 12:24:30.449188  5410 raft_consensus.cc:359] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:30.450161  5410 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:30.450838  5410 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:30.456265  5410 ts_tablet_manager.cc:1434] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:24:30.459283  4691 tablet_copy_service.cc:342] P 3a5aeedd11fe49938079fe86cb01232c: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-ab695b6fa8e145c7926a540c3ba5b84e received from {username='slave'} at 127.0.0.1:48198
I20260709 12:24:30.459789  4691 tablet_copy_service.cc:434] P 3a5aeedd11fe49938079fe86cb01232c: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-ab695b6fa8e145c7926a540c3ba5b84e on tablet ab695b6fa8e145c7926a540c3ba5b84e with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:30.485087  5417 tablet_bootstrap.cc:492] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:30.485983  5417 tablet_bootstrap.cc:492] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:30.486703  5417 ts_tablet_manager.cc:1403] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.016s
I20260709 12:24:30.489464  5417 raft_consensus.cc:359] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:30.490289  5417 raft_consensus.cc:740] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:30.490928  5417 consensus_queue.cc:260] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:30.494531  5412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ca7c574e14c4faeac378ee00c0a261b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:30.500447  5417 ts_tablet_manager.cc:1434] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.005s
I20260709 12:24:30.506903  4691 tablet_copy_service.cc:342] P 3a5aeedd11fe49938079fe86cb01232c: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-c3d4ce22fd7947598cc1d1c7732285da received from {username='slave'} at 127.0.0.1:48198
I20260709 12:24:30.507306  4691 tablet_copy_service.cc:434] P 3a5aeedd11fe49938079fe86cb01232c: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-c3d4ce22fd7947598cc1d1c7732285da on tablet c3d4ce22fd7947598cc1d1c7732285da with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:30.512051  5412 tablet_copy_client.cc:806] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:30.512674  5412 tablet_copy_client.cc:670] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:30.516978  5412 tablet_copy_client.cc:538] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:30.525871  5412 tablet_bootstrap.cc:492] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:30.552425  5412 tablet_bootstrap.cc:492] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:30.553342  5412 tablet_bootstrap.cc:492] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:30.554036  5412 ts_tablet_manager.cc:1403] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.003s
I20260709 12:24:30.556937  5412 raft_consensus.cc:359] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:30.557869  5412 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:30.558526  5412 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:30.560823  5412 ts_tablet_manager.cc:1434] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:24:30.562424  4615 tablet_copy_service.cc:342] P 5c3201048f8346009aefca07428828f3: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-9ca7c574e14c4faeac378ee00c0a261b received from {username='slave'} at 127.0.0.1:53448
I20260709 12:24:30.562808  4615 tablet_copy_service.cc:434] P 5c3201048f8346009aefca07428828f3: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-9ca7c574e14c4faeac378ee00c0a261b on tablet 9ca7c574e14c4faeac378ee00c0a261b with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:30.611357  5349 raft_consensus.cc:1217] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:30.760021  5349 raft_consensus.cc:1217] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:30.776125  5349 raft_consensus.cc:1217] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:30.868908  5349 raft_consensus.cc:1217] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:31.131003  5188 raft_consensus.cc:1064] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:24:31.133659  5188 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:31.140316  5349 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:31.140679  4599 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:31.141500  4752 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:31.141958  5255 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:31.142915  5188 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:31.143922  5420 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:31.164363  5401 raft_consensus.cc:1064] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:24:31.164351  5255 raft_consensus.cc:2955] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.170461  5348 raft_consensus.cc:2955] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.177973  5401 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:31.179322  4749 raft_consensus.cc:2955] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.191269  4501 catalog_manager.cc:5697] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:31.202355  4602 raft_consensus.cc:2955] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.218313  4752 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:31.228348  5400 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:31.229099  5348 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:31.232301  5195 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:31.242065  4672 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:31.244858  5401 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:31.262915  5400 raft_consensus.cc:2955] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.274871  4672 raft_consensus.cc:2955] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.277870  5348 raft_consensus.cc:2955] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.279307  4675 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:31.288501  5420 raft_consensus.cc:1064] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:24:31.288843  4751 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:31.289354  5349 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:31.290486  5255 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:31.291076  5420 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:31.293671  5428 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:31.297451  5429 raft_consensus.cc:2955] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.301860  4752 raft_consensus.cc:2955] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.304623  5255 raft_consensus.cc:1064] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:24:31.305500  4501 catalog_manager.cc:5697] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:31.306998  5255 consensus_queue.cc:237] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:31.312566  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ab695b6fa8e145c7926a540c3ba5b84e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:31.319242  5348 raft_consensus.cc:2955] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.339716  5349 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:31.343019  4751 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:31.345114  5188 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:31.350248  5188 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:31.358259  4602 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:31.360945  5429 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:31.360862  4499 catalog_manager.cc:5697] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:31.375442  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 66783.6us
I20260709 12:24:31.376154  5349 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:31.383123  4599 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:31.386273  5428 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:31.389513  4589 tablet_service.cc:1558] Processing DeleteTablet for tablet ab695b6fa8e145c7926a540c3ba5b84e with delete_type TABLET_DATA_TOMBSTONED (TS 5c3201048f8346009aefca07428828f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60018
I20260709 12:24:31.389791  4751 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:31.392818  5428 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:31.396417  5420 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:31.409168  4596 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:31.410387  4747 raft_consensus.cc:2955] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.424201  5448 tablet_replica.cc:333] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:24:31.425670  5448 raft_consensus.cc:2243] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:31.431730  5347 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:31.435526  5448 raft_consensus.cc:2272] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:31.435595  5442 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:31.483937  5448 ts_tablet_manager.cc:1916] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:31.484632  5439 raft_consensus.cc:2955] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:24:31.542666  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 65511.4us
I20260709 12:24:31.545653  5188 raft_consensus.cc:2955] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.536077  5347 raft_consensus.cc:2955] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.515446  4749 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:24:31.564553  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 118196us
I20260709 12:24:31.557905  4751 raft_consensus.cc:2955] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.589942  4602 raft_consensus.cc:2955] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:24:31.589509  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 133763us
I20260709 12:24:31.610064  5348 raft_consensus.cc:2955] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.568998  5420 raft_consensus.cc:2955] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.592409  5349 raft_consensus.cc:2955] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.636143  5448 ts_tablet_manager.cc:1929] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:31.636682  5448 log.cc:1199] T ab695b6fa8e145c7926a540c3ba5b84e P 5c3201048f8346009aefca07428828f3: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-0-root/wals/ab695b6fa8e145c7926a540c3ba5b84e
I20260709 12:24:31.632704  4499 catalog_manager.cc:5697] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.642351  4501 catalog_manager.cc:5697] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.648934  5195 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:24:31.655297  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 74655.2us
I20260709 12:24:31.716125  4747 raft_consensus.cc:2955] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.743510  4756 raft_consensus.cc:2955] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.783118  4665 tablet_service.cc:1558] Processing DeleteTablet for tablet 9ca7c574e14c4faeac378ee00c0a261b with delete_type TABLET_DATA_TOMBSTONED (TS 3a5aeedd11fe49938079fe86cb01232c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51854
I20260709 12:24:31.791836  4599 raft_consensus.cc:2955] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.788399  4501 catalog_manager.cc:5697] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:31.817473  4675 consensus_queue.cc:237] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:31.857695  4672 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:24:31.858001  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 258138us
I20260709 12:24:31.866613  5461 tablet_replica.cc:333] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:24:31.868232  5461 raft_consensus.cc:2243] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:31.871366  4486 catalog_manager.cc:5028] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): tablet ab695b6fa8e145c7926a540c3ba5b84e (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:24:31.872215  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9ca7c574e14c4faeac378ee00c0a261b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:31.882934  5461 raft_consensus.cc:2272] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:31.909488  5461 ts_tablet_manager.cc:1916] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:24:31.917953  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63275.8us
I20260709 12:24:31.921708  4748 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:31.924479  5462 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:31.925258  4600 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:31.931870  5255 raft_consensus.cc:2955] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.938890  5348 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:31.938740  4748 raft_consensus.cc:2955] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.941224  5348 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:31.943461  5447 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:31.946062  5428 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:31.951567  5428 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:31.958729  5349 raft_consensus.cc:2955] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.963539  5461 ts_tablet_manager.cc:1929] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:31.964124  5461 log.cc:1199] T 9ca7c574e14c4faeac378ee00c0a261b P 3a5aeedd11fe49938079fe86cb01232c: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root/wals/9ca7c574e14c4faeac378ee00c0a261b
I20260709 12:24:31.966403  5420 raft_consensus.cc:2955] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.969092  5348 raft_consensus.cc:2955] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.975028  4600 raft_consensus.cc:2955] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.978192  4501 catalog_manager.cc:5697] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:31.986265  4486 catalog_manager.cc:5028] TS 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): tablet 9ca7c574e14c4faeac378ee00c0a261b (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:24:31.997265  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c3d4ce22fd7947598cc1d1c7732285da with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:32.002898  4589 tablet_service.cc:1558] Processing DeleteTablet for tablet c3d4ce22fd7947598cc1d1c7732285da with delete_type TABLET_DATA_TOMBSTONED (TS 5c3201048f8346009aefca07428828f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60018
I20260709 12:24:32.003935  5448 tablet_replica.cc:333] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:24:32.005024  5448 raft_consensus.cc:2243] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:32.005825  5448 raft_consensus.cc:2272] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:32.006600  4501 catalog_manager.cc:5697] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 reported cstate change: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:32.012544  5448 ts_tablet_manager.cc:1916] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:32.019404  4737 tablet_service.cc:1558] Processing DeleteTablet for tablet a8ee74acbd5c45f0934f6f37d2154bc7 with delete_type TABLET_DATA_TOMBSTONED (TS 49277a7653524787a715232ab908bbc6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43852
I20260709 12:24:32.024682  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a8ee74acbd5c45f0934f6f37d2154bc7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:32.036412  5448 ts_tablet_manager.cc:1929] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:32.037002  5448 log.cc:1199] T c3d4ce22fd7947598cc1d1c7732285da P 5c3201048f8346009aefca07428828f3: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-0-root/wals/c3d4ce22fd7947598cc1d1c7732285da
I20260709 12:24:32.039825  5463 tablet_replica.cc:333] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:24:32.043385  4486 catalog_manager.cc:5028] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): tablet c3d4ce22fd7947598cc1d1c7732285da (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:24:32.068579  5463 raft_consensus.cc:2243] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:32.069391  5463 raft_consensus.cc:2272] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:32.072846  5463 ts_tablet_manager.cc:1916] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:32.085662  5463 ts_tablet_manager.cc:1929] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:32.086179  5463 log.cc:1199] T a8ee74acbd5c45f0934f6f37d2154bc7 P 49277a7653524787a715232ab908bbc6: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root/wals/a8ee74acbd5c45f0934f6f37d2154bc7
I20260709 12:24:32.088217  4488 catalog_manager.cc:5028] TS 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): tablet a8ee74acbd5c45f0934f6f37d2154bc7 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:24:33.095901  4600 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:33.103971  4748 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:24:33.104508  4568 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dafe783bf87845d6bb64a279ed7d0f75. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:33.105393  5401 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:33.105909  4675 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:33.107209  5452 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:33.114590  5452 raft_consensus.cc:2955] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:33.119978  4748 raft_consensus.cc:2955] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:33.128211  4675 raft_consensus.cc:2955] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:33.155501  4498 catalog_manager.cc:5697] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
W20260709 12:24:33.189782  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49791us
I20260709 12:24:33.203672  4748 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:33.218398  4672 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:33.219765  4600 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:33.220533  5454 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:33.225602  5454 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:33.232360  5457 raft_consensus.cc:2955] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
W20260709 12:24:33.237751  4719 consensus_peers.cc:597] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12796fc41f234615a607ba222506e893. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:33.236999  4672 raft_consensus.cc:2955] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:33.243669  4600 raft_consensus.cc:2955] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:33.251497  4596 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:33.256533  4499 catalog_manager.cc:5697] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:33.280902  4747 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:33.283115  4675 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:33.285126  5452 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:24:33.286157  4568 consensus_peers.cc:597] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c6f49e7bc416420cb85a6d4319f46c4d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:33.290343  5401 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:24:33.311159  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 64153.4us
I20260709 12:24:33.322134  4675 raft_consensus.cc:2955] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:33.320698  5401 raft_consensus.cc:2955] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:33.331161  4747 raft_consensus.cc:2955] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:33.347451  4675 consensus_queue.cc:237] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:33.352878  4499 catalog_manager.cc:5697] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:33.363298  4644 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d410e14541184b6bbaff38737ce4bc02. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:33.364115  4747 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:33.363086  4600 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:33.365962  5255 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:33.366973  5420 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:33.380995  5429 raft_consensus.cc:2955] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:33.383880  4600 raft_consensus.cc:2955] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:33.386910  4756 raft_consensus.cc:2955] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:33.403232  4499 catalog_manager.cc:5697] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:33.668138  5492 ts_tablet_manager.cc:933] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:24:33.671545  5492 tablet_copy_client.cc:323] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.193:45077
I20260709 12:24:33.673121  4615 tablet_copy_service.cc:140] P 5c3201048f8346009aefca07428828f3: Received BeginTabletCopySession request for tablet dafe783bf87845d6bb64a279ed7d0f75 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:53448)
I20260709 12:24:33.673727  4615 tablet_copy_service.cc:161] P 5c3201048f8346009aefca07428828f3: Beginning new tablet copy session on tablet dafe783bf87845d6bb64a279ed7d0f75 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:53448: session id = dbe960c35c84451faf4e5b5c9233fa39-dafe783bf87845d6bb64a279ed7d0f75
I20260709 12:24:33.684412  4615 tablet_copy_source_session.cc:215] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:33.687304  5494 ts_tablet_manager.cc:933] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:33.689420  5494 tablet_copy_client.cc:323] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.195:41049
I20260709 12:24:33.691833  5492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dafe783bf87845d6bb64a279ed7d0f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.708297  5492 tablet_copy_client.cc:806] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:33.708884  5492 tablet_copy_client.cc:670] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:33.712949  4766 tablet_copy_service.cc:140] P 49277a7653524787a715232ab908bbc6: Received BeginTabletCopySession request for tablet 12796fc41f234615a607ba222506e893 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:42632)
I20260709 12:24:33.713138  5492 tablet_copy_client.cc:538] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:33.713717  4766 tablet_copy_service.cc:161] P 49277a7653524787a715232ab908bbc6: Beginning new tablet copy session on tablet 12796fc41f234615a607ba222506e893 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:42632: session id = dbe960c35c84451faf4e5b5c9233fa39-12796fc41f234615a607ba222506e893
I20260709 12:24:33.722276  4766 tablet_copy_source_session.cc:215] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:33.724654  5492 tablet_bootstrap.cc:492] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:33.729445  5494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12796fc41f234615a607ba222506e893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.753873  5497 ts_tablet_manager.cc:933] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:24:33.754935  5494 tablet_copy_client.cc:806] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:33.755584  5494 tablet_copy_client.cc:670] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:33.762614  5497 tablet_copy_client.cc:323] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.193:45077
I20260709 12:24:33.763484  5494 tablet_copy_client.cc:538] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:33.764015  5492 tablet_bootstrap.cc:492] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:33.764894  5492 tablet_bootstrap.cc:492] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:33.765556  4615 tablet_copy_service.cc:140] P 5c3201048f8346009aefca07428828f3: Received BeginTabletCopySession request for tablet c6f49e7bc416420cb85a6d4319f46c4d from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:53448)
I20260709 12:24:33.766144  4615 tablet_copy_service.cc:161] P 5c3201048f8346009aefca07428828f3: Beginning new tablet copy session on tablet c6f49e7bc416420cb85a6d4319f46c4d from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:53448: session id = dbe960c35c84451faf4e5b5c9233fa39-c6f49e7bc416420cb85a6d4319f46c4d
I20260709 12:24:33.775383  5494 tablet_bootstrap.cc:492] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:33.778540  4615 tablet_copy_source_session.cc:215] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:33.765719  5492 ts_tablet_manager.cc:1403] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.041s	user 0.028s	sys 0.004s
I20260709 12:24:33.783159  5497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6f49e7bc416420cb85a6d4319f46c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.786360  5492 raft_consensus.cc:359] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:33.787598  5492 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:33.788321  5492 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:33.817054  5492 ts_tablet_manager.cc:1434] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.036s	user 0.011s	sys 0.003s
I20260709 12:24:33.821887  4615 tablet_copy_service.cc:342] P 5c3201048f8346009aefca07428828f3: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-dafe783bf87845d6bb64a279ed7d0f75 received from {username='slave'} at 127.0.0.1:53448
I20260709 12:24:33.822353  4615 tablet_copy_service.cc:434] P 5c3201048f8346009aefca07428828f3: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-dafe783bf87845d6bb64a279ed7d0f75 on tablet dafe783bf87845d6bb64a279ed7d0f75 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:33.828780  5494 tablet_bootstrap.cc:492] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:33.829917  5494 tablet_bootstrap.cc:492] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:33.830657  5494 ts_tablet_manager.cc:1403] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.056s	user 0.034s	sys 0.022s
I20260709 12:24:33.833978  5494 raft_consensus.cc:359] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:33.834887  5494 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:33.835601  5494 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:33.836630  5497 tablet_copy_client.cc:806] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:33.837941  5497 tablet_copy_client.cc:670] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:33.838368  5494 ts_tablet_manager.cc:1434] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:24:33.840013  4766 tablet_copy_service.cc:342] P 49277a7653524787a715232ab908bbc6: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-12796fc41f234615a607ba222506e893 received from {username='slave'} at 127.0.0.1:42632
I20260709 12:24:33.840457  4766 tablet_copy_service.cc:434] P 49277a7653524787a715232ab908bbc6: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-12796fc41f234615a607ba222506e893 on tablet 12796fc41f234615a607ba222506e893 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:33.841799  5497 tablet_copy_client.cc:538] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:33.858140  5497 tablet_bootstrap.cc:492] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:33.877946  5494 ts_tablet_manager.cc:933] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:33.878156  5497 tablet_bootstrap.cc:492] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:33.879110  5497 tablet_bootstrap.cc:492] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:33.879771  5497 ts_tablet_manager.cc:1403] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.011s
I20260709 12:24:33.882498  5494 tablet_copy_client.cc:323] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.194:42599
I20260709 12:24:33.884284  4691 tablet_copy_service.cc:140] P 3a5aeedd11fe49938079fe86cb01232c: Received BeginTabletCopySession request for tablet d410e14541184b6bbaff38737ce4bc02 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:48198)
I20260709 12:24:33.884838  4691 tablet_copy_service.cc:161] P 3a5aeedd11fe49938079fe86cb01232c: Beginning new tablet copy session on tablet d410e14541184b6bbaff38737ce4bc02 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:48198: session id = dbe960c35c84451faf4e5b5c9233fa39-d410e14541184b6bbaff38737ce4bc02
I20260709 12:24:33.884793  5497 raft_consensus.cc:359] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:33.885775  5497 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:33.886487  5497 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:33.891974  5497 ts_tablet_manager.cc:1434] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.005s
I20260709 12:24:33.892467  4691 tablet_copy_source_session.cc:215] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:33.893712  4615 tablet_copy_service.cc:342] P 5c3201048f8346009aefca07428828f3: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-c6f49e7bc416420cb85a6d4319f46c4d received from {username='slave'} at 127.0.0.1:53448
I20260709 12:24:33.894169  4615 tablet_copy_service.cc:434] P 5c3201048f8346009aefca07428828f3: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-c6f49e7bc416420cb85a6d4319f46c4d on tablet c6f49e7bc416420cb85a6d4319f46c4d with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:33.900255  5494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d410e14541184b6bbaff38737ce4bc02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:33.912298  5494 tablet_copy_client.cc:806] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:33.912940  5494 tablet_copy_client.cc:670] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:33.917223  5494 tablet_copy_client.cc:538] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:33.925133  5494 tablet_bootstrap.cc:492] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:33.944608  5494 tablet_bootstrap.cc:492] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:33.945559  5494 tablet_bootstrap.cc:492] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:33.946363  5494 ts_tablet_manager.cc:1403] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.008s
I20260709 12:24:33.949545  5494 raft_consensus.cc:359] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:33.950518  5494 raft_consensus.cc:740] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:33.951184  5494 consensus_queue.cc:260] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:33.953682  5494 ts_tablet_manager.cc:1434] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:24:33.956163  4691 tablet_copy_service.cc:342] P 3a5aeedd11fe49938079fe86cb01232c: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-d410e14541184b6bbaff38737ce4bc02 received from {username='slave'} at 127.0.0.1:48198
I20260709 12:24:33.956590  4691 tablet_copy_service.cc:434] P 3a5aeedd11fe49938079fe86cb01232c: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-d410e14541184b6bbaff38737ce4bc02 on tablet d410e14541184b6bbaff38737ce4bc02 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:34.126803  5349 raft_consensus.cc:1217] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:34.138343  5349 raft_consensus.cc:1217] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:34.243913  5349 raft_consensus.cc:1217] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:34.257308  5349 raft_consensus.cc:1217] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:34.662235  5454 raft_consensus.cc:1064] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:24:34.665102  5454 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:34.680527  4675 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:34.680543  5349 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:34.683336  4600 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:34.682404  5454 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:24:34.684478  5508 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:34.685392  5507 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:34.708091  5508 raft_consensus.cc:2955] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:34.709679  4675 raft_consensus.cc:2955] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:34.710772  4600 raft_consensus.cc:2955] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:34.722756  5349 raft_consensus.cc:2955] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:34.728888  4499 catalog_manager.cc:5697] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:34.748457  5452 raft_consensus.cc:1064] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:24:34.753703  5452 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:34.786429  4749 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:34.816730  5349 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:34.819828  5508 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:34.820127  4600 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:34.823609  5508 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:34.832618  5507 raft_consensus.cc:2955] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:34.840860  4749 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:34.840416  5349 raft_consensus.cc:2955] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:34.844254  4600 raft_consensus.cc:2955] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:34.848456  5348 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:34.849737  5452 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:24:34.850301  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49668us
I20260709 12:24:34.850893  5519 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
W20260709 12:24:34.858878  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 92808us
I20260709 12:24:34.862311  4677 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:34.867162  5470 raft_consensus.cc:1064] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:24:34.870793  4488 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 12796fc41f234615a607ba222506e893 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:34.877588  5506 raft_consensus.cc:1064] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:24:34.889334  4499 catalog_manager.cc:5697] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:34.902539  4665 tablet_service.cc:1558] Processing DeleteTablet for tablet 12796fc41f234615a607ba222506e893 with delete_type TABLET_DATA_TOMBSTONED (TS 3a5aeedd11fe49938079fe86cb01232c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51854
I20260709 12:24:34.901837  5470 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:34.898686  5506 consensus_queue.cc:237] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:24:34.940361  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 81717.8us
W20260709 12:24:34.941476  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 81934.5us
I20260709 12:24:34.949867  5471 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:34.952195  5525 tablet_replica.cc:333] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:24:34.961952  5348 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:34.964828  5470 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:34.965981  4677 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:34.969378  4748 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:34.973275  5525 raft_consensus.cc:2243] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:34.974139  5525 raft_consensus.cc:2272] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:34.973913  5523 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:34.974941  5471 raft_consensus.cc:2955] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:34.980438  5525 ts_tablet_manager.cc:1916] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:34.978070  5523 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:34.991072  4749 raft_consensus.cc:2955] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:34.993636  5349 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:34.996539  5506 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:34.996775  4747 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:34.999799  5505 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:35.000418  4600 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:35.005991  5518 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:35.010437  4672 raft_consensus.cc:2955] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.015209  4498 catalog_manager.cc:5697] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.081664  5452 raft_consensus.cc:2955] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:24:35.162544  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 122976us
W20260709 12:24:35.179168  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 85991.7us
I20260709 12:24:35.087430  4677 raft_consensus.cc:2955] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.214586  5525 ts_tablet_manager.cc:1929] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
W20260709 12:24:35.218055  5517 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.083s	user 0.000s	sys 0.005s
I20260709 12:24:35.222129  5525 log.cc:1199] T 12796fc41f234615a607ba222506e893 P 3a5aeedd11fe49938079fe86cb01232c: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root/wals/12796fc41f234615a607ba222506e893
I20260709 12:24:35.224155  4486 catalog_manager.cc:5028] TS 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): tablet 12796fc41f234615a607ba222506e893 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:24:35.244910  5505 raft_consensus.cc:2955] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.258718  5347 raft_consensus.cc:2955] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.258719  5349 raft_consensus.cc:2955] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.268159  4750 raft_consensus.cc:2955] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.268709  4752 raft_consensus.cc:2955] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.278499  4498 catalog_manager.cc:5697] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.290143  4599 raft_consensus.cc:2955] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.295483  5348 raft_consensus.cc:2955] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.303963  4598 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:35.316149  4499 catalog_manager.cc:5697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.318670  5348 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:35.324442  5537 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:35.341339  4750 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:35.345912  5523 raft_consensus.cc:2955] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.351325  4673 consensus_queue.cc:237] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:35.355597  5520 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:35.359261  5348 raft_consensus.cc:2955] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.364499  4598 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:35.372292  4672 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:35.375901  5537 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:35.376184  5349 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:35.377267  4599 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:35.372534  4750 raft_consensus.cc:2955] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.378582  5420 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:35.380692  4499 catalog_manager.cc:5697] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.383661  5543 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:35.390924  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c6f49e7bc416420cb85a6d4319f46c4d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:35.390976  5452 raft_consensus.cc:2955] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.391070  5538 raft_consensus.cc:2955] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.399458  5349 raft_consensus.cc:2955] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.401131  4665 tablet_service.cc:1558] Processing DeleteTablet for tablet c6f49e7bc416420cb85a6d4319f46c4d with delete_type TABLET_DATA_TOMBSTONED (TS 3a5aeedd11fe49938079fe86cb01232c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51854
I20260709 12:24:35.402230  5525 tablet_replica.cc:333] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:24:35.403231  5525 raft_consensus.cc:2243] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:35.403913  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d410e14541184b6bbaff38737ce4bc02 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:35.403936  5525 raft_consensus.cc:2272] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:35.410759  4599 raft_consensus.cc:2955] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.412017  5525 ts_tablet_manager.cc:1916] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:35.415555  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dafe783bf87845d6bb64a279ed7d0f75 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:35.418159  4499 catalog_manager.cc:5697] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:35.430275  5347 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:35.443692  5420 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:35.444941  4672 raft_consensus.cc:2955] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.452515  4737 tablet_service.cc:1558] Processing DeleteTablet for tablet d410e14541184b6bbaff38737ce4bc02 with delete_type TABLET_DATA_TOMBSTONED (TS 49277a7653524787a715232ab908bbc6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43852
I20260709 12:24:35.452491  5347 raft_consensus.cc:2955] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.460541  4500 catalog_manager.cc:5697] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:35.482338  5545 tablet_replica.cc:333] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:24:35.483413  5545 raft_consensus.cc:2243] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:35.484158  5545 raft_consensus.cc:2272] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:35.490077  4737 tablet_service.cc:1558] Processing DeleteTablet for tablet dafe783bf87845d6bb64a279ed7d0f75 with delete_type TABLET_DATA_TOMBSTONED (TS 49277a7653524787a715232ab908bbc6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43852
I20260709 12:24:35.493207  5545 ts_tablet_manager.cc:1916] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:35.493458  5525 ts_tablet_manager.cc:1929] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:35.494076  5525 log.cc:1199] T c6f49e7bc416420cb85a6d4319f46c4d P 3a5aeedd11fe49938079fe86cb01232c: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root/wals/c6f49e7bc416420cb85a6d4319f46c4d
I20260709 12:24:35.499099  4486 catalog_manager.cc:5028] TS 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): tablet c6f49e7bc416420cb85a6d4319f46c4d (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:24:35.508772  5545 ts_tablet_manager.cc:1929] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:35.509259  5545 log.cc:1199] T d410e14541184b6bbaff38737ce4bc02 P 49277a7653524787a715232ab908bbc6: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root/wals/d410e14541184b6bbaff38737ce4bc02
I20260709 12:24:35.511083  5545 tablet_replica.cc:333] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:24:35.511178  4488 catalog_manager.cc:5028] TS 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): tablet d410e14541184b6bbaff38737ce4bc02 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:24:35.512111  5545 raft_consensus.cc:2243] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:35.512919  5545 raft_consensus.cc:2272] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:35.516616  5545 ts_tablet_manager.cc:1916] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:35.528548  5545 ts_tablet_manager.cc:1929] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:35.528851  5545 log.cc:1199] T dafe783bf87845d6bb64a279ed7d0f75 P 49277a7653524787a715232ab908bbc6: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root/wals/dafe783bf87845d6bb64a279ed7d0f75
I20260709 12:24:35.530053  4488 catalog_manager.cc:5028] TS 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): tablet dafe783bf87845d6bb64a279ed7d0f75 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:24:36.576092  4673 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:36.583207  4756 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:24:36.583241  4644 consensus_peers.cc:597] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3cbe5bef0f514b01ac552ff54d4d13be. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:36.584874  4598 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:36.584750  5420 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:36.586251  5548 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:36.594902  5420 raft_consensus.cc:2955] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.596900  4598 raft_consensus.cc:2955] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.599781  4756 raft_consensus.cc:2955] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.612346  4599 consensus_queue.cc:237] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:36.616585  4498 catalog_manager.cc:5697] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.646332  4756 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:36.648195  5537 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:24:36.649745  4568 consensus_peers.cc:597] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23027d24ef1943bdbea76085a97b055c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:36.652918  4673 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:36.656093  5537 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:36.659089  5554 raft_consensus.cc:2955] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.661312  4750 raft_consensus.cc:2955] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.700527  4673 raft_consensus.cc:2955] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.698482  4598 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:36.709517  4501 catalog_manager.cc:5697] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.721670  4750 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:36.723831  4672 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:36.723070  5554 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 12:24:36.728063  4568 consensus_peers.cc:597] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8fc78aa9a3bf48afb52246b50ad414eb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:36.728107  5537 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:36.739430  5537 raft_consensus.cc:2955] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.742439  4750 raft_consensus.cc:2955] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.746608  4673 raft_consensus.cc:2955] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.760262  4672 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:36.761365  4501 catalog_manager.cc:5697] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.770149  4750 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:24:36.770143  4644 consensus_peers.cc:597] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1fdfc68cfdd1411cad1b4e974e65067a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:36.771682  4599 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:36.771797  5536 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:36.774922  5548 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:36.797904  5536 raft_consensus.cc:2955] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.799556  4750 raft_consensus.cc:2955] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.806946  4599 raft_consensus.cc:2955] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:36.819100  4498 catalog_manager.cc:5697] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:36.965670  5566 ts_tablet_manager.cc:933] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:36.967783  5566 tablet_copy_client.cc:323] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.194:42599
I20260709 12:24:36.969655  4691 tablet_copy_service.cc:140] P 3a5aeedd11fe49938079fe86cb01232c: Received BeginTabletCopySession request for tablet 3cbe5bef0f514b01ac552ff54d4d13be from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:48198)
I20260709 12:24:36.970278  4691 tablet_copy_service.cc:161] P 3a5aeedd11fe49938079fe86cb01232c: Beginning new tablet copy session on tablet 3cbe5bef0f514b01ac552ff54d4d13be from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:48198: session id = dbe960c35c84451faf4e5b5c9233fa39-3cbe5bef0f514b01ac552ff54d4d13be
I20260709 12:24:36.976104  4691 tablet_copy_source_session.cc:215] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:36.979712  5566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cbe5bef0f514b01ac552ff54d4d13be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:36.997309  5566 tablet_copy_client.cc:806] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:36.998049  5566 tablet_copy_client.cc:670] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:37.002038  5566 tablet_copy_client.cc:538] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:37.011485  5566 tablet_bootstrap.cc:492] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:37.033190  5566 tablet_bootstrap.cc:492] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:37.034222  5566 tablet_bootstrap.cc:492] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:37.034919  5566 ts_tablet_manager.cc:1403] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.024s	user 0.023s	sys 0.000s
I20260709 12:24:37.038095  5566 raft_consensus.cc:359] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:37.038961  5566 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:37.039652  5566 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:37.044749  5566 ts_tablet_manager.cc:1434] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:24:37.046439  4691 tablet_copy_service.cc:342] P 3a5aeedd11fe49938079fe86cb01232c: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-3cbe5bef0f514b01ac552ff54d4d13be received from {username='slave'} at 127.0.0.1:48198
I20260709 12:24:37.046900  4691 tablet_copy_service.cc:434] P 3a5aeedd11fe49938079fe86cb01232c: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-3cbe5bef0f514b01ac552ff54d4d13be on tablet 3cbe5bef0f514b01ac552ff54d4d13be with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:37.138058  5566 ts_tablet_manager.cc:933] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:24:37.139828  5566 tablet_copy_client.cc:323] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.193:45077
I20260709 12:24:37.141674  4615 tablet_copy_service.cc:140] P 5c3201048f8346009aefca07428828f3: Received BeginTabletCopySession request for tablet 8fc78aa9a3bf48afb52246b50ad414eb from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:53448)
I20260709 12:24:37.142220  4615 tablet_copy_service.cc:161] P 5c3201048f8346009aefca07428828f3: Beginning new tablet copy session on tablet 8fc78aa9a3bf48afb52246b50ad414eb from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:53448: session id = dbe960c35c84451faf4e5b5c9233fa39-8fc78aa9a3bf48afb52246b50ad414eb
I20260709 12:24:37.151242  4615 tablet_copy_source_session.cc:215] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:37.154865  5566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc78aa9a3bf48afb52246b50ad414eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:37.170549  5566 tablet_copy_client.cc:806] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:37.171113  5566 tablet_copy_client.cc:670] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:37.183893  5566 tablet_copy_client.cc:538] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:37.192566  5566 tablet_bootstrap.cc:492] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
W20260709 12:24:37.280586  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 64740.5us
W20260709 12:24:37.283258  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 61789.5us
I20260709 12:24:37.344594  5575 ts_tablet_manager.cc:933] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:24:37.356344  5575 tablet_copy_client.cc:323] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.193:45077
I20260709 12:24:37.358830  4615 tablet_copy_service.cc:140] P 5c3201048f8346009aefca07428828f3: Received BeginTabletCopySession request for tablet 23027d24ef1943bdbea76085a97b055c from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:53448)
I20260709 12:24:37.359359  4615 tablet_copy_service.cc:161] P 5c3201048f8346009aefca07428828f3: Beginning new tablet copy session on tablet 23027d24ef1943bdbea76085a97b055c from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:53448: session id = dbe960c35c84451faf4e5b5c9233fa39-23027d24ef1943bdbea76085a97b055c
I20260709 12:24:37.366863  4615 tablet_copy_source_session.cc:215] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:37.372833  5566 tablet_bootstrap.cc:492] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:37.374071  5566 tablet_bootstrap.cc:492] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:37.375672  5566 ts_tablet_manager.cc:1403] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.183s	user 0.040s	sys 0.063s
I20260709 12:24:37.381204  5566 raft_consensus.cc:359] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:37.382967  5566 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:37.383755  5566 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:37.387305  5566 ts_tablet_manager.cc:1434] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:24:37.389411  4615 tablet_copy_service.cc:342] P 5c3201048f8346009aefca07428828f3: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-8fc78aa9a3bf48afb52246b50ad414eb received from {username='slave'} at 127.0.0.1:53448
I20260709 12:24:37.389891  4615 tablet_copy_service.cc:434] P 5c3201048f8346009aefca07428828f3: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-8fc78aa9a3bf48afb52246b50ad414eb on tablet 8fc78aa9a3bf48afb52246b50ad414eb with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:37.396906  5575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23027d24ef1943bdbea76085a97b055c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:37.399441  5566 ts_tablet_manager.cc:933] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:37.415485  5349 raft_consensus.cc:1217] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:37.431211  5566 tablet_copy_client.cc:323] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.194:42599
I20260709 12:24:37.433013  4691 tablet_copy_service.cc:140] P 3a5aeedd11fe49938079fe86cb01232c: Received BeginTabletCopySession request for tablet 1fdfc68cfdd1411cad1b4e974e65067a from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:48198)
I20260709 12:24:37.433588  4691 tablet_copy_service.cc:161] P 3a5aeedd11fe49938079fe86cb01232c: Beginning new tablet copy session on tablet 1fdfc68cfdd1411cad1b4e974e65067a from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:48198: session id = dbe960c35c84451faf4e5b5c9233fa39-1fdfc68cfdd1411cad1b4e974e65067a
I20260709 12:24:37.441022  4691 tablet_copy_source_session.cc:215] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:37.444491  5575 tablet_copy_client.cc:806] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:37.444552  5566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdfc68cfdd1411cad1b4e974e65067a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:37.445182  5575 tablet_copy_client.cc:670] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:37.449833  5575 tablet_copy_client.cc:538] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:37.463495  5575 tablet_bootstrap.cc:492] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:37.467304  5566 tablet_copy_client.cc:806] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:37.467957  5566 tablet_copy_client.cc:670] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:37.471622  5566 tablet_copy_client.cc:538] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:37.483177  5566 tablet_bootstrap.cc:492] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:37.508225  5566 tablet_bootstrap.cc:492] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:37.509290  5566 tablet_bootstrap.cc:492] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:37.509976  5566 ts_tablet_manager.cc:1403] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.011s
I20260709 12:24:37.512771  5566 raft_consensus.cc:359] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:37.513378  5566 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:37.513996  5566 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:37.517709  5575 tablet_bootstrap.cc:492] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:37.518668  5575 tablet_bootstrap.cc:492] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:37.518682  5566 ts_tablet_manager.cc:1434] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.004s
I20260709 12:24:37.520016  5347 raft_consensus.cc:1217] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:37.520586  4691 tablet_copy_service.cc:342] P 3a5aeedd11fe49938079fe86cb01232c: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-1fdfc68cfdd1411cad1b4e974e65067a received from {username='slave'} at 127.0.0.1:48198
I20260709 12:24:37.520993  4691 tablet_copy_service.cc:434] P 3a5aeedd11fe49938079fe86cb01232c: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-1fdfc68cfdd1411cad1b4e974e65067a on tablet 1fdfc68cfdd1411cad1b4e974e65067a with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:37.528918  5575 ts_tablet_manager.cc:1403] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.066s	user 0.030s	sys 0.007s
I20260709 12:24:37.532948  5575 raft_consensus.cc:359] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:37.533849  5575 raft_consensus.cc:740] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:37.534462  5575 consensus_queue.cc:260] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:37.536932  5575 ts_tablet_manager.cc:1434] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:24:37.538600  4615 tablet_copy_service.cc:342] P 5c3201048f8346009aefca07428828f3: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-23027d24ef1943bdbea76085a97b055c received from {username='slave'} at 127.0.0.1:53448
I20260709 12:24:37.538987  4615 tablet_copy_service.cc:434] P 5c3201048f8346009aefca07428828f3: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-23027d24ef1943bdbea76085a97b055c on tablet 23027d24ef1943bdbea76085a97b055c with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:37.730614  5347 raft_consensus.cc:1217] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:37.838343  5420 raft_consensus.cc:1064] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:24:37.840413  5420 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:37.845932  5347 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:37.848366  5536 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:37.850457  4756 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:37.852178  4598 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:37.854362  5548 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:37.862882  5536 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:37.891263  5548 raft_consensus.cc:2955] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:24:37.892632  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 44709.6us
I20260709 12:24:37.902583  4598 raft_consensus.cc:2955] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:37.903916  5347 raft_consensus.cc:2955] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:37.909883  4756 raft_consensus.cc:2955] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:37.922698  5349 raft_consensus.cc:1217] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:37.935868  4498 catalog_manager.cc:5697] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:24:37.988039  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47346.1us
I20260709 12:24:38.019586  4677 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:38.028723  4756 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:38.029526  5347 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:38.031875  5420 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:24:38.034935  5587 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:38.040486  5586 raft_consensus.cc:2955] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.046978  4756 raft_consensus.cc:2955] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.052224  5347 raft_consensus.cc:2955] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.064982  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3cbe5bef0f514b01ac552ff54d4d13be with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:38.069725  4499 catalog_manager.cc:5697] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.084968  4589 tablet_service.cc:1558] Processing DeleteTablet for tablet 3cbe5bef0f514b01ac552ff54d4d13be with delete_type TABLET_DATA_TOMBSTONED (TS 5c3201048f8346009aefca07428828f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60018
I20260709 12:24:38.086977  5594 tablet_replica.cc:333] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:24:38.088037  5594 raft_consensus.cc:2243] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:38.088770  5594 raft_consensus.cc:2272] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:38.095206  5594 ts_tablet_manager.cc:1916] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:38.107597  5594 ts_tablet_manager.cc:1929] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:38.108000  5594 log.cc:1199] T 3cbe5bef0f514b01ac552ff54d4d13be P 5c3201048f8346009aefca07428828f3: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-0-root/wals/3cbe5bef0f514b01ac552ff54d4d13be
I20260709 12:24:38.109766  4486 catalog_manager.cc:5028] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): tablet 3cbe5bef0f514b01ac552ff54d4d13be (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:24:38.117709  5573 raft_consensus.cc:1064] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:24:38.120139  5573 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:38.130395  5347 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:38.132110  5554 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:38.132934  4677 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:38.134594  5573 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:38.138185  4748 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:38.144210  5554 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:38.167579  5598 raft_consensus.cc:2955] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.176221  5537 raft_consensus.cc:1064] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:24:38.178422  5537 consensus_queue.cc:237] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:38.188339  5347 raft_consensus.cc:2955] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.204607  4500 catalog_manager.cc:5697] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:38.213197  4677 raft_consensus.cc:2955] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.216348  4752 raft_consensus.cc:2955] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.252039  5347 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:38.254096  4598 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:38.257678  4677 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:38.261718  5554 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:38.264387  5554 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:38.268211  5348 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:38.268950  4752 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:38.270947  4677 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:38.271304  5598 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:38.272502  5537 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:38.273694  5598 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:38.280215  5554 raft_consensus.cc:2955] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.299283  4673 raft_consensus.cc:2955] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.300042  5347 raft_consensus.cc:2955] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.308849  5537 raft_consensus.cc:2955] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.317418  4500 catalog_manager.cc:5697] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 3 to 4, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.318148  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8fc78aa9a3bf48afb52246b50ad414eb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:38.322786  4752 raft_consensus.cc:2955] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.334663  4737 tablet_service.cc:1558] Processing DeleteTablet for tablet 8fc78aa9a3bf48afb52246b50ad414eb with delete_type TABLET_DATA_TOMBSTONED (TS 49277a7653524787a715232ab908bbc6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43852
I20260709 12:24:38.338562  5347 raft_consensus.cc:2955] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.378978  4499 catalog_manager.cc:5697] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.393469  5605 tablet_replica.cc:333] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:24:38.396696  5605 raft_consensus.cc:2243] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:38.397537  5605 raft_consensus.cc:2272] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:38.399580  4677 raft_consensus.cc:2955] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.419945  4602 consensus_queue.cc:237] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:38.422261  5605 ts_tablet_manager.cc:1916] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:38.431528  4752 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:38.432044  5347 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:38.433733  5573 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:38.437431  5600 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:38.450356  5605 ts_tablet_manager.cc:1929] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:38.450892  5605 log.cc:1199] T 8fc78aa9a3bf48afb52246b50ad414eb P 49277a7653524787a715232ab908bbc6: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root/wals/8fc78aa9a3bf48afb52246b50ad414eb
I20260709 12:24:38.452713  4488 catalog_manager.cc:5028] TS 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): tablet 8fc78aa9a3bf48afb52246b50ad414eb (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:24:38.449941  5554 raft_consensus.cc:2955] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.453755  4752 raft_consensus.cc:2955] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.455991  5347 raft_consensus.cc:2955] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.471936  4500 catalog_manager.cc:5697] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.485225  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 23027d24ef1943bdbea76085a97b055c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:38.497061  4665 tablet_service.cc:1558] Processing DeleteTablet for tablet 23027d24ef1943bdbea76085a97b055c with delete_type TABLET_DATA_TOMBSTONED (TS 3a5aeedd11fe49938079fe86cb01232c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51854
I20260709 12:24:38.499782  5606 tablet_replica.cc:333] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:24:38.501225  5606 raft_consensus.cc:2243] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:38.501971  5606 raft_consensus.cc:2272] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:38.506635  5606 ts_tablet_manager.cc:1916] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:38.521963  5606 ts_tablet_manager.cc:1929] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:38.522361  5606 log.cc:1199] T 23027d24ef1943bdbea76085a97b055c P 3a5aeedd11fe49938079fe86cb01232c: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root/wals/23027d24ef1943bdbea76085a97b055c
I20260709 12:24:38.524511  4486 catalog_manager.cc:5028] TS 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): tablet 23027d24ef1943bdbea76085a97b055c (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:24:38.542801  5588 raft_consensus.cc:1064] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:24:38.545336  5588 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:38.555812  5348 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:38.557195  4752 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:38.557723  4602 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:38.558663  5588 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:38.559682  5586 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:38.560693  5587 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:38.583489  5588 raft_consensus.cc:2955] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.584934  5347 raft_consensus.cc:2955] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.598475  4500 catalog_manager.cc:5697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.602504  4752 raft_consensus.cc:2955] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.606278  4598 raft_consensus.cc:2955] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.620363  4677 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:38.628258  5347 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:38.628546  4756 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:38.629806  5588 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:38.631013  5586 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:38.637812  5586 raft_consensus.cc:2955] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.642496  4752 raft_consensus.cc:2955] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.651892  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1fdfc68cfdd1411cad1b4e974e65067a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:38.654203  5348 raft_consensus.cc:2955] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:38.661479  4499 catalog_manager.cc:5697] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New cstate: current_term: 1 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:38.676460  4589 tablet_service.cc:1558] Processing DeleteTablet for tablet 1fdfc68cfdd1411cad1b4e974e65067a with delete_type TABLET_DATA_TOMBSTONED (TS 5c3201048f8346009aefca07428828f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60018
I20260709 12:24:38.678970  5611 tablet_replica.cc:333] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:24:38.679960  5611 raft_consensus.cc:2243] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:38.680662  5611 raft_consensus.cc:2272] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:38.683674  5611 ts_tablet_manager.cc:1916] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:38.695535  5611 ts_tablet_manager.cc:1929] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:38.696182  5611 log.cc:1199] T 1fdfc68cfdd1411cad1b4e974e65067a P 5c3201048f8346009aefca07428828f3: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-0-root/wals/1fdfc68cfdd1411cad1b4e974e65067a
I20260709 12:24:38.697677  4486 catalog_manager.cc:5028] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): tablet 1fdfc68cfdd1411cad1b4e974e65067a (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
W20260709 12:24:40.206408  4626 debug-util.cc:398] Leaking SignalData structure 0x7b0800061360 after lost signal to thread 3434
W20260709 12:24:40.208058  4626 debug-util.cc:398] Leaking SignalData structure 0x7b08000c0d80 after lost signal to thread 4536
W20260709 12:24:40.209721  4626 debug-util.cc:398] Leaking SignalData structure 0x7b08000edd00 after lost signal to thread 4630
W20260709 12:24:40.211161  4626 debug-util.cc:398] Leaking SignalData structure 0x7b080032a7c0 after lost signal to thread 4706
W20260709 12:24:40.212604  4626 debug-util.cc:398] Leaking SignalData structure 0x7b0800205da0 after lost signal to thread 4780
W20260709 12:24:40.214126  4626 debug-util.cc:398] Leaking SignalData structure 0x7b08002a2b80 after lost signal to thread 5373
I20260709 12:24:40.665194  5573 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.713750  5573 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:40.736282  5573 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:40.788223  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 825) took 1010 ms. Trace:
I20260709 12:24:40.789183  4598 rpcz_store.cc:276] 0709 12:24:39.777450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.787342 (+1009892us) service_pool.cc:224] Handling call
0709 12:24:40.787671 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.787861 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.787890 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:40.787929 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:40.787959 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.788014 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.788147 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.810984  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:40.797267  5573 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.814451  5573 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.817471  5573 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:40.833861  5621 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:40.834610  5621 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.836997  5621 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:40.860577  5598 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.861332  5598 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.863241  5574 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.863759  5598 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:40.863840  5574 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.867213  5612 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.869925  5616 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.867839  5612 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:40.870517  5616 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.872277  5612 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:40.872679  5616 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:40.873890  5615 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.874486  5615 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.876458  5616 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:40.876966  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 824) took 1099 ms. Trace:
I20260709 12:24:40.877483  4602 rpcz_store.cc:276] 0709 12:24:39.777164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.876142 (+1098978us) service_pool.cc:224] Handling call
0709 12:24:40.876459 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.876632 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.876661 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:40.876704 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:40.876738 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.876792 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.876910 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.880599  5615 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:40.887684  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:40.889920  5629 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.882481  5613 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.890873  5629 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:40.892004  5613 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.894671  5613 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:40.876982  5616 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.900194  5616 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:40.895216  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:40.895982  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:40.904134  5612 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:40.905046  5612 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.907876  5612 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:40.914400  5616 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:40.915364  5616 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.916920  5628 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.917639  5628 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.918504  5616 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:40.919329  5616 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:40.920130  5628 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:40.920011  5616 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.920737  5628 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:40.923336  5616 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:40.926362  5598 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.935226  5598 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.897676  5629 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:40.936756  5629 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.937397  5629 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:40.938444  5598 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:40.928448  5574 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:40.939599  5574 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:40.940375  5574 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.943557  5574 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:40.945267  5635 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:40.947687  5635 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:40.913496  5615 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.956959  5617 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:40.958721  5617 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:40.961107  5617 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:40.957707  5615 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.921264  5628 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:40.964262  5615 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:40.978264  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63402.6us
I20260709 12:24:40.984547  5629 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.028542  5616 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.030190  5625 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.030062  5613 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:40.966306  5627 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:40.999512  5612 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.031042  5621 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.032541  5598 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.033052  5620 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.033529  5573 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.033937  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.033960  5537 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.051008  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.039122  5616 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.042095  5629 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.048426  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.039732  5628 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.053772  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.053522  5537 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.061039  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.062047  4599 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:41.039238  5625 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.066435  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.067183  5625 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.054322  5621 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.064699  5629 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:41.079051  5537 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.079890  5537 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.060281  5612 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.092998  5537 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.050069  5613 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.098421  5631 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.069360  5598 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.076889  5637 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.070039  5628 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.106089  5537 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.106840  5537 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.107556  5537 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.086134  5620 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.115715  5620 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.059491  5573 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.109941  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:41.096413  5615 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:41.111707  5628 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.137831  5619 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.098995  5631 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:41.145879  5639 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:41.126778  5616 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.156474  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.158583  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.160135  5641 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.047425  5627 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.136077  5624 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:41.136529  5623 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.163728  5618 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.146730  5639 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.138625  5619 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.143934  5632 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.173949  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:41.168807  5628 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.174768  5638 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.175658  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.175897  5631 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.176654  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.179111  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.179567  5537 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.127475  5637 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:41.139235  5615 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.168679  5630 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:41.186311  5574 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.193943  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.194175  5633 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.160706  5641 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.169160  5626 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.176553  5624 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.187420  5635 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.189494  5612 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.193065  5625 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:41.197113  5628 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.200841  5621 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.215005  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.196563  5616 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.212957  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.207509  5617 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:41.190697  5623 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.190487  5618 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.212241  5631 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.448845  5631 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:41.449658  5631 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:24:41.427403  4533 debug-util.cc:398] Leaking SignalData structure 0x7b0800039440 after lost signal to thread 3434
I20260709 12:24:41.229552  5598 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.437250  5645 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.230525  5637 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.231518  5615 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.446094  5573 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.231700  5629 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:24:41.231997  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 63033.4us
I20260709 12:24:41.429016  5641 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.232645  5613 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.231678  5574 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.398584  5627 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.231474  5634 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.481910  5639 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.399533  5644 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.452647  5628 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.491410  5623 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.267501  5646 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.463739  5616 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.469751  5619 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.501415  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.229643  5643 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.488897  5650 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.520398  5654 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.230767  5640 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:41.467284  5617 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.470059  5625 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.520664  5650 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:41.278489  5633 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.499997  5651 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.511541  5631 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.237769  5630 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.209038  5638 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:41.526262  5619 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.545667  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.481820  5645 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.306733  5626 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.520932  5646 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.208817  5632 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:41.521277  5634 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:41.520983  5654 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.545663  5651 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.552600  5619 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.522533  5644 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.533975  5643 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.540079  5640 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.586980  5616 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.836095  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.835256  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.825074  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.834107  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.838546  5616 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.839267  5616 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.815100  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.816882  5653 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.791368  5645 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.841544  5653 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.842000  5645 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.842641  5645 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.844045  5653 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.844638  5653 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.625214  5650 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:41.845134  5653 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.845623  5650 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.846136  5656 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.846347  5650 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.847980  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.847523  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.814579  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.850769  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:41.805111  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.851037  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:41.851382  5347 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:41.857837  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.571890  5574 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.859972  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.621429  5644 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.805650  5638 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.824153  5641 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.871836  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.836794  5628 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.874815  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.881029  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.882802  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:41.847000  5656 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.880895  5649 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.886206  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.893270  5641 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:41.873299  4533 debug-util.cc:398] Leaking SignalData structure 0x7b0800038b00 after lost signal to thread 4536
I20260709 12:24:41.875085  5616 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.756155  5619 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.624341  5618 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.645273  5634 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:41.929829  5624 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.755352  5654 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.609284  5651 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.918519  5616 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:41.918545  5657 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.933362  5616 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.920751  5626 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.929703  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.897903  5649 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:41.623654  5633 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.803632  5643 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.935657  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.936972  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.854792  5645 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.874266  5632 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.933640  5657 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:41.941424  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.941823  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.942286  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.942950  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.943650  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.943735  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.944339  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.944615  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.945222  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.945948  5650 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.944693  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.974371  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.976516  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:41.950603  4533 debug-util.cc:398] Leaking SignalData structure 0x7b080032e4a0 after lost signal to thread 4630
I20260709 12:24:41.976890  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.951756  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:41.973348  5638 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.972600  5574 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.980376  5573 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.990540  5649 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:41.990952  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:41.991930  5616 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:41.992071  5649 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:41.992551  5649 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.992993  5649 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.987914  5630 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.990043  5645 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:41.999886  4600 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:41.990728  5573 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.003023  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.003260  5573 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.003679  4748 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:42.004889  5656 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:42.005614  5653 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:42.006306  5640 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.007567  5649 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.007802  5645 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.009634  5649 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:42.009646  5645 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:24:42.010072  4533 debug-util.cc:398] Leaking SignalData structure 0x7b0800326540 after lost signal to thread 4706
W20260709 12:24:42.011212  4533 debug-util.cc:398] Leaking SignalData structure 0x7b08002e7e80 after lost signal to thread 4780
W20260709 12:24:42.012444  4533 debug-util.cc:398] Leaking SignalData structure 0x7b0800315a20 after lost signal to thread 5373
I20260709 12:24:41.985368  5625 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:41.985555  5617 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.019093  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.019096  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.999167  5646 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.023128  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.030543  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.032385  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.016455  5641 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:42.016252  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.034940  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.035892  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.036785  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.038739  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.040603  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:41.997819  5574 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.043118  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.017412  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.043812  4748 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:42.044826  5638 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.045830  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.047269  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.044845  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.051092  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.051657  4756 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:42.046398  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.052556  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.053912  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.055133  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.055883  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.056152  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.056502  4749 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:42.058020  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.061264  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.060746  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.061959  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.063300  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.063882  4748 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
W20260709 12:24:42.049695  4722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.003s	sys 0.015s
I20260709 12:24:42.046314  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.064591  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.069100  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.069924  4750 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 2: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:42.071233  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.072396  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.072965  4750 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:42.074465  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.074679  5574 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:42.075593  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.076149  4749 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:42.070621  5638 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.077520  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.079700  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.080042  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.081267  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.082494  4644 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:42.084573  5639 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.085402  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.085692  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.086069  4750 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:42.086262  4756 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:42.086530  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.087074  4749 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:42.087468  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.088020  4756 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:42.088037  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.089748  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.090482  4749 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:42.089829  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.091388  4756 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:42.093050  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.095651  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:42.068928  4722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.003s	sys 0.016s
I20260709 12:24:42.099236  5574 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.099666  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.098484  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:42.104030  4568 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e20c8bae3534224bb7a82db42cab1a5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:42.105120  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.099750  5574 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.105935  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.106597  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.106755  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.107601  5574 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:42.109639  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.110450  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.110859  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.111197  4748 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 1.
I20260709 12:24:42.111479  4756 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 1.
I20260709 12:24:42.111522  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.111915  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.112181  4750 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 1.
I20260709 12:24:42.112514  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.112532  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:42.112895  5657 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:42.112993  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.113734  4752 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 1.
I20260709 12:24:42.113763  5638 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.114629  5638 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.115010  5638 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.116952  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.117249  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.118254  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.118582  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.119215  4749 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:42.119731  4750 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:42.120690  4747 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.118595  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.123952  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.125483  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.132000  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.130380  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.132819  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.133159  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.133674  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.129629  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.132906  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.132833  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.131798  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.135908  5314 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:42.136305  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.133065  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.137111  5632 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.137519  5632 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.137917  5632 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.137815  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.139179  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.140206  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.134016  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.140350  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.142251  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.142591  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.142189  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.143958  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.144640  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:42.146027  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.147722  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.148514  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.147770  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.149981  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.152151  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.153147  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.154560  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.157056  5638 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.160423  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.197983  5348 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:42.161104  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 75680.8us
I20260709 12:24:42.211144  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.182047  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:42.160355  5632 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.214319  5657 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:42.223469  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 78935.5us
I20260709 12:24:42.225669  5663 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.227451  5663 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.226045  5314 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.228107  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.228942  5663 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.233665  5573 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.241057  5573 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.228888  5660 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:24:42.243157  5573 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.244716  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:42.245092  5638 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.245530  5627 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.224707  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.247131  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.247874  5574 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.248759  5348 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.249598  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.249939  5632 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:42.250525  5632 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.250753  5314 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:42.247812  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:42.251606  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.252249  5664 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.253149  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.253687  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.254367  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:42.251601  5638 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.256211  5638 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.256655  5638 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.257777  5623 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:42.259032  5623 raft_consensus.cc:3055] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:42.259404  5623 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:42.260787  4571 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:42.260438  5623 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.262313  5623 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.263055  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:42.263315  5664 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.263741  5664 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.264132  5664 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.264730  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.266041  5314 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.266140  5632 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.267350  5632 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.269296  4571 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:42.269426  5639 raft_consensus.cc:3055] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:42.270390  5639 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:42.270262  5663 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.271165  5664 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.271381  5639 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.272557  5639 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.273080  5662 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.273610  5662 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.274061  5662 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.274869  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.279626  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.280014  4571 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:42.281678  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:42.281913  4571 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:42.282609  5642 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.282984  5642 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.283337  5642 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.284876  4571 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:42.284991  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.288276  5573 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.288800  5573 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.289251  5573 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.272583  5663 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.294268  5613 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.294833  5613 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.295296  5613 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.302310  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.304553  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.315414  5660 consensus_queue.cc:1059] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:42.317907  5660 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:42.318390  5660 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:42.318912  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.320580  5660 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.320924  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.321787  5660 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.322747  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.279520  5664 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.325522  4681 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:42.328526  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.329748  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.296732  5657 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.330693  5574 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.325260  5642 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.343566  4719 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:24:42.344657  5616 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.345170  5616 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.344591  5642 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.345597  5616 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.347540  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.336381  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.349179  5650 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.350984  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.352488  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:24:42.356995  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 72365.4us
I20260709 12:24:42.361693  4569 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.364677  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.347545  5659 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.366641  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.367300  5573 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:42.367717  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 82920.8us
I20260709 12:24:42.368588  4598 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.368932  4571 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:42.369515  5573 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.373488  5573 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.373497  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.354977  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.328711  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.355672  5650 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.365226  5643 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.375751  5574 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.380015  5642 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.376693  5651 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.376542  5633 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.388862  5616 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.394194  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:42.399456  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.383728  5665 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.401690  5643 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.402453  5650 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.400614  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:24:42.417757  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 87584.3us
I20260709 12:24:42.431015  5616 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:42.432190  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:24:42.392865  5642 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.441301  5642 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.443652  5662 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.432058  5630 consensus_queue.cc:1059] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:24:42.446247  5630 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:24:42.391327  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 78706.9us
I20260709 12:24:42.444761  4602 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.446652  5630 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:42.450114  5630 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.451439  5630 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:42.454154  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 53555.6us
I20260709 12:24:42.455250  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.458168  5616 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.459201  5641 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.466324  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.466468  5642 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.469682  5613 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.472057  5642 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.473253  5651 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.474948  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.483142  5651 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.484700  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.488885  4720 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.478754  5613 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.492193  5616 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.439697  5638 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.499368  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.501932  5638 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.503353  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.503749  5662 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.503535  5616 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.504796  5314 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.505168  4720 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:42.506156  5638 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.507292  5616 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.509038  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.457904  4674 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:42.511744  5616 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.513448  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.514851  5616 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:42.519248  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 168206us
I20260709 12:24:42.518219  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.523993  5616 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.524402  4647 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:24:42.526737  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 157868us
I20260709 12:24:42.528205  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.528918  5616 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.530202  4720 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.531396  4602 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.531649  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.532359  5616 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.532323  5669 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:24:42.533054  5648 consensus_queue.cc:1059] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:42.533937  5616 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.534370  5669 raft_consensus.cc:3055] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:42.534441  5617 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:42.534948  5669 raft_consensus.cc:740] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:42.534996  5617 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:42.536013  5617 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.536013  5669 consensus_queue.cc:260] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:42.537150  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 175590us
I20260709 12:24:42.537508  5617 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.537698  5669 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.538926  4647 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.539764  4571 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:42.529904  5672 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.538952  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.543252  5630 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.543752  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.545150  4647 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.545406  5623 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.542578  4569 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.546263  5671 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.546826  5671 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.547245  5671 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.547701  4602 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.548666  4571 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:42.548866  5641 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.549548  5673 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.549656  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:42.550140  5673 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.550577  5673 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.550644  5660 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.551684  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.552042  4647 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.552946  5642 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.553028  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.555577  5656 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.555265  5641 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.556170  5667 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.556613  5667 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.557060  5667 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.559084  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.561049  5641 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.562278  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.562994  5641 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.563382  5648 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:42.564163  5642 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.564337  4674 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:42.564579  5648 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.565747  5648 raft_consensus.cc:3055] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:42.566112  5648 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:42.566289  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.567240  5641 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.567039  5648 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.568526  5648 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.568421  5667 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.568801  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:42.569924  5669 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:42.570214  4571 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:42.573987  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.575562  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.576344  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.576692  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.574678  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.577592  4571 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:42.578860  5641 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.578903  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.579002  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.579900  5660 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.580317  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.580598  5660 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.581310  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.581108  5672 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.578199  5642 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.583173  5642 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.583745  5642 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.584514  5670 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:42.585016  5670 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.586602  4647 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.570559  5669 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.625793  5672 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.626014  5669 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.626163  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.627537  5673 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.630035  5673 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.630256  5670 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.631093  4571 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:42.632643  5617 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:24:42.633555  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 51611.2us
I20260709 12:24:42.634078  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.635803  5630 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:42.636384  5630 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.638403  5630 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.639966  4600 raft_consensus.cc:1240] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.644661  5348 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.644429  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.645931  5630 consensus_queue.cc:1059] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:42.647868  5672 raft_consensus.cc:3055] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:42.648233  5672 raft_consensus.cc:740] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:42.648387  5669 consensus_queue.cc:1059] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:42.649369  5623 raft_consensus.cc:3055] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:42.649111  5672 consensus_queue.cc:260] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.649811  5623 raft_consensus.cc:740] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:42.650159  4720 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.650722  5672 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.650925  5641 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.652096  5573 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.633249  5617 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.650725  5623 consensus_queue.cc:260] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.652493  5573 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.654307  5573 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.654394  5670 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.654837  5670 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.655197  5670 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.652740  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:24:42.653913  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 67391.2us
I20260709 12:24:42.653713  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:42.652066  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.657325  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.658083  4672 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:42.641909  5671 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.658409  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:42.659298  5623 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.661768  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.662053  4598 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.665022  5656 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.665442  5656 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.665858  5656 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.665910  5616 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:42.652321  5667 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.669468  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.672096  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.673177  5626 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.675657  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.676687  4571 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.677759  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:42.681032  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.680528  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.684118  5668 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.684235  5613 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.684626  5668 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.685158  5668 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.684716  5613 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:42.666512  5616 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:42.661175  5641 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.687322  5664 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.690138  5666 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.687507  5625 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.706231  5348 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.709833  5625 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:42.694458  4677 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:42.697343  4676 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:42.697992  5626 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:42.699258  5640 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:42.675751  5669 consensus_queue.cc:1059] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:24:42.701753  5616 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.671170  5660 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.693153  5613 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.700712  5642 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.668879  5630 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:42.710918  5648 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:42.718983  4678 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:42.718551  5630 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.722889  5630 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:42.728744  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:42.714905  5626 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.731729  5626 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.739533  5573 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:42.742599  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 64910us
I20260709 12:24:42.745534  5573 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.745747  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.746915  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.747925  5667 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:42.751569  5676 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:42.752184  5676 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.754501  5676 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.718786  5648 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:42.756006  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.757182  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.757932  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.790831  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.791193  5625 raft_consensus.cc:3055] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:42.791617  5625 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:42.791790  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.792122  4647 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.774679  4598 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.781164  4720 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:42.737677  5640 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.795962  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.797097  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.780769  4602 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.799116  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.803762  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.768894  5671 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.769170  4674 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:42.810804  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.795217  5626 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.797168  5639 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.798379  5616 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:42.815990  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.800140  5617 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.824033  4600 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.792594  5625 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.818956  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.776202  5668 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.826126  5616 raft_consensus.cc:697] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:42.839888  5616 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.838111  4499 catalog_manager.cc:5697] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:42.844959  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.845197  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.853458  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.855118  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.856596  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.859887  5613 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.863950  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.824905  5626 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:42.867466  4569 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.865705  5674 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:24:42.873958  5642 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.811805  5648 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.865917  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:42.898048  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.899838  5613 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.904356  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:42.899319  4569 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.877660  5626 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.903590  5666 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.911223  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.895176  5681 ts_tablet_manager.cc:933] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:24:42.917294  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:42.907831  5648 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.916218  5662 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.918992  5675 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.925477  4569 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.926782  5626 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.920899  5677 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.927161  5662 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:42.927942  5662 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:42.928110  5640 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.914800  5656 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:42.930101  5625 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.930647  5656 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:42.929134  5662 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.935537  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:24:42.937040  5348 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:24:42.939162  4569 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.939486  5668 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.940603  5668 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:42.941165  5668 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:42.932948  5670 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.943902  4596 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.944881  5670 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:42.943377  5668 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.949821  4569 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.952041  5664 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:42.952889  5682 consensus_queue.cc:1059] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:42.952499  4569 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.953138  5677 consensus_queue.cc:1059] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:42.953444  5664 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:42.956683  4596 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:42.962358  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.956987  5673 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:42.966120  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.966491  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.969735  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:42.970839  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:24:42.971701  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 88614.1us
I20260709 12:24:42.972013  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:42.973007  4647 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.964417  5673 raft_consensus.cc:697] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:42.975492  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.975790  4569 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:42.978704  4647 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:42.964022  5624 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:42.967322  5626 consensus_queue.cc:1059] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:24:42.974924  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:42.976543  5675 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.991744  5646 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.976519  5617 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:42.998028  5664 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.953809  5648 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:24:43.010104  5640 raft_consensus.cc:3055] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:43.010589  5640 raft_consensus.cc:740] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:43.011881  5624 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:43.011539  5640 consensus_queue.cc:260] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:43.012281  5624 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:43.012434  5671 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:43.013134  5624 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:43.014492  5624 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.014374  5673 consensus_queue.cc:237] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:42.979761  5677 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:43.057036  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:43.061379  5682 raft_consensus.cc:3055] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:43.061276  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:43.061993  5682 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:43.064742  4571 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:43.066509  4645 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:24:43.067373  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 64682.1us
I20260709 12:24:43.059454  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:24:43.074205  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 46906.6us
I20260709 12:24:43.078003  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:43.079414  5640 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.080924  4645 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:43.007292  5680 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:43.084833  5675 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:43.092969  4678 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:43.094579  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:43.088593  5639 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:43.087687  5677 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:43.115670  4500 catalog_manager.cc:5697] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 1 to 2, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:43.062990  5682 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:43.099726  5656 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:43.132551  5676 raft_consensus.cc:3055] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:43.133193  5676 raft_consensus.cc:740] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:43.134769  5676 consensus_queue.cc:260] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:43.136693  5676 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.141901  5656 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:43.143601  5656 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:43.156302  5617 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:43.122512  5682 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.158533  5617 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:43.172036  4673 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
W20260709 12:24:43.181183  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 115273us
I20260709 12:24:43.193245  4571 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:43.193995  5675 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:43.195200  5680 raft_consensus.cc:2955] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:43.201779  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:43.202903  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:43.214248  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:43.214448  4747 raft_consensus.cc:2955] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:43.215556  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:43.243479  4599 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:43.210359  5681 tablet_copy_client.cc:323] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.193:45077
I20260709 12:24:43.250669  5617 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:43.257357  5617 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.254498  5649 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:24:43.291271  4681 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:24:43.295006  5685 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.232s	user 0.004s	sys 0.002s
I20260709 12:24:43.302866  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:43.307243  5671 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:43.307277  4571 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:43.308487  5671 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:43.310343  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:43.312100  5664 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:43.323568  4569 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:43.327392  4596 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:43.331418  4596 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:43.338354  5348 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:43.349220  4596 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:43.346509  4615 tablet_copy_service.cc:140] P 5c3201048f8346009aefca07428828f3: Received BeginTabletCopySession request for tablet 3e20c8bae3534224bb7a82db42cab1a5 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:53448)
I20260709 12:24:43.355232  4615 tablet_copy_service.cc:161] P 5c3201048f8346009aefca07428828f3: Beginning new tablet copy session on tablet 3e20c8bae3534224bb7a82db42cab1a5 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:53448: session id = dbe960c35c84451faf4e5b5c9233fa39-3e20c8bae3534224bb7a82db42cab1a5
I20260709 12:24:43.356153  5653 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:43.365952  4674 raft_consensus.cc:2955] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:43.338550  5642 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:43.388984  5677 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:24:43.398191  5677 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:43.398713  5677 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:43.399904  5677 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:43.401558  5677 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:43.419140  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 143160us
I20260709 12:24:43.391676  5624 consensus_queue.cc:1059] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:24:43.418731  5642 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:43.391984  5639 consensus_queue.cc:1059] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:24:43.460088  5639 raft_consensus.cc:3055] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:24:43.473058  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 63106.3us
I20260709 12:24:43.467195  5639 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:43.453059  5645 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:24:43.502570  5642 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:43.528410  4754 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.529284  4751 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.533774  5639 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:43.546353  4676 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:43.558097  4749 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.566148  4751 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:43.582423  4749 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:43.588351  4754 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:43.601182  5670 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:24:43.615046  5672 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:43.615635  5672 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:43.623646  5672 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:43.625131  5672 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.625413  5639 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.630137  5649 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:43.630730  5649 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:43.651531  4755 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.653326  5641 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.653848  5641 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:43.655822  5641 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:43.659008  5649 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:43.650924  5705 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:43.659693  5702 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:43.661494  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:43.662300  5347 raft_consensus.cc:2393] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:24:43.659970  5705 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:43.650569  5668 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
W20260709 12:24:43.666764  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 45751.7us
I20260709 12:24:43.666566  4500 catalog_manager.cc:5697] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:43.683588  4500 catalog_manager.cc:5697] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:43.685389  4675 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:43.686079  4673 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:43.668771  4678 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:43.692065  5671 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:43.692461  5702 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:43.693507  5671 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:24:43.698575  5347 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:43.685405  4500 catalog_manager.cc:5697] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:43.706286  5706 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:43.675275  4755 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:43.714999  5694 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:43.720674  5680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:53440 (request call id 5) took 1836 ms. Trace:
I20260709 12:24:43.725467  4675 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:43.745985  5705 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:43.781584  5675 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:24:43.772898  5680 rpcz_store.cc:276] 0709 12:24:41.882660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.884116 (+  1456us) service_pool.cc:224] Handling call
0709 12:24:42.011675 (+127559us) log.cc:844] Serialized 499 byte log entry
0709 12:24:42.054780 (+ 43105us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b180030a420. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c9522 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:24:42.083350 (+ 28570us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b180030a420. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007effeb3cba40 00007effeb3c64e4 00007effeb3c6455 00007effee40fb0b 00007effee494b63 00007effee4bc752 00007efff347cd29
0709 12:24:43.716865 (+1633515us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33113,"raft.queue_time_us":47773,"raft.run_cpu_time_us":1929,"raft.run_wall_time_us":2503,"spinlock_wait_cycles":8850048,"thread_start_us":29054,"threads_started":1,"wal-append.queue_time_us":59223,"wal-append.run_cpu_time_us":1729,"wal-append.run_wall_time_us":1003248}
W20260709 12:24:43.848567  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 325375us
I20260709 12:24:43.883373  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:43.884061  4672 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:43.889119  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:43.893286  4677 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:43.866153  5706 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:24:43.902639  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:43.904299  4677 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:43.911607  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:43.912262  4615 tablet_copy_source_session.cc:215] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:43.920204  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:43.927218  4603 raft_consensus.cc:2393] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:44.011483  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:44.025100  5681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e20c8bae3534224bb7a82db42cab1a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:44.038072  4755 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:44.045604  5713 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:44.049077  5713 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:44.053020  5713 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:44.051612  4500 catalog_manager.cc:5697] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:24:44.060019  4500 catalog_manager.cc:5697] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:24:44.062108  4500 catalog_manager.cc:5697] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:24:44.088876  5703 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.089538  5703 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:24:44.079517  5722 log.cc:927] Time spent T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.077s	user 0.002s	sys 0.000s
I20260709 12:24:44.108664  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:44.129935  5705 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:44.146908  5703 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.119524  5706 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:24:44.143663  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:44.161343  4678 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:44.162129  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:44.162909  5312 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:24:44.163777  5703 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:44.164240  5703 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.148900  5705 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:44.164670  5703 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.174611  5713 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:44.164731  5705 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:44.178532  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:24:44.168617  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:44.179312  5349 raft_consensus.cc:2393] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:24:44.183007  5616 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:44.198861  5616 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:44.199659  5616 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.204488  5676 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:44.211321  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 241085us
I20260709 12:24:44.278506  5641 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:44.279229  5641 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:44.299331  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.207445  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 46557us
I20260709 12:24:44.228597  4681 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:44.179248  5713 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:44.251163  5706 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:44.216326  5616 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:44.205119  5676 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.232007  5703 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:44.339839  5641 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:44.359459  5677 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.360244  5677 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.362455  5677 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.365659  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:44.366498  4598 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:44.367925  4645 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:44.325845  5705 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.371337  5740 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:44.382858  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:44.383589  5740 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.383785  4675 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.384050  5740 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.387771  5703 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.389227  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:44.390345  4756 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.331147  5706 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.395182  4747 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:44.396607  5706 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:44.400312  5676 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.400612  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:44.401293  4598 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:44.402547  4645 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:44.405622  5676 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:44.406020  5676 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.406369  5676 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.383005  5705 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:44.409384  5681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:50644 (request call id 105) took 1552 ms. Trace:
I20260709 12:24:44.409843  5681 rpcz_store.cc:276] 0709 12:24:42.857308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.857475 (+   167us) service_pool.cc:224] Handling call
0709 12:24:42.918659 (+ 61184us) ts_tablet_manager.cc:934] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
0709 12:24:44.023830 (+1105171us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:44.091078 (+ 67248us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:44.091209 (+   131us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:44.365583 (+274374us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:44.408928 (+ 43345us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:44.409240 (+   312us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:44.409305 (+    65us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":28833,"tablet-copy.queue_time_us":37185,"thread_start_us":223752,"threads_started":2}
I20260709 12:24:44.411320  5681 tablet_copy_client.cc:806] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:44.411962  5681 tablet_copy_client.cc:670] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:44.416263  5681 tablet_copy_client.cc:538] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:44.392792  4675 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:24:44.423964  5312 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:24:44.424489  5740 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.424952  5737 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:44.425595  5737 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:24:44.426450  5737 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:44.428273  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:44.428982  4598 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.430424  5740 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.430606  4747 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:24:44.434283  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 74854.7us
I20260709 12:24:44.445387  5681 tablet_bootstrap.cc:492] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:44.452839  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:44.452796  5743 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:44.453487  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:44.453836  4749 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:44.459172  4756 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:24:44.459440  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:44.460184  4751 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.469775  5701 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.470619  5701 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:44.475112  5701 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:44.474579  4598 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:44.478785  4645 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:44.477208  4751 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:44.479686  5740 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:44.480261  5740 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:44.481171  5740 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.486896  4500 catalog_manager.cc:5697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 1 to 3, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 3 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:44.554965  4719 consensus_peers.cc:597] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 28f87749718d445ba1615ff33028c4d8. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:44.555145  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 233167us
I20260709 12:24:44.560518  5681 tablet_bootstrap.cc:492] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:44.561650  5681 tablet_bootstrap.cc:492] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:44.562407  5681 ts_tablet_manager.cc:1403] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.117s	user 0.033s	sys 0.004s
W20260709 12:24:44.563180  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.570300  5677 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.564699  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:44.566279  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 62303.6us
I20260709 12:24:44.570974  5677 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.571547  4603 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:24:44.573462  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 66439.9us
I20260709 12:24:44.567742  5681 raft_consensus.cc:359] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:44.570290  5676 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.579078  5664 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:44.579820  5677 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.584451  4499 catalog_manager.cc:5697] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 1 to 2, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:44.606613  5676 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.587685  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:44.611889  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:44.612712  4598 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:44.613343  5681 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:44.614101  5681 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:44.617100  5681 ts_tablet_manager.cc:1434] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.000s
I20260709 12:24:44.621279  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:44.622004  4603 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:44.628971  4615 tablet_copy_service.cc:342] P 5c3201048f8346009aefca07428828f3: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-3e20c8bae3534224bb7a82db42cab1a5 received from {username='slave'} at 127.0.0.1:53448
I20260709 12:24:44.629227  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:44.629472  4615 tablet_copy_service.cc:434] P 5c3201048f8346009aefca07428828f3: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-3e20c8bae3534224bb7a82db42cab1a5 on tablet 3e20c8bae3534224bb7a82db42cab1a5 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:44.630149  4603 raft_consensus.cc:2393] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:44.633170  4603 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:44.637470  4498 catalog_manager.cc:5697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:44.646968  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:44.647156  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:44.647749  4674 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:44.649782  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:44.650074  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:44.650938  4747 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:44.651932  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:44.652659  4676 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:44.656412  5649 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:44.657033  5649 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.659307  5649 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:44.659816  4645 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:44.660595  5676 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:44.660967  5676 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.661296  5676 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.664774  4675 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:44.666793  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:44.682618  5701 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:44.686997  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:44.687918  4603 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.695425  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:44.700752  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:44.700937  4676 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:44.702140  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:44.704502  4674 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:24:44.708141  5737 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.708748  5737 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:44.710645  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:44.710919  5737 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.711462  4676 raft_consensus.cc:2368] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:44.711613  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:44.714680  5677 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.715173  5677 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.716004  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:44.716200  4756 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.716965  5701 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:44.721050  5701 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:44.721585  5701 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:44.724372  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:44.725127  4749 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:44.726233  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:44.726553  5314 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:44.727350  5737 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:44.727747  5737 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.728099  5737 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.717913  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:44.718144  5677 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.734406  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:44.735033  4596 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:44.735316  5649 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:44.735760  5649 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:44.736225  5649 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:44.735473  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:44.736917  5641 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:44.737424  5641 raft_consensus.cc:697] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:44.737926  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:44.738438  5713 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:44.739177  5616 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:44.739588  5616 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.738308  5641 consensus_queue.cc:237] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.739812  4720 leader_election.cc:400] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:44.740773  4720 leader_election.cc:403] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:44.744339  5616 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:44.745085  4645 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:44.745958  5677 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:44.746342  5677 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.746701  5677 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.747193  5649 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.752269  5616 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:44.751489  5737 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:44.720439  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:44.757903  4677 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:44.764869  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:44.755600  5677 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.768607  4596 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:44.771076  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:44.771389  5677 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.772444  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:44.773108  4596 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.775426  5676 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.776674  5737 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.780548  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:24:44.781246  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:44.781934  4752 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.783914  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:44.784535  4681 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.783483  5701 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.789512  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:44.790161  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:44.790771  4675 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:44.791163  5616 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:44.790709  4598 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:44.792543  5701 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:44.794663  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:44.796572  5713 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:44.797324  5713 raft_consensus.cc:2955] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:44.798607  4756 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:44.799075  5676 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.792548  5649 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:44.800908  4603 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:44.801764  5744 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:44.804664  4681 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:24:44.806241  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:44.803860  4596 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:44.803166  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:44.807634  4756 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.808081  4675 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:24:44.810336  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:44.811177  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:44.813169  4756 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:44.803380  4752 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:24:44.813791  5312 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:24:44.828635  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:44.829766  5677 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:44.830242  5677 raft_consensus.cc:697] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:44.831138  5677 consensus_queue.cc:237] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.813862  5645 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:44.833881  5645 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:44.834766  5645 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:44.817828  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:44.817472  5348 raft_consensus.cc:1217] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:44.820958  4498 catalog_manager.cc:5697] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:44.822343  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:44.823351  4596 raft_consensus.cc:2955] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:44.824865  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:44.828713  5737 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:44.840929  4645 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:44.838900  5616 raft_consensus.cc:2764] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:24:44.819067  5676 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:44.845968  4598 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.862864  4675 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:44.866782  5737 raft_consensus.cc:697] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:24:44.866103  5740 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:44.867558  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:44.873482  4598 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:44.866909  5713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:42620 (request call id 1) took 1010 ms. Trace:
I20260709 12:24:44.868742  5737 consensus_queue.cc:237] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:44.866766  5744 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:44.879495  4681 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:44.887456  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:44.887540  5744 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.889703  5744 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:44.898583  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:44.902395  4598 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:44.879982  5701 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:44.908427  5701 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:44.906960  5744 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:44.906219  5727 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.880636  5713 rpcz_store.cc:276] 0709 12:24:43.861294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.871040 (+  9746us) service_pool.cc:224] Handling call
0709 12:24:44.018618 (+147578us) spinlock_profiling.cc:243] Waited 4.31 ms on lock 0x7b540033af20. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee433520 00007effee4bcb42 00007effee4bb800 00007efff347cd29 00007effec9fb1d4 00007effec9fb162 00007effec9fb091 00007effec9fb00c 00007effec9fa2c2 00007effec7f148c
0709 12:24:44.055929 (+ 37311us) log.cc:844] Serialized 499 byte log entry
0709 12:24:44.362002 (+306073us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b180000d000. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:24:44.390642 (+ 28640us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b180000d000. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007effeb3cba40 00007effeb3c64e4 00007effeb3c6455 00007effee40fb0b 00007effee494b63 00007effee4bc752 00007efff347cd29
0709 12:24:44.858570 (+467928us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82394,"raft.queue_time_us":145360,"raft.run_cpu_time_us":13677,"raft.run_wall_time_us":123082,"spinlock_wait_cycles":16215552,"thread_start_us":195847,"threads_started":1,"wal-append.queue_time_us":263636}
I20260709 12:24:44.931890  5701 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.922060  5744 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:44.947547  5681 ts_tablet_manager.cc:933] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.944741  5702 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.938468  5727 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.951848  5677 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.955289  5740 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:44.970932  4756 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:44.961053  5702 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.969614  5677 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:44.986675  4681 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:44.987097  5677 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:44.992272  4755 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:44.971307  5676 raft_consensus.cc:697] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:45.021229  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:44.984894  5736 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:24:44.983193  5744 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:45.037659  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:45.038597  4596 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:45.041433  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:45.042196  4681 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:45.043074  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:45.043464  4645 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:45.014767  5740 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:45.047220  5727 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:45.047359  5757 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.048282  5676 consensus_queue.cc:237] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:45.059113  5736 consensus_queue.cc:1243] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:45.115187  5616 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:45.047359  5758 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:45.130247  5764 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:45.130369  5616 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:45.120237  5675 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.131779  5675 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:45.132645  5616 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:45.130846  5758 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.116744  5681 tablet_copy_client.cc:323] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.195:41049
I20260709 12:24:45.134405  5758 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:45.128010  5706 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:45.136327  4766 tablet_copy_service.cc:140] P 49277a7653524787a715232ab908bbc6: Received BeginTabletCopySession request for tablet 28f87749718d445ba1615ff33028c4d8 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:42632)
I20260709 12:24:45.136904  4766 tablet_copy_service.cc:161] P 49277a7653524787a715232ab908bbc6: Beginning new tablet copy session on tablet 28f87749718d445ba1615ff33028c4d8 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:42632: session id = dbe960c35c84451faf4e5b5c9233fa39-28f87749718d445ba1615ff33028c4d8
I20260709 12:24:45.136746  5706 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:45.138785  5706 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:45.083175  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:45.145533  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:45.082041  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:45.081664  4675 consensus_queue.cc:237] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:45.092337  4498 catalog_manager.cc:5697] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
W20260709 12:24:45.085702  5755 log.cc:927] Time spent T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:24:45.061829  5702 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:45.066706  5649 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:45.195457  4749 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
W20260709 12:24:45.073123  5756 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:24:45.198102  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 153119us
I20260709 12:24:45.134100  5675 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:45.074901  5677 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.124s
I20260709 12:24:45.203686  5736 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:45.221637  5736 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:24:45.182407  5757 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:45.222647  5736 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:45.212122  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:45.212668  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:45.215505  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:45.226094  4599 raft_consensus.cc:2368] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:45.228870  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:45.229746  4599 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:45.231446  4645 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:45.196318  5649 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:45.262688  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:45.257269  5649 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:45.262972  5736 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:45.273468  4603 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:45.254683  5760 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:45.307863  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 85715.8us
I20260709 12:24:45.321719  4598 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:45.298149  5757 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:45.330444  4673 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:45.300606  5760 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.333400  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:45.313179  4766 tablet_copy_source_session.cc:215] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:45.345474  5737 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:45.344930  4674 raft_consensus.cc:2955] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:45.348471  4720 leader_election.cc:400] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:45.348801  4720 leader_election.cc:403] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:45.359283  5613 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:45.359759  5613 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.360111  5613 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:45.361505  5758 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:45.363488  4680 raft_consensus.cc:2368] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:45.334138  5760 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:45.370272  5701 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:45.381361  5767 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:45.382619  4676 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:45.386946  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:45.387591  4603 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:45.403489  4598 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:45.405068  5681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28f87749718d445ba1615ff33028c4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:45.406829  5761 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:45.410285  5758 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:45.419480  5613 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:45.421509  5649 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:45.425117  5613 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:45.427930  4500 catalog_manager.cc:5697] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:45.442068  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 107128us
I20260709 12:24:45.442806  4500 catalog_manager.cc:5697] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:45.446871  4500 catalog_manager.cc:5697] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:45.443698  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 110398us
I20260709 12:24:45.451279  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:45.454798  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 103901us
I20260709 12:24:45.455765  5348 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:24:45.459836  5348 pending_rounds.cc:85] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 4
I20260709 12:24:45.460186  5348 pending_rounds.cc:107] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:24:45.460462  5348 raft_consensus.cc:2889] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:45.461448  5760 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:45.466032  4680 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:45.466339  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:45.467013  5760 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:45.467168  4604 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 3: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:45.470010  4603 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:45.474028  5649 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:45.475171  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:45.478057  4674 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:45.479031  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:45.479962  4645 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:45.480110  4676 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:45.481249  5758 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:24:45.485466  4644 consensus_peers.cc:597] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53908897ce974579947af90acaf0aed7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:45.499814  4603 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:45.489130  5758 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:45.502027  5778 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:45.501967  5758 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:45.506253  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:45.506969  4749 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:45.508342  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:45.510308  4720 leader_election.cc:400] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:45.510723  4720 leader_election.cc:403] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:45.512732  5783 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:45.517621  4749 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:45.518443  5701 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:45.521286  5737 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:45.523914  4678 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:45.547290  4752 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:45.547834  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:45.548609  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:45.549271  4756 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:45.564962  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:45.565997  4599 raft_consensus.cc:2368] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:45.570367  4756 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:45.567261  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:45.577975  5778 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:45.577518  4720 leader_election.cc:400] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:45.585397  4720 leader_election.cc:403] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:45.553678  5649 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:24:45.598277  4675 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:45.601778  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:45.568503  5616 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:24:45.605189  5677 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:45.607093  5677 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:45.608119  5677 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:45.611155  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:45.601519  5765 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:45.658764  4756 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:45.659950  5681 tablet_copy_client.cc:806] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:45.598892  5744 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:24:45.681641  4599 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:45.667339  5783 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:45.695983  4754 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:45.671257  5681 tablet_copy_client.cc:670] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:45.671614  5705 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:24:45.695533  5778 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:24:45.787665  5759 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:24:46.062749  5814 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.074663  4596 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:46.075028  4756 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:46.068377  5814 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:46.109185  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:46.119789  5814 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.163796  5812 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:24:46.143203  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 419284us
W20260709 12:24:46.197742  4598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.002s	sys 0.027s
W20260709 12:24:46.210279  4598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.003s	sys 0.028s
W20260709 12:24:46.224123  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 82114.3us
I20260709 12:24:46.226372  5816 ts_tablet_manager.cc:933] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:46.216898  5761 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 12:24:46.231226  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:46.233731  4568 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.032s	sys 0.038s
W20260709 12:24:46.234328  4568 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.033s	sys 0.038s
I20260709 12:24:46.238881  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:46.240788  5819 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.241204  5805 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:46.241827  5805 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.242237  5805 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:46.242043  5819 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.260232  5819 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:46.269829  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:46.270498  5804 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.271359  5813 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:46.272091  5813 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.271564  5804 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.272547  5813 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:46.289059  4755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.005s	sys 0.039s
W20260709 12:24:46.291513  4755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.008s	sys 0.039s
I20260709 12:24:46.291270  5649 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:46.292333  5649 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.295785  5805 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.300252  5677 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.300945  5677 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.224436  5812 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.303560  5812 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.305581  5677 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:46.315771  4571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.011s	sys 0.043s
I20260709 12:24:46.298976  5804 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:46.298504  5761 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:24:46.321121  4571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.012s	sys 0.043s
I20260709 12:24:46.315872  5805 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.328387  5822 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:46.332028  4754 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:24:46.334756  5792 log.cc:927] Time spent T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.633s	user 0.003s	sys 0.010s
I20260709 12:24:46.338982  5822 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:46.340251  4498 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52808 (request call id 171) took 1389 ms. Trace:
I20260709 12:24:46.351629  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:46.340700  4498 rpcz_store.cc:276] 0709 12:24:44.950628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.950819 (+   191us) service_pool.cc:224] Handling call
0709 12:24:46.339997 (+1389178us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:45.198632 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:45.311695 (+113063us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:45.311734 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:24:45.313141 (+  1407us) tablet.cc:662] Decoding operations
0709 12:24:45.336173 (+ 23032us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:45.336232 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:45.336261 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:45.338824 (+  2563us) tablet.cc:701] Row locks acquired
0709 12:24:45.338853 (+    29us) write_op.cc:260] PREPARE: finished
0709 12:24:45.339010 (+   157us) write_op.cc:270] Start()
0709 12:24:45.343692 (+  4682us) write_op.cc:276] Timestamp: P: 1783599885338965 usec, L: 0
0709 12:24:45.352408 (+  8716us) op_driver.cc:348] REPLICATION: starting
0709 12:24:45.391651 (+ 39243us) log.cc:844] Serialized 594 byte log entry
0709 12:24:45.513614 (+121963us) op_driver.cc:464] REPLICATION: finished
0709 12:24:45.701614 (+188000us) write_op.cc:301] APPLY: starting
0709 12:24:45.703054 (+  1440us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:45.711329 (+  8275us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:45.739338 (+ 28009us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:45.782800 (+ 43462us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:45.782998 (+   198us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:45.783070 (+    72us) write_op.cc:312] APPLY: finished
0709 12:24:45.806744 (+ 23674us) log.cc:844] Serialized 22 byte log entry
0709 12:24:45.807028 (+   284us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:45.807143 (+   115us) write_op.cc:454] Released schema lock
0709 12:24:45.807198 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:45.807619 (+   421us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:45.807659 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":127875,"apply.run_cpu_time_us":2920,"apply.run_wall_time_us":111360,"num_ops":1,"prepare.queue_time_us":235,"prepare.run_cpu_time_us":3735,"prepare.run_wall_time_us":197944,"replication_time_us":158044,"thread_start_us":37648,"threads_started":1}]]}
I20260709 12:24:46.368585  5823 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.368872  5737 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.369553  5737 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:46.370664  5822 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:46.369527  5823 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.346057  5804 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.371268  4598 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:46.379568  5822 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.380343  5822 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.382411  5822 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.384280  5822 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.384637  5821 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.385066  5822 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.385309  5821 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.387632  5821 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.388213  5822 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.369736  5820 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.394270  5820 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.396375  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:46.369437  5649 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:46.374346  5761 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:24:46.498566  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 67815us
I20260709 12:24:46.505072  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:46.508677  4596 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:46.525097  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:46.506145  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:46.524942  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:46.506613  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:46.521275  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:46.506068  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:46.536088  5820 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.530158  5823 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.509550  5824 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.544327  5818 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.511807  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.562556  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.509857  5737 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.511453  5816 tablet_copy_client.cc:323] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.194:42599
I20260709 12:24:46.544862  5824 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.522562  5804 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.570735  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.571553  5823 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.576362  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:46.581033  5823 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.584008  5645 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.584234  5823 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.585215  5649 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.585765  5649 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.545140  5818 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:46.590914  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:46.584777  5645 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.604179  5824 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.608790  4691 tablet_copy_service.cc:140] P 3a5aeedd11fe49938079fe86cb01232c: Received BeginTabletCopySession request for tablet 53908897ce974579947af90acaf0aed7 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:48198)
I20260709 12:24:46.609115  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:46.609619  4691 tablet_copy_service.cc:161] P 3a5aeedd11fe49938079fe86cb01232c: Beginning new tablet copy session on tablet 53908897ce974579947af90acaf0aed7 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:48198: session id = dbe960c35c84451faf4e5b5c9233fa39-53908897ce974579947af90acaf0aed7
I20260709 12:24:46.593933  5759 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:24:46.615437  5799 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.879s	user 0.002s	sys 0.019s
I20260709 12:24:46.615803  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:46.614655  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.622661  5778 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:24:46.648172  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 52118.2us
I20260709 12:24:46.650415  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.655125  5761 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:46.656280  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:46.614910  5649 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:46.669934  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:46.670743  4678 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
W20260709 12:24:46.672094  5831 log.cc:927] Time spent T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 12:24:46.675144  5832 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:24:46.679047  5783 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.679215  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:46.679387  5804 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:46.680192  5818 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:46.680300  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:46.679780  5783 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.680840  4750 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:46.679894  4679 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:46.626271  5778 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.699601  5824 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.700412  5824 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.700661  5818 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:46.710790  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:46.730631  5781 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.730746  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.731420  5781 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.738040  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:46.743363  5783 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:46.727805  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:46.715186  5677 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:24:46.750238  4679 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:46.751099  5670 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:46.772840  5670 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.754702  4647 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:24:46.774983  5813 consensus_meta.cc:287] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.003s
I20260709 12:24:46.776106  4647 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:46.758522  5818 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:46.685937  5759 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:46.769094  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 54217.8us
I20260709 12:24:46.786588  5811 raft_consensus.cc:1064] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:24:46.748638  5645 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:46.800156  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:46.779557  5814 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:46.809742  5820 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:46.811741  5824 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.813737  4754 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:46.819310  5670 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.794025  5822 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:46.796909  5778 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.835465  5798 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:46.836833  5781 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.841850  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.842868  4600 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:46.849189  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:46.783905  5813 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:46.852382  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:46.854779  5811 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:46.857040  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.825461  5758 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 12:24:46.868194  4677 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:46.869107  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 940) took 1267 ms. Trace:
I20260709 12:24:46.869836  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.869560  4753 rpcz_store.cc:276] 0709 12:24:45.602022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.602353 (+   331us) service_pool.cc:224] Handling call
0709 12:24:45.602608 (+   255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.730726 (+128118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.730911 (+   185us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:46.756266 (+1025355us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b2c000d0108. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:24:46.756998 (+   732us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.821228 (+ 64230us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:46.821276 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.821371 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:46.868632 (+ 47261us) raft_consensus.cc:1685] finished
0709 12:24:46.868661 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.868910 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13644,"raft.queue_time_us":23607,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":5072,"spinlock_wait_cycles":7852800,"thread_start_us":174514,"threads_started":2,"wal-append.queue_time_us":109756}
I20260709 12:24:46.871052  4604 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:46.873376  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:46.874090  4678 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:46.876050  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:46.878739  4645 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:46.880319  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.880679  5814 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:46.882053  5759 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.883739  5813 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:46.884697  4598 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:46.943146  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 131387us
I20260709 12:24:46.942471  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.948005  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:46.953838  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 65808.8us
I20260709 12:24:46.954437  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.968446  5314 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:46.968605  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.965245  4691 tablet_copy_source_session.cc:215] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:46.977824  5761 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:24:46.980337  4750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 938) took 1433 ms (client timeout 1693 ms). Trace:
I20260709 12:24:46.978035  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:46.980842  4750 rpcz_store.cc:269] 0709 12:24:45.546960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.547545 (+   585us) service_pool.cc:224] Handling call
0709 12:24:46.980083 (+1432538us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400264654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:24:46.980245 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2537478528}
I20260709 12:24:46.982394  4756 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:46.984730  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.985988  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:46.989872  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 47394.6us
I20260709 12:24:46.988826  4501 catalog_manager.cc:5697] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:46.985985  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.986766  5681 tablet_copy_client.cc:538] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:46.998338  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.998562  5766 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:46.998806  4596 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:46.998914  4645 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:46.987870  5776 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:47.001924  5776 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:47.004418  5776 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:47.005798  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:47.007144  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:47.008208  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.988055  4500 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52812 (request call id 187) took 1626 ms. Trace:
I20260709 12:24:46.989038  5713 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:46.989619  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:46.991291  4501 catalog_manager.cc:5697] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 3 to 4, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 4 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.013505  5803 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:47.013765  5713 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:47.014282  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:47.016052  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:47.017905  5815 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:47.018128  4645 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.018391  5815 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:46.988942  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:47.018860  5815 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:47.019402  4645 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.020241  5816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53908897ce974579947af90acaf0aed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:47.020205  5814 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:47.020579  4645 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.021847  5840 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.023066  5713 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:47.023767  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:47.026710  5778 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.027186  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:47.028720  5803 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:47.029422  5713 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:24:47.030692  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 59600.9us
I20260709 12:24:47.029901  5803 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:47.030566  5761 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:47.032930  5645 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:24:47.035774  5761 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:47.038944  5706 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:47.039310  5778 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:47.039431  5706 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:47.039983  5778 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:47.040623  5706 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:47.030071  5713 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:47.043422  5840 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.048441  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:24:47.098783  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 68320.7us
W20260709 12:24:47.170596  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 197864us
I20260709 12:24:47.098678  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:47.048944  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:24:47.144636  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.170197  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 138716us
W20260709 12:24:47.168540  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 148008us
I20260709 12:24:47.182479  4645 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.188133  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.097446  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:47.191987  5760 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:47.195578  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:47.196826  4645 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.197890  4645 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.101922  5706 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:47.105683  5713 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:47.205725  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.209687  4645 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.210624  5677 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.210899  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:47.192004  5778 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:47.010618  4500 rpcz_store.cc:276] 0709 12:24:45.361897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.362062 (+   165us) service_pool.cc:224] Handling call
0709 12:24:45.394317 (+ 32255us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee420090 00007effee4bef3a 00007efff4274e9c 00007efff437304e 00007efff28342a4 00007efff2834232 00007efff2834161 00007efff28340dc 00007efff2833392 00007effec7f148c 00007effec7f08c6
0709 12:24:46.987847 (+1593530us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:45.466740 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:45.467361 (+   621us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:45.467394 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:24:45.467411 (+    17us) tablet.cc:662] Decoding operations
0709 12:24:45.468013 (+   602us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:45.468063 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:45.468102 (+    39us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:24:45.468447 (+   345us) tablet.cc:701] Row locks acquired
0709 12:24:45.468468 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:24:45.468625 (+   157us) write_op.cc:270] Start()
0709 12:24:45.468731 (+   106us) write_op.cc:276] Timestamp: P: 1783599885468574 usec, L: 0
0709 12:24:45.468756 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:24:45.469434 (+   678us) log.cc:844] Serialized 1307 byte log entry
0709 12:24:45.628383 (+158949us) op_driver.cc:464] REPLICATION: finished
0709 12:24:45.711581 (+ 83198us) write_op.cc:301] APPLY: starting
0709 12:24:45.717953 (+  6372us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:45.730801 (+ 12848us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:45.777984 (+ 47183us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:45.778543 (+   559us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:45.778734 (+   191us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:24:45.778780 (+    46us) write_op.cc:312] APPLY: finished
0709 12:24:45.779015 (+   235us) log.cc:844] Serialized 34 byte log entry
0709 12:24:45.779208 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:45.779305 (+    97us) write_op.cc:454] Released schema lock
0709 12:24:45.779346 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:45.779705 (+   359us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:45.779734 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":380333,"reported_tablets":6,"spinlock_wait_cycles":69673856,"child_traces":[["op",{"apply.queue_time_us":74473,"apply.run_cpu_time_us":2676,"apply.run_wall_time_us":74950,"num_ops":3,"prepare.queue_time_us":879,"prepare.run_cpu_time_us":3132,"prepare.run_wall_time_us":26565,"replication_time_us":159494,"thread_start_us":21066,"threads_started":1}]]}
W20260709 12:24:47.218362  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 145829us
I20260709 12:24:47.218905  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:47.219933  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:47.195158  5818 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.217125  5761 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:47.103713  5815 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:47.201333  5758 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.180033  5840 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:24:47.207429  5821 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.202515  5814 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.214807  5822 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:47.257745  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 43968us
I20260709 12:24:47.222054  5760 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.268939  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.272850  5701 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:47.273391  5701 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:47.273818  5701 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:47.276443  4756 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: [1.2-1.2]
I20260709 12:24:47.278050  5815 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:47.271675  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.216315  5820 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:24:47.316608  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:47.259635  5778 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:47.326398  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:24:47.319909  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 61078.2us
I20260709 12:24:47.321412  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:47.294106  5815 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.337056  5815 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.337157  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:24:47.338497  5848 log.cc:927] Time spent T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:24:47.367645  5701 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:47.373054  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.379156  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.355372  5776 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:47.388756  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:47.390349  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:47.390777  5348 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:47.407766  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:47.401485  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:47.411396  4596 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:47.413998  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:47.385136  5815 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.370059  5778 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:47.391500  5780 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:24:47.409430  5852 log.cc:927] Time spent T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:24:47.474222  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:47.439224  5701 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:47.439067  5780 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:47.482450  5767 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:47.483196  5767 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:47.484277  5645 consensus_queue.cc:1059] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:24:47.485842  5701 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.420593  5713 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:24:47.493809  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 93420.7us
W20260709 12:24:47.514802  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 106180us
W20260709 12:24:47.520226  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 46039.5us
W20260709 12:24:47.533030  5846 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:24:47.473593  5813 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:24:47.533205  4677 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:24:47.538719  5849 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.364s	user 0.001s	sys 0.006s
I20260709 12:24:47.536000  5767 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:47.540323  5713 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:47.475334  5776 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:47.480288  5780 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:47.545701  4596 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:47.548139  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:47.551771  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:47.552412  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:47.554343  5811 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:24:47.556434  5767 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.559306  5681 tablet_bootstrap.cc:492] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:47.561070  4680 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:24:47.576591  5859 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:24:47.563611  5776 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:47.576172  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.582329  5713 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:47.583787  5776 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:47.585404  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:47.586580  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:24:47.587194  5855 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.187s	user 0.000s	sys 0.004s
I20260709 12:24:47.587796  4603 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: [1.2-1.2]
I20260709 12:24:47.595156  5862 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:24:47.598305  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:47.601182  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:47.602061  5677 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:47.603641  5820 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:47.605311  5814 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.617005  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:47.626791  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:47.654433  5816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:50638 (request call id 147) took 1591 ms. Trace:
I20260709 12:24:47.654974  5816 rpcz_store.cc:276] 0709 12:24:46.062931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.063377 (+   446us) service_pool.cc:224] Handling call
0709 12:24:46.226896 (+163519us) ts_tablet_manager.cc:934] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
0709 12:24:47.018723 (+791827us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:47.111423 (+ 92700us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:47.111611 (+   188us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:47.557316 (+445705us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:47.586720 (+ 29404us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:47.654236 (+ 67516us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:47.654327 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":196749,"tablet-copy.queue_time_us":92005,"thread_start_us":133843,"threads_started":2}
I20260709 12:24:47.656730  5816 tablet_copy_client.cc:806] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:47.614256  5767 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:47.657351  5816 tablet_copy_client.cc:670] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:47.686512  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:47.842607  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 347328us
I20260709 12:24:47.668036  5701 raft_consensus.cc:3055] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:47.663797  5778 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:47.646016  5820 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:47.929008  5713 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:47.913861  5821 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:47.895195  5814 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:47.817065  5767 raft_consensus.cc:697] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:48.001875  4645 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:47.649755  5677 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:47.664947  5818 consensus_queue.cc:1059] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.324s
I20260709 12:24:47.775619  5780 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:47.983984  5701 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:24:48.071579  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 69798.3us
I20260709 12:24:48.004468  5840 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:48.101987  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.102867  4673 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:48.072790  5677 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.003669  5821 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.093171  5812 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.003561  5713 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.115996  5740 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:48.004010  5814 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.116788  5820 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.116622  5740 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:48.118746  5814 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.120460  5701 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:48.122284  5878 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:24:48.126070  5767 consensus_queue.cc:237] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.127959  5821 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.130945  5780 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:24:48.142669  5838 log.cc:927] Time spent T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.751s	user 0.009s	sys 0.000s
W20260709 12:24:48.144610  5875 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:24:48.100013  5840 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:48.152096  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 353339us
I20260709 12:24:48.188654  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.199518  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.203598  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:48.145673  5821 consensus_queue.cc:1059] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:24:48.178355  5812 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:48.185134  5776 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.209076  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.178040  5760 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:48.213994  4596 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:48.217558  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.218235  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:48.223644  4605 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:48.243077  5883 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.245687  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.256207  5882 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.257792  5867 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.257261  5882 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.214211  5812 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.242334  5677 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.274684  5677 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:48.275321  5677 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.237447  5780 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.278048  5780 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.225498  5776 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.222239  5760 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.244562  5821 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.284055  5822 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.294076  5780 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.243765  5883 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.310313  5883 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.316305  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.301492  5706 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.322512  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:48.305534  5744 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.324939  4748 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:48.326128  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.258404  5867 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.330003  5880 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.330817  5880 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.334649  5867 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.304996  5821 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.352562  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.354099  5887 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.354893  5887 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.358422  4755 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:24:48.358795  4755 pending_rounds.cc:85] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 1
I20260709 12:24:48.306191  5822 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.327582  5883 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.368320  5776 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.369040  5879 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.371002  5677 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.359025  4755 pending_rounds.cc:107] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:24:48.386185  4755 raft_consensus.cc:2889] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:48.358202  5740 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.390287  4750 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:48.341629  5706 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.377466  5766 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.339895  5744 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.373734  5882 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.388092  5780 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.409787  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:48.408519  5877 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.417459  4750 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:48.394256  5880 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.395089  5879 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.386853  5883 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.426754  5816 tablet_copy_client.cc:538] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:48.447110  5879 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.447932  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.416792  5877 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:48.454298  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.443143  5812 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.420077  5780 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:48.460887  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.460675  5880 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.462754  5880 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.464903  5880 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.410216  5766 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:48.473305  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.469779  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.474416  4647 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.470472  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.480908  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.449280  5897 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.483582  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.481675  4647 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.479157  5818 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.470328  5677 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.471451  5822 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.469681  5883 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.472610  5887 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.492774  5767 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.510061  5767 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.512791  5767 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.514093  5889 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.520814  5877 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.500607  5880 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.522001  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.521633  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.525060  5744 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.526574  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.531978  5780 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:48.533198  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.521426  5877 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.517129  5706 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.500216  5897 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.537673  4755 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:48.537448  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:48.545401  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.548321  5766 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.545042  5713 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.534946  5880 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.560566  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.539336  5877 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:48.564544  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.568274  5897 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.569440  5877 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.570029  5877 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.570178  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.570775  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.571485  4680 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 1.
I20260709 12:24:48.572011  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.572922  5887 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.572992  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.575142  5877 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.575632  5877 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.576084  5877 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.580184  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:48.581373  5887 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.581857  5887 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.582297  5887 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.584911  5883 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:24:48.560109  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 146229us
I20260709 12:24:48.567070  5767 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:48.585474  5883 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.587649  5882 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.605401  5821 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.614423  5758 raft_consensus.cc:3055] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:48.614900  5758 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:48.613278  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.617849  4750 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 1.
W20260709 12:24:48.619942  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 127875us
I20260709 12:24:48.615820  5758 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:48.625380  5906 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:48.625975  5906 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.628769  5906 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.630838  5906 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:48.632602  5314 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.634519  5903 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.634917  5903 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.635258  5903 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.645493  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.651824  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.656786  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.660868  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.650787  5706 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.662384  5883 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.662379  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.646507  5758 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.666947  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.668479  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:48.668852  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.670342  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.670367  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.669708  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.672699  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.654806  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.674147  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:48.676074  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.677631  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.678083  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.680501  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.687901  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:48.692365  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.679484  5880 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.679731  5811 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.697754  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.697913  5901 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:24:48.708565  5782 consensus_queue.cc:1059] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:24:48.710353  5782 raft_consensus.cc:3055] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:48.710711  5782 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:48.717267  5865 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:48.699409  5670 consensus_queue.cc:1059] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:24:48.726859  5813 raft_consensus.cc:3055] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:48.727380  5813 raft_consensus.cc:740] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:48.727870  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.728605  4647 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.730370  5821 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.730835  5821 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.731316  5821 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.761673  5909 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.766884  5909 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.770143  5701 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:48.754747  5903 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:48.774482  5347 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:48.778298  5909 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.758311  5782 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.783972  5782 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.786018  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:48.787390  5759 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.787914  5759 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:48.788365  5759 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.790227  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.792094  5909 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.789019  5816 tablet_bootstrap.cc:492] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:24:48.795555  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.797607  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.797747  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:48.800118  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.793814  5889 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.802078  5872 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.804710  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.807363  5890 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.807665  5903 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.802201  5889 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.810336  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.808064  5890 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.806749  5740 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.809973  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.811877  5740 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.813683  4597 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 3: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:48.812944  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:48.814174  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 642002us
I20260709 12:24:48.814528  4753 raft_consensus.cc:2410] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:48.814767  5740 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.815308  5814 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.813524  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:48.815828  5814 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.826049  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:48.835986  5840 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.838766  5814 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.842329  4751 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: [1.2-1.2]
I20260709 12:24:48.836081  5903 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:48.843338  5903 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:48.841199  5889 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.843576  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.845820  5903 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.847810  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:48.848940  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.849519  4756 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:48.850390  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.851460  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.851612  4501 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52818 (request call id 180) took 3526 ms. Trace:
I20260709 12:24:48.852078  4754 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:48.853789  4499 catalog_manager.cc:5697] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 1 to 3, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 3 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:48.845131  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.846535  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.857829  5778 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.858636  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.861320  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:48.858418  5778 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.864173  4647 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:48.868957  4601 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:48.870059  5740 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.870462  5740 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.870824  5740 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.871773  5312 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.872582  5903 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.852017  4501 rpcz_store.cc:276] 0709 12:24:45.325536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.325681 (+   145us) service_pool.cc:224] Handling call
0709 12:24:48.851530 (+3525849us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:47.279743 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:47.280457 (+   714us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:47.280520 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:24:47.280542 (+    22us) tablet.cc:662] Decoding operations
0709 12:24:47.281075 (+   533us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:47.281156 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:47.281195 (+    39us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:24:47.320865 (+ 39670us) tablet.cc:701] Row locks acquired
0709 12:24:47.346472 (+ 25607us) write_op.cc:260] PREPARE: finished
0709 12:24:47.353628 (+  7156us) write_op.cc:270] Start()
0709 12:24:47.353847 (+   219us) write_op.cc:276] Timestamp: P: 1783599887353498 usec, L: 0
0709 12:24:47.353897 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 12:24:47.405736 (+ 51839us) log.cc:844] Serialized 917 byte log entry
0709 12:24:48.368259 (+962523us) op_driver.cc:464] REPLICATION: finished
0709 12:24:48.586841 (+218582us) write_op.cc:301] APPLY: starting
0709 12:24:48.586971 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:48.589553 (+  2582us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:48.602258 (+ 12705us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:48.603537 (+  1279us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:48.603852 (+   315us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:24:48.603909 (+    57us) write_op.cc:312] APPLY: finished
0709 12:24:48.604267 (+   358us) log.cc:844] Serialized 28 byte log entry
0709 12:24:48.604505 (+   238us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:48.604614 (+   109us) write_op.cc:454] Released schema lock
0709 12:24:48.604679 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:48.605113 (+   434us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:48.605147 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":707669,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":190433,"apply.run_cpu_time_us":3006,"apply.run_wall_time_us":38402,"num_ops":2,"prepare.queue_time_us":166282,"prepare.run_cpu_time_us":35199,"prepare.run_wall_time_us":653007,"raft.queue_time_us":179301,"raft.run_cpu_time_us":15754,"raft.run_wall_time_us":128348,"replication_time_us":1014202,"thread_start_us":548588,"threads_started":4,"wal-append.queue_time_us":593945}]]}
I20260709 12:24:48.867498  4647 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:48.812536  5887 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:48.882095  5758 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.883148  5701 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:24:48.899202  5759 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.900053  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.900739  4674 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
W20260709 12:24:48.906051  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.902937  5821 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.906677  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.907920  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.894042  5889 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.908545  4674 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:48.908586  5889 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.909210  5887 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:48.911731  5740 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.915573  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.916249  4678 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:48.918533  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:48.920874  4756 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 12:24:48.920126  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.921926  4678 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:48.923372  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.925410  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.925590  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.926110  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:48.926852  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:48.923494  4571 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:48.903743  5758 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.925405  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.930928  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.931216  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.931941  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.909560  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.890399  5877 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:48.934784  5767 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:48.935158  5812 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.935974  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.936776  4645 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:48.937773  5681 tablet_bootstrap.cc:492] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:48.945102  5887 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.939522  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.933866  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:48.946524  5760 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:48.966713  5767 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:48.998912  5767 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:48.949182  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.943166  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.953907  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.949335  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:49.008136  5914 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:24:49.009243  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:49.006579  4677 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:48.964442  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:49.013432  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:48.989008  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:48.927924  5744 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:48.970932  5886 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:48.938205  5813 consensus_queue.cc:260] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:48.932322  5758 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.016757  4676 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:49.040268  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:24:48.990195  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57379us
I20260709 12:24:48.995532  5681 tablet_bootstrap.cc:492] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:49.042331  5681 ts_tablet_manager.cc:1403] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 1.484s	user 0.057s	sys 0.045s
I20260709 12:24:49.043118  4645 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.044281  4645 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.045447  5681 raft_consensus.cc:359] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:49.046288  5681 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:49.047025  5760 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.046960  5681 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:49.047698  5760 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.995414  5818 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.989884  5812 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:49.049527  5812 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.051736  5812 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:49.052332  4672 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:48.957118  4719 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:49.019140  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:49.055369  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 98383.3us
I20260709 12:24:49.057694  5812 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:49.059053  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:49.060953  5778 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:49.061492  5818 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:49.064067  5818 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.066174  5818 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:49.068630  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:49.069476  5890 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:49.071880  5349 raft_consensus.cc:1217] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:49.063402  5681 ts_tablet_manager.cc:1434] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.021s	user 0.002s	sys 0.005s
I20260709 12:24:49.076853  4766 tablet_copy_service.cc:342] P 49277a7653524787a715232ab908bbc6: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-28f87749718d445ba1615ff33028c4d8 received from {username='slave'} at 127.0.0.1:42632
I20260709 12:24:49.077349  4766 tablet_copy_service.cc:434] P 49277a7653524787a715232ab908bbc6: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-28f87749718d445ba1615ff33028c4d8 on tablet 28f87749718d445ba1615ff33028c4d8 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:49.077306  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.021863  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:49.022701  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:49.024672  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:49.017629  5903 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:49.029673  5346 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:48.999709  5889 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:49.000767  5906 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:24:49.035295  5877 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:49.036779  5740 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:49.092922  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:49.045326  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.086030  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:49.086716  4680 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:49.062656  5645 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.090409  5903 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:49.090910  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:49.091204  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:49.095511  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:49.091305  5737 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.090902  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.092106  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:49.092182  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:49.022120  5744 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:49.100359  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:24:49.096973  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 51716.1us
I20260709 12:24:49.058261  5812 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.101548  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.101272  5889 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:49.108426  5761 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:49.102865  5744 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:49.108991  4645 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.106920  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.112907  5903 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:49.118011  4645 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:24:49.118131  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.119277  5909 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.121290  5725 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:49.126092  5761 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:49.124202  4647 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.127283  5761 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.127646  5758 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.117079  5902 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.130093  4647 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.130967  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.121848  5778 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.196516  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:49.197710  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.208869  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 78825.9us
I20260709 12:24:49.212782  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.226349  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:49.126237  5725 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:49.231181  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:49.242635  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:49.210908  5740 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.241755  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:49.247308  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 116411us
I20260709 12:24:49.241434  5820 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:49.231187  5818 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.228785  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:49.240155  5761 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:49.252270  5804 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:49.249924  5890 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:49.254364  4674 raft_consensus.cc:2393] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:49.246963  4676 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 2. (term mismatch)
I20260709 12:24:49.241693  5778 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:49.265542  4645 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.267127  4676 pending_rounds.cc:85] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 1
I20260709 12:24:49.269989  5818 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.267302  5804 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:49.258608  5820 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:49.279665  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:49.278659  5840 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:49.266127  5761 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.279763  5804 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.269637  5778 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:49.269244  5890 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:49.276752  4676 pending_rounds.cc:107] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.2
I20260709 12:24:49.293926  5840 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.304787  4675 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:49.306443  4675 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:49.301666  5821 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:49.310544  5758 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:49.301098  4676 raft_consensus.cc:2917] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Aborting config change with OpId 1.2: Aborted: Op aborted by new leader
I20260709 12:24:49.313175  5725 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:49.316768  4676 raft_consensus.cc:2889] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:49.323791  5761 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:49.325901  5890 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:49.319084  5767 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.352541  5804 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.355967  5820 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:49.357914  4679 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:49.334233  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:48182 (request call id 6) took 4293 ms. Trace:
I20260709 12:24:49.383005  5884 raft_consensus.cc:3055] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:49.362308  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:49.363034  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:49.358646  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.363986  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:49.366257  5759 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:49.408860  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 50320.3us
I20260709 12:24:49.368355  5858 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:49.373189  5349 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:49.373708  5778 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:49.374895  5767 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:49.376696  4645 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:24:49.429268  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 52649.2us
I20260709 12:24:49.381109  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:49.382019  5906 consensus_queue.cc:1059] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:49.383081  4753 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:49.385289  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:49.369074  5889 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:49.398164  4569 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:49.400521  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:49.361085  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:49.400875  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:24:49.406270  5917 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:24:49.406739  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:49.375546  5840 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:49.430317  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.432403  4752 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:49.446303  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 48232.3us
I20260709 12:24:49.437950  4681 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:49.401808  5884 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:49.440016  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.440591  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:49.417281  5903 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.400733  4676 rpcz_store.cc:276] 0709 12:24:45.040774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.043291 (+  2517us) service_pool.cc:224] Handling call
0709 12:24:45.080756 (+ 37465us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b5800189bd0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4120aa 00007effee494447 00007effee4b170a 00007effee4b276a 00007effee4b191b 00007effee4bd03c 00007effee4bc6b3 00007efff347cd29 00007effec9fb1d4 00007effec9fb162
0709 12:24:45.196557 (+115801us) log.cc:844] Serialized 499 byte log entry
0709 12:24:45.480537 (+283980us) spinlock_profiling.cc:243] Waited 89.9 ms on lock 0x7b54002d8fa0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee436b26 00007effee43049a 00007effee42f714 00007effee40feaa 00007effee416d53 00007effee416cd9 00007effee416c69 00007effee416c31 00007effee415dfd 0000000000388634
0709 12:24:45.563223 (+ 82686us) spinlock_profiling.cc:243] Waited 82.3 ms on lock 0x7b54002d8fa0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee42bdbe 00007effee436930 00007effee4368d8 00007effee436858 00007effee436813 00007effee435a9c 00007efff43212f6 00007efff431baa8 00007effee4727cb 00007effee4752c0
0709 12:24:46.282312 (+719089us) spinlock_profiling.cc:243] Waited 91.7 ms on lock 0x7b54002d8fa0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee42f142 00007effee40feaa 00007effee416d53 00007effee416cd9 00007effee416c69 00007effee416c31 00007effee415dfd 0000000000388634 0000000000388479 00007effeb3c9475
0709 12:24:46.545268 (+262956us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b1800269200. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3972a6 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee436b26 00007effee43049a 00007effee42f714
0709 12:24:46.615350 (+ 70082us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b54002d8fa0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee436b26 00007effee43049a 00007effee42f714 00007effee40feaa 00007effee416d53 00007effee416cd9 00007effee416c69 00007effee416c31 00007effee415dfd 0000000000388634
0709 12:24:49.334110 (+2718760us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":275,"mutex_wait_us":13685,"raft.queue_time_us":9102,"raft.run_cpu_time_us":54773,"raft.run_wall_time_us":2368595,"spinlock_wait_cycles":642243072,"thread_start_us":108522,"threads_started":1,"wal-append.queue_time_us":124039,"wal-append.run_cpu_time_us":5995,"wal-append.run_wall_time_us":1249875}
I20260709 12:24:49.436342  5759 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.409442  5813 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.438061  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:49.447089  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.464604  5879 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.468498  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:49.467599  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:49.461898  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.472311  4720 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.465906  5759 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.489439  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.448958  5889 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:49.492066  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.493150  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:49.435580  5744 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.497068  4720 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.500907  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:49.501267  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.474239  5890 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.506709  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.489955  5879 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:49.500399  5880 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:49.511891  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.502610  5767 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.509177  5897 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.495667  5885 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:24:49.452210  5858 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:24:49.493077  5776 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.525585  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:49.512495  5872 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.515043  5763 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:49.528124  5812 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:49.510524  5879 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:49.529868  5879 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.532403  5725 raft_consensus.cc:3055] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:49.532829  5725 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:49.515134  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:49.513800  4756 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:49.533911  5725 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:49.513787  5887 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:49.535151  5887 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:49.535566  5887 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:49.518569  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:49.519012  5670 consensus_queue.cc:1059] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:49.496001  5645 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:49.538810  5645 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:49.539400  5645 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.532742  4647 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.518776  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.545943  5880 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.546386  5885 raft_consensus.cc:3055] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:49.546816  5885 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:49.546953  5804 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:49.547835  5885 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.549204  5885 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.550318  5860 raft_consensus.cc:3055] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:49.550822  5860 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:49.551908  5860 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.554638  5744 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:49.555286  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:49.558159  5812 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.558429  5860 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.560909  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.562175  5773 raft_consensus.cc:3055] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:49.563287  5761 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:49.563562  4550 auto_rebalancer.cc:629] Failed to schedule move for tablet 53908897ce974579947af90acaf0aed7: Aborted: Op aborted by new leader
I20260709 12:24:49.562623  5773 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:49.578294  5773 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:49.581660  5725 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.583681  5645 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.562124  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:49.572276  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:49.574858  5884 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:49.591777  4720 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.567387  5879 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.593281  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.594110  4647 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.596004  4647 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:49.595957  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:24:49.655226  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 59321.8us
I20260709 12:24:49.657886  5668 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.594729  5880 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:49.660382  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 64502.5us
I20260709 12:24:49.599998  5812 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:49.595602  5773 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.653244  4677 raft_consensus.cc:2955] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:49.665977  4647 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.668912  5814 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.654052  5776 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:49.676148  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 126875us
I20260709 12:24:49.653728  5840 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:49.597236  5857 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:24:49.662850  5897 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:49.659823  5761 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.686414  5897 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:49.693231  5645 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:49.701037  5897 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.710352  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.711882  5926 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.711872  5879 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:49.740672  5805 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:49.740967  5860 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:49.741264  5805 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:49.740391  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.744561  5782 raft_consensus.cc:3055] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:49.745131  5782 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:49.746300  5782 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.748296  5782 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.751324  5858 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:49.746775  4597 consensus_queue.cc:237] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:49.751927  5858 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:49.716117  5744 consensus_queue.cc:1059] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:24:49.755273  5744 raft_consensus.cc:3055] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:49.755681  5744 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:49.756681  5744 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.759946  5816 tablet_bootstrap.cc:492] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:24:49.760231  5887 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.761070  5816 tablet_bootstrap.cc:492] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:49.762095  5858 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.764823  5884 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.765273  5887 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:49.766629  5858 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.768393  5926 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.746690  5805 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:49.788376  5701 raft_consensus.cc:1064] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
W20260709 12:24:49.807638  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 64525.4us
I20260709 12:24:49.810735  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:49.815059  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 98535.3us
I20260709 12:24:49.817065  5930 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:49.814651  4500 catalog_manager.cc:5697] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:49.821738  4500 catalog_manager.cc:5697] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:49.827262  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:49.827558  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:49.828238  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.829335  5767 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.830195  5744 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.831982  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:24:49.835549  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 55865.4us
I20260709 12:24:49.835901  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:49.837456  5927 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:49.839443  5927 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:49.839797  5927 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:49.840134  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:49.840566  5927 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.841178  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:49.842142  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:49.842919  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:49.838158  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:49.846159  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:49.846819  5826 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:49.854671  5826 raft_consensus.cc:3055] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:49.854866  4756 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:49.855142  5826 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:49.856318  5826 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.849013  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:49.859768  5886 consensus_queue.cc:1059] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:49.872385  4499 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 72) took 4456 ms. Trace:
I20260709 12:24:49.876323  5927 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:49.879379  5776 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.822767  5816 ts_tablet_manager.cc:1403] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 1.021s	user 0.025s	sys 0.033s
I20260709 12:24:49.882833  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:49.885016  4677 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:49.818552  5701 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:49.883385  5826 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.882870  5886 raft_consensus.cc:3055] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:49.896521  5886 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:49.780970  5897 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.898967  5897 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:49.907439  5670 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:49.905606  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 45096.5us
I20260709 12:24:49.920338  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.872804  4499 rpcz_store.cc:276] 0709 12:24:45.416299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.416442 (+   143us) service_pool.cc:224] Handling call
0709 12:24:49.872299 (+4455857us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:48.960667 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:49.348539 (+387872us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b5c0012638c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff46a8331 00007efff46a8221 00007efff46a2f6c 00007efff1f00d69 00007efff1eec0a6 00007efff1eebd58 00007efff1ef09f1 00007efff1ef09a9 00007efff1ef0939 00007efff1ef0901 00007efff1eefb2d
0709 12:24:49.348601 (+    62us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:49.348631 (+    30us) write_op.cc:435] Acquired schema lock
0709 12:24:49.348649 (+    18us) tablet.cc:662] Decoding operations
0709 12:24:49.348953 (+   304us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:49.349024 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:49.349048 (+    24us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:49.349215 (+   167us) tablet.cc:701] Row locks acquired
0709 12:24:49.349227 (+    12us) write_op.cc:260] PREPARE: finished
0709 12:24:49.349376 (+   149us) write_op.cc:270] Start()
0709 12:24:49.350040 (+   664us) write_op.cc:276] Timestamp: P: 1783599889349302 usec, L: 0
0709 12:24:49.350071 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:24:49.351357 (+  1286us) log.cc:844] Serialized 532 byte log entry
0709 12:24:49.466173 (+114816us) op_driver.cc:464] REPLICATION: finished
0709 12:24:49.695053 (+228880us) write_op.cc:301] APPLY: starting
0709 12:24:49.697376 (+  2323us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:49.701181 (+  3805us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:49.703147 (+  1966us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:49.715363 (+ 12216us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:49.733685 (+ 18322us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:49.739370 (+  5685us) write_op.cc:312] APPLY: finished
0709 12:24:49.754698 (+ 15328us) log.cc:844] Serialized 22 byte log entry
0709 12:24:49.754905 (+   207us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:49.754995 (+    90us) write_op.cc:454] Released schema lock
0709 12:24:49.755035 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:49.755934 (+   899us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:49.755966 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":101832,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":220373,"apply.run_cpu_time_us":2039,"apply.run_wall_time_us":66349,"num_ops":1,"prepare.queue_time_us":88506,"prepare.run_cpu_time_us":20101,"prepare.run_wall_time_us":428355,"replication_time_us":115938,"spinlock_wait_cycles":44478848,"thread_start_us":186128,"threads_started":2}]]}
I20260709 12:24:49.928792  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.930096  5767 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:49.930547  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:49.932871  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.931370  5897 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.936026  5897 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.936969  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.937814  5930 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:49.938205  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:49.939890  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:49.940176  5897 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:49.915242  5886 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:49.962602  5886 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.966571  4498 catalog_manager.cc:5697] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 1 to 3, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 3 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:49.951242  5767 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:24:49.979095  5767 raft_consensus.cc:3055] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:49.979499  5767 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:49.980469  5767 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:49.981129  4568 consensus_peers.cc:597] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34a0b6acf7c6497e8ef8ffd63b4127d6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:49.981974  5767 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:49.984725  4679 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:49.986996  4756 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:49.987672  4748 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:49.995507  4748 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:49.996436  5925 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:50.006228  4748 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:50.007032  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:50.010707  4498 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52808 (request call id 173) took 1823 ms. Trace:
I20260709 12:24:50.007872  5744 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.011152  4498 rpcz_store.cc:276] 0709 12:24:48.187741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.200933 (+ 13192us) service_pool.cc:224] Handling call
0709 12:24:49.978138 (+1777205us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:49.978165 (+    27us) write_op.cc:454] Released schema lock
0709 12:24:50.010643 (+ 32478us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:49.971036 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:49.971621 (+   585us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:49.971660 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:24:49.971681 (+    21us) tablet.cc:662] Decoding operations
0709 12:24:49.972048 (+   367us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:49.972105 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:49.972140 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:49.972327 (+   187us) tablet.cc:701] Row locks acquired
0709 12:24:49.972346 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:24:49.972524 (+   178us) write_op.cc:270] Start()
0709 12:24:49.972652 (+   128us) write_op.cc:276] Timestamp: P: 1783599889972475 usec, L: 0
0709 12:24:49.972673 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:24:49.973366 (+   693us) log.cc:844] Serialized 537 byte log entry
0709 12:24:49.975522 (+  2156us) op_driver.cc:464] REPLICATION: finished
0709 12:24:49.975962 (+   440us) write_op.cc:301] APPLY: starting
0709 12:24:49.976043 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:49.976145 (+   102us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:49.976236 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:49.976485 (+   249us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:49.976639 (+   154us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:49.976684 (+    45us) write_op.cc:312] APPLY: finished
0709 12:24:49.976881 (+   197us) log.cc:844] Serialized 22 byte log entry
0709 12:24:49.977040 (+   159us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:49.977121 (+    81us) write_op.cc:454] Released schema lock
0709 12:24:49.977160 (+    39us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":418492,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":229,"apply.run_cpu_time_us":1446,"apply.run_wall_time_us":1446,"num_ops":1,"prepare.queue_time_us":106,"prepare.run_cpu_time_us":2635,"prepare.run_wall_time_us":2636,"replication_time_us":2742}]]}
I20260709 12:24:50.020138  4500 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52812 (request call id 188) took 2466 ms. Trace:
I20260709 12:24:50.022454  4500 rpcz_store.cc:276] 0709 12:24:47.553679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.554005 (+   326us) service_pool.cc:224] Handling call
0709 12:24:48.153276 (+599271us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee420090 00007effee4bef3a 00007efff4274e9c 00007efff437304e 00007efff28342a4 00007efff2834232 00007efff2834161 00007efff28340dc 00007efff2833392 00007effec7f148c 00007effec7f08c6
0709 12:24:50.020071 (+1866795us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:49.869156 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:49.869727 (+   571us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:49.869769 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:24:49.869784 (+    15us) tablet.cc:662] Decoding operations
0709 12:24:49.870215 (+   431us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:49.870262 (+    47us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:49.870292 (+    30us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:24:49.870599 (+   307us) tablet.cc:701] Row locks acquired
0709 12:24:49.870617 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:24:49.870749 (+   132us) write_op.cc:270] Start()
0709 12:24:49.870871 (+   122us) write_op.cc:276] Timestamp: P: 1783599889870710 usec, L: 0
0709 12:24:49.870889 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:24:49.871486 (+   597us) log.cc:844] Serialized 918 byte log entry
0709 12:24:49.873650 (+  2164us) op_driver.cc:464] REPLICATION: finished
0709 12:24:49.874199 (+   549us) write_op.cc:301] APPLY: starting
0709 12:24:49.874331 (+   132us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:49.874507 (+   176us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:49.874669 (+   162us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:49.875293 (+   624us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:49.875467 (+   174us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:24:49.875521 (+    54us) write_op.cc:312] APPLY: finished
0709 12:24:49.875720 (+   199us) log.cc:844] Serialized 28 byte log entry
0709 12:24:49.875887 (+   167us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:49.875995 (+   108us) write_op.cc:454] Released schema lock
0709 12:24:49.876040 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:49.876393 (+   353us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:49.876423 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":94022,"reported_tablets":11,"spinlock_wait_cycles":846314240,"child_traces":[["op",{"apply.queue_time_us":281,"apply.run_cpu_time_us":1914,"apply.run_wall_time_us":2147,"num_ops":2,"prepare.queue_time_us":774,"prepare.run_cpu_time_us":2597,"prepare.run_wall_time_us":2598,"replication_time_us":2611}]]}
I20260709 12:24:50.016008  5776 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:50.022902  5858 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:24:50.018250  5927 consensus_queue.cc:1059] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 12:24:50.033445  5857 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:24:50.032429  5744 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.031383  5910 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:24:50.050679  5744 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:50.050580  5816 raft_consensus.cc:359] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.052215  5816 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:50.057413  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:50.058419  4677 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:50.060065  5346 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:24:50.060935  5931 log.cc:927] Time spent T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:24:50.053025  5816 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:50.066880  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:50.064268  5744 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:24:50.064271  5805 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:50.170010  5805 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:50.131434  5925 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:50.171090  5925 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:50.173754  4602 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
W20260709 12:24:50.131224  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 69682.2us
I20260709 12:24:50.195113  5816 ts_tablet_manager.cc:1434] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.304s	user 0.007s	sys 0.004s
I20260709 12:24:50.195596  5872 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:50.197410  5773 raft_consensus.cc:3055] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:50.201805  4603 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:50.202725  4499 catalog_manager.cc:5697] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.203545  5773 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:50.222090  5701 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.243511  4691 tablet_copy_service.cc:342] P 3a5aeedd11fe49938079fe86cb01232c: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-53908897ce974579947af90acaf0aed7 received from {username='slave'} at 127.0.0.1:48198
I20260709 12:24:50.247219  4691 tablet_copy_service.cc:434] P 3a5aeedd11fe49938079fe86cb01232c: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-53908897ce974579947af90acaf0aed7 on tablet 53908897ce974579947af90acaf0aed7 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:50.251266  5773 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.230065  5925 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.253026  5773 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:50.253756  5925 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:50.238667  5701 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.263216  5805 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.264835  5805 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:50.295321  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:50.293737  5928 raft_consensus.cc:2955] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:50.296681  5767 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:50.297091  5767 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.297421  5767 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:50.297735  5804 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:50.304060  5701 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:50.304324  5767 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.307451  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:50.311213  5886 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.311430  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:50.311717  5886 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.313053  5767 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:50.314872  5886 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:50.316023  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:50.316672  4597 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:50.320130  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:50.314899  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:50.320731  4756 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:50.324915  4571 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:50.328859  5857 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:50.329267  5857 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.329667  5857 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:50.334082  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:50.347188  4501 catalog_manager.cc:5697] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:50.352129  5887 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:50.352653  5887 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.353113  5887 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:50.365818  5804 raft_consensus.cc:2955] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:50.371253  4679 raft_consensus.cc:2955] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:50.377987  5887 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.382146  5857 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.384621  5857 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:50.385656  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:50.386423  4756 raft_consensus.cc:2393] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:50.388788  4675 raft_consensus.cc:2955] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:50.397907  5744 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.405730  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:50.405857  5744 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.407776  5887 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:50.408234  5744 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:50.409418  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:50.410722  4603 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:50.411051  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:50.411743  4600 raft_consensus.cc:2393] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 3: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:50.418839  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:50.419143  4569 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:50.424145  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:50.436163  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:50.437126  4605 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:50.437860  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:50.434173  4499 catalog_manager.cc:5697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:50.442569  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:50.444351  5872 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:50.444815  5872 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.445204  5872 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:50.443229  5767 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:50.452226  5767 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.454351  5767 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:50.443467  5930 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:50.440339  5858 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:50.458679  5346 raft_consensus.cc:2955] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:50.460219  4756 raft_consensus.cc:2955] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:24:50.470530  4603 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
W20260709 12:24:50.488539  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 46043.8us
I20260709 12:24:50.491644  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:50.492008  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:50.492722  4605 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:50.495082  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:50.495374  4677 raft_consensus.cc:2393] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:50.514413  5930 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:50.515056  5930 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:50.516136  5930 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.523452  5816 ts_tablet_manager.cc:933] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:24:50.525188  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:50.530408  4602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:24:50.531442  4602 raft_consensus.cc:606] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:50.531873  4602 raft_consensus.cc:614] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:50.538007  5887 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:50.538671  5887 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:50.539178  5887 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:50.544540  4602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:24:50.545259  4602 raft_consensus.cc:606] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:50.550582  5925 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:50.550042  5872 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.561866  4498 catalog_manager.cc:5697] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.556775  5887 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.575213  4602 raft_consensus.cc:2955] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:50.566136  5925 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.570605  5902 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.587091  5902 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:50.588526  5925 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:50.589519  5902 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:50.591418  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:50.589735  5943 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:50.592739  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:50.590004  5814 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:50.593808  5814 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.596283  5814 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:50.592453  5943 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.596881  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:50.597630  4603 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:50.598506  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:50.598717  5943 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:50.593390  4753 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:50.599110  5887 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:50.599375  4756 raft_consensus.cc:2368] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:50.599852  4645 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:50.600749  4571 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:50.601323  4647 leader_election.cc:400] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:50.602105  5925 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:50.602541  5925 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:50.602741  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:50.602983  5925 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:50.603495  4603 raft_consensus.cc:2368] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:50.604395  5943 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:50.604732  4645 leader_election.cc:400] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:50.604785  5943 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:50.605130  4645 leader_election.cc:403] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:50.605297  5943 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:50.605821  5814 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:50.598675  5872 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:50.613925  5814 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:50.617967  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:50.618850  4756 raft_consensus.cc:2368] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:50.621744  4647 leader_election.cc:400] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:50.623181  4603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:24:50.623831  4603 raft_consensus.cc:606] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:50.624181  4603 raft_consensus.cc:614] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:50.628832  4603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8e62ff81eb34853811314530131292b"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:24:50.629699  4603 raft_consensus.cc:606] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:50.630227  4603 raft_consensus.cc:614] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:50.634505  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:50.635367  4675 raft_consensus.cc:2393] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 2: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:50.640148  4603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:24:50.640853  4603 raft_consensus.cc:606] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:50.643412  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:50.643967  5925 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.645183  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:50.646598  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:50.647617  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:50.649785  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:50.650616  4675 raft_consensus.cc:2393] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 2: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:50.647264  4499 catalog_manager.cc:5697] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.657970  4756 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:50.659050  4571 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:50.659915  5906 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:50.660372  5906 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:50.661253  5906 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.649201  5925 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:50.671105  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:50.673221  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:50.676491  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:50.676882  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:50.677507  4602 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:50.677476  4676 raft_consensus.cc:2393] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:50.685875  4602 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:24:50.687352  4603 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:50.688442  4756 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:50.695550  5346 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:50.712353  4496 catalog_manager.cc:5697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 1 to 2, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:50.689719  5943 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.722481  4675 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:50.721621  5816 tablet_copy_client.cc:323] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.193:45077
I20260709 12:24:50.726168  5906 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.726771  5906 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:50.730208  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:50.730147  4675 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:24:50.735669  5944 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.736349  5944 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.739013  5944 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:50.741685  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:50.745103  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:24:50.745445  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 48578.1us
I20260709 12:24:50.746034  4605 raft_consensus.cc:2368] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:50.746831  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:50.747495  4756 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:50.747906  4568 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:50.748490  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:50.748786  5804 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:50.749008  5782 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:50.749166  4748 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:50.747515  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:50.752720  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:50.755511  5820 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:50.755980  5820 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.756358  5820 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:50.763639  4645 leader_election.cc:400] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:50.764056  4645 leader_election.cc:403] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:50.765553  5814 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:50.767278  5872 raft_consensus.cc:2955] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:50.770388  5906 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:50.777129  5346 raft_consensus.cc:2955] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:24:50.779765  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 83361.4us
I20260709 12:24:50.786180  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:50.786590  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:50.787175  4748 raft_consensus.cc:2368] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:50.787663  5804 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:50.788233  5804 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:50.788750  4647 leader_election.cc:400] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:50.789314  5804 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.837177  5943 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:50.842790  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 48177.5us
I20260709 12:24:50.850615  4496 catalog_manager.cc:5697] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:50.851768  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:50.853971  4602 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:50.852996  4615 tablet_copy_service.cc:140] P 5c3201048f8346009aefca07428828f3: Received BeginTabletCopySession request for tablet 34a0b6acf7c6497e8ef8ffd63b4127d6 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:53448)
I20260709 12:24:50.854996  4615 tablet_copy_service.cc:161] P 5c3201048f8346009aefca07428828f3: Beginning new tablet copy session on tablet 34a0b6acf7c6497e8ef8ffd63b4127d6 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:53448: session id = dbe960c35c84451faf4e5b5c9233fa39-34a0b6acf7c6497e8ef8ffd63b4127d6
I20260709 12:24:50.854138  5820 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.857875  5897 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:50.858458  5897 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:50.859272  5820 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:50.860564  5897 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:50.862843  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:50.863644  5346 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:24:50.857697  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:50.871251  5897 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:50.871835  5897 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:50.872753  5897 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.873358  4615 tablet_copy_source_session.cc:215] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:50.894627  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:50.895298  4748 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:50.899927  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:50.900645  5948 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:50.895292  5820 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.901638  5820 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.903548  5948 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:50.906487  5943 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:50.910434  5820 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:50.873919  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 86316.1us
W20260709 12:24:50.904944  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 54807.9us
I20260709 12:24:50.918730  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:50.919857  4756 raft_consensus.cc:2393] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 2: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:50.920310  5948 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:50.924760  4748 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:50.934304  5312 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:50.935467  5948 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:50.936216  5948 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.936695  5948 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:50.948710  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:50.951133  4602 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:50.953265  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:50.953967  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:50.954023  4748 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:50.956225  4647 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:50.907775  5943 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.966187  5820 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:50.954924  5814 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:50.962392  5944 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.974694  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:50.976140  4602 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:50.975692  5944 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:50.970080  5820 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.988792  4603 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:50.989658  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:24:50.974850  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 60520.6us
I20260709 12:24:50.999338  5930 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:51.001046  5930 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:51.002532  5930 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:51.002315  4488 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 28f87749718d445ba1615ff33028c4d8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:50.995076  5820 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:50.994169  5929 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:51.016958  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 56933.8us
I20260709 12:24:51.045153  4679 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:51.049160  4675 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:51.060572  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 76365.6us
I20260709 12:24:51.010995  5943 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.053974  5887 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:24:51.090346  4589 tablet_service.cc:1558] Processing DeleteTablet for tablet 28f87749718d445ba1615ff33028c4d8 with delete_type TABLET_DATA_TOMBSTONED (TS 5c3201048f8346009aefca07428828f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60018
I20260709 12:24:51.053395  5929 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:51.111128  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 85945.9us
W20260709 12:24:51.121675  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 161470us
I20260709 12:24:51.124436  5814 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:51.125770  5814 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.089643  5944 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.106098  5948 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:51.140307  4675 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:51.141990  5902 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.142683  5902 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:51.143213  5948 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.148511  5928 raft_consensus.cc:993] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:24:51.148610  5929 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.149281  5928 raft_consensus.cc:1081] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:24:51.155401  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:51.156118  4679 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 1.
I20260709 12:24:51.145430  5902 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:51.170418  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:51.173996  5816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34a0b6acf7c6497e8ef8ffd63b4127d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:51.181473  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:51.182137  4679 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 1.
I20260709 12:24:51.182821  5953 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:51.183306  5953 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.179600  5954 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.186408  5954 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.188434  5954 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.185421  5953 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.178292  5782 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:51.192519  5782 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.194846  5782 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.170365  5930 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:51.196360  4748 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.197636  5930 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:51.199074  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:24:51.200109  5346 raft_consensus.cc:2368] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:51.202247  4719 leader_election.cc:400] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:51.202533  4719 leader_election.cc:403] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:51.203905  5820 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:51.205663  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 179591us
I20260709 12:24:51.207969  4748 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:24:51.212327  4756 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:51.218621  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:51.219252  4677 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:51.220191  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:51.223421  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:51.229071  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.230297  4748 raft_consensus.cc:2393] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 2: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:51.232643  5967 tablet_replica.cc:333] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:24:51.234304  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.235288  4750 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:51.238492  5804 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:51.246065  5804 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:51.252127  4672 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 2: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:51.221606  5961 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:51.257318  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.258008  4752 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:51.250590  4681 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51856
I20260709 12:24:51.259936  5314 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:24:51.271073  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:51.271977  4603 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:51.280315  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:51.280881  4603 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:24:51.283813  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.303501  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.306274  4748 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 2: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:24:51.310091  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 53453.2us
I20260709 12:24:51.276998  5902 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:51.322925  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:51.323942  4602 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:24:51.325042  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:51.325716  4600 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.282605  5965 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:51.329717  5965 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.330112  5965 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.330384  4681 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:51.330830  4681 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:51.331353  4645 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:51.332643  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:51.333887  5902 raft_consensus.cc:697] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:24:51.334964  5957 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:51.335429  5957 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:51.335830  5957 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:51.338403  5944 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:51.338923  5944 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.339288  5944 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.312997  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:51.287345  4672 raft_consensus.cc:2955] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:51.343228  4603 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:51.316953  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.317451  5820 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.267422  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:24:51.281747  5952 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:51.344012  4752 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:51.351836  5952 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:51.353979  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 119537us
I20260709 12:24:51.354090  5948 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:51.354668  5948 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.363039  5948 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.334967  5902 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:51.364693  5958 raft_consensus.cc:993] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:24:51.379901  4600 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:51.407546  5967 raft_consensus.cc:2243] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:51.408447  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:51.409188  4600 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:51.409458  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:51.408439  5967 raft_consensus.cc:2272] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:51.410130  4602 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.410742  5782 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.411253  5782 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.413949  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:51.414635  4605 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:51.415427  4681 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:51.417982  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 106087us
I20260709 12:24:51.419266  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:51.421769  5956 consensus_queue.cc:1059] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:51.424110  5956 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:51.424492  5956 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:51.424877  5956 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:51.425889  5958 raft_consensus.cc:3055] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:51.426287  5958 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:51.427083  5906 raft_consensus.cc:1081] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:24:51.416353  5967 ts_tablet_manager.cc:1916] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:51.432214  4602 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:24:51.429271  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:24:51.473937  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 190049us
I20260709 12:24:51.463300  4496 catalog_manager.cc:5697] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 3, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:51.476955  4496 catalog_manager.cc:5697] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:51.484519  5948 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:51.478858  5978 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.504029  4675 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:51.506989  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:51.537860  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:51.544157  4645 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:51.539222  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:51.546134  4600 raft_consensus.cc:3055] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:51.546515  4600 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:51.536268  4596 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:24:51.498891  4601 raft_consensus.cc:2368] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:51.556510  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:51.557271  4752 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.559043  4596 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:51.547425  4600 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.498595  5973 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:51.509744  5972 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:51.537107  5971 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:51.592952  5966 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:51.593444  5966 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:51.593876  5966 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:51.550256  5312 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:24:51.598134  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63131us
I20260709 12:24:51.601025  4676 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:51.603408  5928 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.604182  5902 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:51.603936  5928 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.604584  5902 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:51.604996  5902 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:51.608553  5954 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:51.608971  5954 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:51.609321  5954 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:51.614995  5928 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.582202  5952 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.622408  5948 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:51.624266  5965 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.582867  5973 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.627995  5973 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.588341  5972 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.631490  5972 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.633675  5971 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.635928  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:51.636713  4672 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.636937  5971 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.536266  5978 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.645943  5957 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.657670  4752 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
W20260709 12:24:51.648280  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 104233us
I20260709 12:24:51.664475  4645 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:51.666136  4672 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:24:51.668248  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:51.669023  4605 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.681219  4499 catalog_manager.cc:5697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 2 to 3, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:51.684198  4605 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:24:51.654834  5978 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.686247  5971 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:51.678165  5944 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.714975  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:51.715651  4672 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:51.710600  5953 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.728937  5953 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.730180  5954 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.730688  4675 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:51.732015  5953 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.733206  5954 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:51.730154  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 50196.8us
I20260709 12:24:51.717470  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:51.718259  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:51.734979  4678 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:51.719607  5957 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.726238  5965 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.734552  5902 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:51.738426  5782 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:51.726439  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.711975  4676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51856
I20260709 12:24:51.768496  4676 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:51.768919  4676 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:51.717640  5971 raft_consensus.cc:697] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:51.714123  5910 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.776619  5910 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.728121  5944 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.735028  4679 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:51.695102  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:24:51.778494  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 95388us
I20260709 12:24:51.780045  5948 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:51.780594  5948 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:24:51.781471  5948 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:51.735421  4645 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:24:51.785879  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 50543.1us
I20260709 12:24:51.757690  4681 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.759049  5816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:50644 (request call id 159) took 1236 ms. Trace:
I20260709 12:24:51.764482  4498 catalog_manager.cc:5697] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 1 to 3, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 3 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:51.767380  5966 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:51.775573  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 213077us
I20260709 12:24:51.795794  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:51.796602  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:51.797137  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:51.797474  4602 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:51.797828  4598 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:51.799913  5991 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.800405  5991 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.801529  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:51.802572  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:51.804033  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:51.804757  4605 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:51.806627  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:51.807512  4602 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:51.784766  5971 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.809770  5958 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.810818  5958 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.790146  5816 rpcz_store.cc:276] 0709 12:24:50.522614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.522794 (+   180us) service_pool.cc:224] Handling call
0709 12:24:50.523851 (+  1057us) ts_tablet_manager.cc:934] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
0709 12:24:51.172173 (+648322us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:51.413295 (+241122us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:51.413437 (+   142us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:51.541664 (+128227us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:51.547639 (+  5975us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:51.758918 (+211279us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:51.758983 (+    65us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":243,"tablet-copy.queue_time_us":120,"thread_start_us":196886,"threads_started":1}
I20260709 12:24:51.812739  5816 tablet_copy_client.cc:806] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:51.813270  5816 tablet_copy_client.cc:670] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:51.817035  4600 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:51.818058  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:51.818667  4681 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:51.819903  5966 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.821861  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.822842  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.823616  4750 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 2: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:51.823729  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:51.824138  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:51.786790  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:51.825280  4756 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.823150  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.826402  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.826601  4753 raft_consensus.cc:2393] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 3: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:51.829280  4600 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:51.830451  4749 raft_consensus.cc:2393] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 2: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:51.831169  5857 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:51.831568  5857 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.831915  5857 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:51.832895  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.833768  4747 raft_consensus.cc:2393] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 2: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:51.833856  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:51.838977  4569 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:51.840546  4756 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:51.840521  4569 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:51.790787  5985 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:51.852056  4752 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 4: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:51.852197  5985 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:24:51.851102  5868 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:51.784759  4676 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:51.862980  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 67530.7us
I20260709 12:24:51.866518  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.867139  4753 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:51.851799  5906 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:51.871851  4600 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:51.873242  4676 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.875037  4603 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:24:51.875396  4603 pending_rounds.cc:85] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 1
I20260709 12:24:51.875612  4603 pending_rounds.cc:107] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:24:51.875834  4603 raft_consensus.cc:2889] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:51.855096  5929 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:51.857926  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.885555  5929 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.885798  4750 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:51.886035  5929 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.824030  5701 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.886751  5701 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.888665  4750 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:51.888912  5701 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:51.861094  4599 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:51.855651  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:51.898818  4751 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 2: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:51.877949  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:51.899814  4753 raft_consensus.cc:2393] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 2: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:51.868988  5902 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.878060  5816 tablet_copy_client.cc:538] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:24:51.903223  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 108067us
I20260709 12:24:51.890466  5910 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.890503  5977 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:51.906308  4678 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:51.892377  5956 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.892670  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:51.908077  4756 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:51.863291  5985 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:51.894616  4676 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:51.850162  4681 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:51.902127  4604 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:51.869931  5906 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.906191  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:51.937414  4672 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.891748  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:51.907302  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:51.906883  4645 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:51.914599  5930 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:24:51.932981  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:51.934337  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:51.940595  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:51.934774  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.936159  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:51.941771  4752 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:51.938622  4749 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:51.917089  5701 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:24:51.943193  4597 raft_consensus.cc:3055] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:51.943549  4597 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:51.944499  5956 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:51.944367  4597 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.945046  5943 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:51.945473  5943 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:51.945930  5943 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:51.941054  4603 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:51.940656  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:51.949878  4753 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.951092  4749 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:51.939225  5906 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.939986  4598 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:51.956319  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:51.951895  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:51.954258  4672 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:51.954777  4756 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:24:51.908618  5967 ts_tablet_manager.cc:1929] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:51.945320  4597 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:51.985917  4598 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:51.959059  4720 leader_election.cc:400] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3.
I20260709 12:24:51.960225  5971 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:51.963616  5943 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.963824  5985 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.962630  4601 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:51.966205  5982 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:51.966037  5960 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:51.970796  5857 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.949970  5991 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:51.973675  4603 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:51.974455  5929 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:51.981127  5816 tablet_bootstrap.cc:492] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
W20260709 12:24:51.987116  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 49289.4us
I20260709 12:24:51.992826  5982 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:51.992620  4645 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:51.987993  5967 log.cc:1199] T 28f87749718d445ba1615ff33028c4d8 P 5c3201048f8346009aefca07428828f3: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-0-root/wals/28f87749718d445ba1615ff33028c4d8
I20260709 12:24:51.995558  5961 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:51.987782  5971 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:51.986040  5956 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.997506  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:52.013653  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:52.007189  5929 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:52.019309  4678 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:52.032573  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:52.033380  4749 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:52.034092  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:52.034786  4755 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 2: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:52.035934  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:52.069449  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:52.038414  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:52.070099  4748 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:52.070407  4755 raft_consensus.cc:2368] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3.
I20260709 12:24:52.072865  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:52.074759  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:52.075475  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:52.005365  5954 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:52.076731  5954 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.077090  5954 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:52.006654  5967 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60018 (request call id 68) took 1031 ms. Trace:
I20260709 12:24:52.007510  5971 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:52.016646  4681 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:52.053169  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:52.038461  4750 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:52.071395  4749 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:52.079487  4748 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:24:52.004972  5982 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:52.001039  4600 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:52.080155  4751 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:52.032732  4753 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
W20260709 12:24:52.068234  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 62692.1us
I20260709 12:24:52.104508  5670 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:52.105094  5670 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:52.107070  4569 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:52.019675  5943 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:52.070668  5988 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.040179  5953 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:52.086860  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:52.016610  5804 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.114686  4751 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:52.113711  5927 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:52.116526  5927 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.116887  5927 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:52.117076  5804 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.119517  5804 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:52.088542  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:52.001988  5312 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:52.124060  4754 raft_consensus.cc:2368] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5.
I20260709 12:24:52.011468  5985 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:52.030599  5963 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.072263  5961 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:52.083588  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 51366.5us
I20260709 12:24:52.079027  5967 rpcz_store.cc:276] 0709 12:24:50.993620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.061109 (+ 67489us) service_pool.cc:224] Handling call
0709 12:24:51.232140 (+171031us) ts_tablet_manager.cc:1151] Deleting tablet 28f87749718d445ba1615ff33028c4d8
0709 12:24:51.232247 (+   107us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:24:51.760454 (+528207us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:51.903001 (+142547us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:52.006562 (+103561us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":156611,"tablet-delete.queue_time_us":139582,"thread_start_us":129826,"threads_started":1}
W20260709 12:24:52.123976  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 122084us
I20260709 12:24:52.008004  5956 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.125331  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:52.113180  4672 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:52.021628  4645 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:52.000304  5857 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:52.023291  4597 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:52.009033  5906 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.136256  4571 leader_election.cc:400] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3.
I20260709 12:24:52.162281  4571 leader_election.cc:403] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:52.162674  4598 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:24:52.127218  5902 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:24:52.140905  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 135307us
I20260709 12:24:52.165267  4645 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:52.133229  5963 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.166975  5902 raft_consensus.cc:697] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:24:52.163235  4571 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:52.169078  4645 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:24:52.146348  4672 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:24:52.128417  5953 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:52.133438  5961 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.149883  4486 catalog_manager.cc:5028] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): tablet 28f87749718d445ba1615ff33028c4d8 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:24:52.150425  5670 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.148404  5944 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:52.124640  5988 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.169844  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:52.181172  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:52.157156  5971 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.170825  5910 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3.
I20260709 12:24:52.199605  5971 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:52.200269  5971 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:52.200709  5971 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:52.178170  5961 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:52.177855  5925 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:52.179991  5966 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:52.186596  5957 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:52.180243  5943 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:24:52.203413  5977 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:24:52.259737  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:52.262368  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:52.263063  4754 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:52.213027  5944 raft_consensus.cc:697] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:52.201545  5971 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.215802  5925 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:52.254628  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:52.225049  5964 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:52.256543  5982 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.256781  5956 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:52.213454  5902 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:52.283454  4679 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:52.269752  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 88611us
I20260709 12:24:52.292210  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:52.293720  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:52.299033  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:52.300609  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:52.301363  4681 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:52.314594  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:52.314775  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:52.313856  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:52.316854  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:52.336383  4571 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:52.336627  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:52.337550  4677 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:24:52.340058  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:52.341064  4679 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:52.275602  5988 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:52.272339  5964 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:52.313712  5985 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.275591  4597 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:52.284548  5982 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:52.407491  4600 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
W20260709 12:24:52.408730  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 72444.8us
I20260709 12:24:52.394959  5906 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:52.280773  5954 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.415261  4498 catalog_manager.cc:5697] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 1 to 3, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 3 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:52.273900  5953 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.271883  5925 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.360589  5887 consensus_queue.cc:1059] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 12:24:52.421676  5966 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:52.398447  5963 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:52.410890  6008 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:52.434496  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 68795.2us
I20260709 12:24:52.437517  4677 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:52.440061  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:52.440706  4748 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:52.408337  5985 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.451243  4748 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:52.443650  4681 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:52.475459  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:52.438273  6008 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.477483  6002 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:52.477895  6008 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:52.478139  6002 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:52.479206  6002 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.498107  4675 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
W20260709 12:24:52.517654  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 179725us
I20260709 12:24:52.519814  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:52.523499  4597 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:52.433545  5944 consensus_queue.cc:237] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.552038  5966 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:52.559998  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 530456us
I20260709 12:24:52.538919  6008 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.515697  5902 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
W20260709 12:24:52.566401  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 46000.9us
I20260709 12:24:52.567018  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:52.442070  5964 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.582711  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:52.590658  4751 raft_consensus.cc:2393] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 3: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:52.583307  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:52.591411  4750 raft_consensus.cc:2368] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:52.585723  4673 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:24:52.584750  5868 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:24:52.667673  5957 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:52.670130  5957 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.615299  5977 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 12:24:52.719089  5963 raft_consensus.cc:3055] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:52.725270  5804 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.732635  4679 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:52.687952  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:52.742849  4599 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:52.742771  4647 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:52.728474  5954 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:52.763121  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:52.764012  4602 raft_consensus.cc:2393] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:24:52.742201  5927 consensus_meta.cc:287] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.003s
I20260709 12:24:52.764609  5927 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.731644  5963 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:52.777223  5963 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.779145  5963 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:52.780920  5816 tablet_bootstrap.cc:492] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:52.761720  5971 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:52.751080  6011 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:24:52.767603  5985 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:52.766999  5927 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:52.767576  4603 raft_consensus.cc:2368] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:52.788631  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:52.739601  5804 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.771638  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:52.789266  4751 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:52.795424  4675 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:52.791177  4571 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:52.794306  4754 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:52.801025  5929 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:52.802027  4673 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:52.802253  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:24:52.797333  5961 consensus_meta.cc:287] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.003s
I20260709 12:24:52.812968  4675 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:52.820302  5701 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:52.805728  5982 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:24:52.835031  6008 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:52.805101  5929 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:52.837404  5986 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:52.838594  5929 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.843784  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:52.845808  4647 leader_election.cc:400] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5.
I20260709 12:24:52.846248  4647 leader_election.cc:403] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:24:52.822126  4751 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:52.854104  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:52.854945  4748 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:52.855618  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:52.867544  4748 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:24:52.871639  4647 leader_election.cc:400] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:52.872210  4647 leader_election.cc:403] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:52.881531  5971 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:52.881728  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:52.870154  5985 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5.
I20260709 12:24:52.876202  5816 tablet_bootstrap.cc:492] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:24:52.898984  4571 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:52.825455  5804 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:52.892597  4499 catalog_manager.cc:5697] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 3, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:52.897330  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:52.874645  5701 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:52.870782  5930 raft_consensus.cc:2764] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 12:24:52.886034  6006 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.331s	user 0.002s	sys 0.002s
I20260709 12:24:52.912361  5816 ts_tablet_manager.cc:1403] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.931s	user 0.025s	sys 0.018s
I20260709 12:24:52.923966  4747 raft_consensus.cc:2393] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:52.924705  6011 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:52.924952  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:52.925237  6011 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:52.854189  5961 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.858069  5953 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
W20260709 12:24:52.917830  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 108265us
I20260709 12:24:52.906783  5701 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.936127  6011 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:52.935616  5701 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:52.875407  5960 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:24:52.916659  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:24:52.908550  6022 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:24:52.950033  4749 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:52.952796  4756 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:52.961704  5960 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:52.979595  5960 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.994511  4600 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:52.975587  6032 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:24:52.991127  5960 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:53.012020  4676 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 12:24:53.003345  6035 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 12:24:53.002988  5982 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 12:24:53.020946  4600 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:53.114596  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:53.028262  4753 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:53.115202  4675 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:53.061435  4498 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 79) took 1035 ms. Trace:
I20260709 12:24:53.124518  4675 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:24:53.116667  4498 rpcz_store.cc:276] 0709 12:24:52.026096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.026235 (+   139us) service_pool.cc:224] Handling call
0709 12:24:52.265198 (+238963us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee420090 00007effee4bef3a 00007efff4274e9c 00007efff437304e 00007efff28342a4 00007efff2834232 00007efff2834161 00007efff28340dc 00007efff2833392 00007effec7f148c 00007effec7f08c6
0709 12:24:53.061366 (+796168us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:52.442481 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:52.477152 (+ 34671us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:52.477191 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:24:52.477207 (+    16us) tablet.cc:662] Decoding operations
0709 12:24:52.477621 (+   414us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:52.477682 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:52.477706 (+    24us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:52.477894 (+   188us) tablet.cc:701] Row locks acquired
0709 12:24:52.477906 (+    12us) write_op.cc:260] PREPARE: finished
0709 12:24:52.478058 (+   152us) write_op.cc:270] Start()
0709 12:24:52.478160 (+   102us) write_op.cc:276] Timestamp: P: 1783599892478013 usec, L: 0
0709 12:24:52.478176 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 12:24:52.478917 (+   741us) log.cc:844] Serialized 532 byte log entry
0709 12:24:52.724703 (+245786us) op_driver.cc:464] REPLICATION: finished
0709 12:24:52.923939 (+199236us) write_op.cc:301] APPLY: starting
0709 12:24:52.924033 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:52.924132 (+    99us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:52.924238 (+   106us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:52.924464 (+   226us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:52.924611 (+   147us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:52.924657 (+    46us) write_op.cc:312] APPLY: finished
0709 12:24:52.924835 (+   178us) log.cc:844] Serialized 22 byte log entry
0709 12:24:52.925031 (+   196us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:52.925111 (+    80us) write_op.cc:454] Released schema lock
0709 12:24:52.925147 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:52.925468 (+   321us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:52.925496 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":244038912,"child_traces":[["op",{"apply.queue_time_us":194203,"apply.run_cpu_time_us":1537,"apply.run_wall_time_us":1760,"num_ops":1,"prepare.queue_time_us":1527,"prepare.run_cpu_time_us":2719,"prepare.run_wall_time_us":40835,"replication_time_us":246381,"thread_start_us":94782,"threads_started":1}]]}
I20260709 12:24:53.130563  4600 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:24:53.137074  6039 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 12:24:53.142798  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.143318  4754 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:53.062233  5991 raft_consensus.cc:3055] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:53.030352  5983 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:53.086669  6011 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:53.126207  5961 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:24:53.144958  6002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.007s	sys 0.013s
I20260709 12:24:53.160965  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:53.166342  4569 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:53.147892  5991 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:53.180591  5953 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:24:53.167372  6002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.007s	sys 0.013s
I20260709 12:24:53.174999  5925 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:53.164507  6011 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:53.171456  5944 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:24:53.157611  5983 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:53.226294  5701 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:53.235622  5701 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:53.240651  4645 leader_election.cc:400] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:53.241140  4645 leader_election.cc:403] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:53.242352  6002 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:53.204821  5925 raft_consensus.cc:697] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:53.254678  5925 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:53.214272  5816 raft_consensus.cc:359] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:53.259649  5816 raft_consensus.cc:740] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:24:53.280716  4750 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:53.282728  4498 catalog_manager.cc:5697] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 3 to 2, term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from VOTER to NON_VOTER, now has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 2 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:53.287046  4602 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:53.290685  4499 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52812 (request call id 193) took 1210 ms. Trace:
I20260709 12:24:53.297358  4751 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:53.308398  4647 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:53.324227  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:53.327816  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:53.340332  4748 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:53.260620  5816 consensus_queue.cc:260] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:53.317806  4499 rpcz_store.cc:276] 0709 12:24:52.080226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.080823 (+   597us) service_pool.cc:224] Handling call
0709 12:24:52.226318 (+145495us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee420090 00007effee4bef3a 00007efff4274e9c 00007efff437304e 00007efff28342a4 00007efff2834232 00007efff2834161 00007efff28340dc 00007efff2833392 00007effec7f148c 00007effec7f08c6
0709 12:24:53.290528 (+1064210us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:52.929204 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:52.932017 (+  2813us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:52.932051 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:24:52.932068 (+    17us) tablet.cc:662] Decoding operations
0709 12:24:52.932392 (+   324us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:52.932447 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:52.932474 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:52.932663 (+   189us) tablet.cc:701] Row locks acquired
0709 12:24:52.932676 (+    13us) write_op.cc:260] PREPARE: finished
0709 12:24:52.932821 (+   145us) write_op.cc:270] Start()
0709 12:24:52.932917 (+    96us) write_op.cc:276] Timestamp: P: 1783599892932780 usec, L: 0
0709 12:24:52.932934 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 12:24:52.933660 (+   726us) log.cc:844] Serialized 537 byte log entry
0709 12:24:52.965377 (+ 31717us) op_driver.cc:464] REPLICATION: finished
0709 12:24:52.975680 (+ 10303us) write_op.cc:301] APPLY: starting
0709 12:24:52.979816 (+  4136us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:52.985191 (+  5375us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:52.991030 (+  5839us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:53.010328 (+ 19298us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:53.028720 (+ 18392us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:53.053611 (+ 24891us) write_op.cc:312] APPLY: finished
0709 12:24:53.068301 (+ 14690us) log.cc:844] Serialized 22 byte log entry
0709 12:24:53.126580 (+ 58279us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:53.128910 (+  2330us) write_op.cc:454] Released schema lock
0709 12:24:53.142529 (+ 13619us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:53.163609 (+ 21080us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:53.165271 (+  1662us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20603,"reported_tablets":3,"spinlock_wait_cycles":23537408,"child_traces":[["op",{"apply.queue_time_us":1811,"apply.run_cpu_time_us":2618,"apply.run_wall_time_us":176993,"num_ops":1,"prepare.queue_time_us":8957,"prepare.run_cpu_time_us":2467,"prepare.run_wall_time_us":11680,"replication_time_us":21327}]]}
I20260709 12:24:53.363757  5348 raft_consensus.cc:1217] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:24:53.366385  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 53416.9us
W20260709 12:24:53.357717  6047 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.096s	user 0.001s	sys 0.002s
W20260709 12:24:53.338145  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 118798us
W20260709 12:24:53.364039  6048 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.102s	user 0.003s	sys 0.000s
I20260709 12:24:53.358058  6026 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:53.407709  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:53.359611  5929 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.215261  5991 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:53.366693  6001 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:53.345722  5902 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:24:53.363782  6013 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:53.351712  6002 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:24:53.393616  5954 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:24:53.462723  6013 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:53.469048  5983 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:53.478032  6011 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:53.482050  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 116289us
I20260709 12:24:53.396030  5982 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:24:53.488370  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:53.419795  5929 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:53.491135  4599 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:53.492245  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 131737us
I20260709 12:24:53.506268  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:53.506871  4600 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:53.486191  5988 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:53.486469  5897 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.500773  6013 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:53.512151  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:53.526695  4599 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:24:53.527253  4600 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:24:53.488881  6023 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:24:53.499913  6001 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:24:53.491936  5966 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:24:53.531880  5961 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:53.566408  4604 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 12:24:53.561581  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 173077us
W20260709 12:24:53.569005  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 47085.7us
I20260709 12:24:53.559105  5816 ts_tablet_manager.cc:1434] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.630s	user 0.007s	sys 0.003s
I20260709 12:24:53.520700  5897 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:53.518647  5988 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:53.566373  5960 consensus_meta.cc:287] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.000s
I20260709 12:24:53.576861  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:53.579250  4599 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:53.574539  4604 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:53.583858  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:53.586673  5897 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:53.587302  5988 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:24:53.574251  5816 ts_tablet_manager.cc:1467] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Tablet startup took 650ms
I20260709 12:24:53.594601  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:53.595237  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:53.590566  5959 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:53.593358  5701 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:53.612519  5977 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:53.613018  5929 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:24:53.615248  6051 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:53.604415  6023 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
W20260709 12:24:53.610239  6055 log.cc:927] Time spent T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:24:53.583663  5960 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:53.653079  5960 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:24:53.654881  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 80932.9us
I20260709 12:24:53.656497  4672 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:53.657855  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:53.660401  4672 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:53.661926  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:53.664886  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:53.666553  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:53.671435  4679 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:53.628046  5988 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:24:53.708832  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 62003.1us
I20260709 12:24:53.712939  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:53.916802  4599 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:53.588164  5816 ts_tablet_manager.cc:1469] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Trace:
0709 12:24:50.522614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.522794 (+   180us) service_pool.cc:224] Handling call
0709 12:24:50.523851 (+  1057us) ts_tablet_manager.cc:934] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
0709 12:24:51.172173 (+648322us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:51.413295 (+241122us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:51.413437 (+   142us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:51.541664 (+128227us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:51.547639 (+  5975us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:51.758918 (+211279us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:51.758983 (+    65us) inbound_call.cc:177] Queueing success response
0709 12:24:51.980786 (+221803us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:24:52.926718 (+945932us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:24:53.010022 (+ 83304us) tablet_replica.cc:261] Starting instrumentation
0709 12:24:53.019230 (+  9208us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":243,"mutex_wait_us":7038,"raft.queue_time_us":4886,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":26345,"tablet-copy.queue_time_us":120,"tablet-download-pool.queue_time_us":112,"tablet-download-pool.run_cpu_time_us":1251,"tablet-download-pool.run_wall_time_us":50046,"thread_start_us":196886,"threads_started":1}
I20260709 12:24:53.894901  4754 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:24:53.923055  4754 raft_consensus.cc:606] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:53.674662  5959 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:53.683144  5857 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.262s
I20260709 12:24:53.712414  5701 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:53.887387  5897 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:53.818861  5956 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.136s
I20260709 12:24:53.709641  5988 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:53.971500  4599 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:24:53.974772  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:53.977995  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:53.983500  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:53.984091  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:53.984086  4750 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:53.988629  4754 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:53.989888  5988 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:53.959174  5897 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:53.959730  6008 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:24:53.986233  5872 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:24:54.001845  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 71520.5us
I20260709 12:24:53.998613  4615 tablet_copy_service.cc:342] P 5c3201048f8346009aefca07428828f3: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-34a0b6acf7c6497e8ef8ffd63b4127d6 received from {username='slave'} at 127.0.0.1:53448
I20260709 12:24:54.008065  4615 tablet_copy_service.cc:434] P 5c3201048f8346009aefca07428828f3: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-34a0b6acf7c6497e8ef8ffd63b4127d6 on tablet 34a0b6acf7c6497e8ef8ffd63b4127d6 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:54.009797  6066 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:24:54.010601  6049 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.661s	user 0.011s	sys 0.003s
I20260709 12:24:53.993244  5701 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.004645  5959 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:54.026309  5957 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.022s	sys 0.017s
W20260709 12:24:54.026746  5957 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.022s	sys 0.017s
I20260709 12:24:54.028348  6002 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:54.028864  6002 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:54.036505  6002 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.037679  6002 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:54.039077  5897 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:54.048538  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:54.049537  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:54.055213  5872 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:54.056315  5872 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.062579  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:54.139298  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:54.164072  4568 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:54.168664  6066 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:54.203984  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 66244.4us
W20260709 12:24:54.226090  6085 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:24:54.243330  4755 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:54.257777  5902 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:54.263602  5902 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:54.266896  5902 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:54.274977  6074 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.284s	user 0.003s	sys 0.000s
I20260709 12:24:54.277550  5966 raft_consensus.cc:3055] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:54.278054  5966 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:54.278926  5966 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.280531  5966 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:54.285729  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:54.289791  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:54.307288  5948 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:54.307897  5948 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:54.310149  5948 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:54.329959  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:54.330637  6021 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.331132  6021 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.315948  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:54.336436  5959 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:54.337558  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:54.336905  5948 raft_consensus.cc:2955] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:54.338287  5959 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:54.441951  6096 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:24:54.451848  5963 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:54.452438  5963 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.454821  5963 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:54.333397  6021 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:54.457883  6021 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:54.458590  5961 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:54.459172  5961 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:54.460675  6053 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 1.001s	user 0.016s	sys 0.000s
I20260709 12:24:54.461354  5961 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:54.462205  5961 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:54.462721  5961 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:54.445423  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 204562us
I20260709 12:24:54.449962  4751 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:24:54.448757  5902 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:54.458416  6021 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.463002  5930 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:24:54.443399  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 189676us
I20260709 12:24:54.444276  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:54.479903  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:54.468416  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:54.469594  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:54.472095  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:54.449604  6097 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.119s	user 0.000s	sys 0.013s
I20260709 12:24:54.498577  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1088) took 1311 ms. Trace:
I20260709 12:24:54.495052  4750 raft_consensus.cc:2955] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:54.494309  5872 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.501328  5872 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.504179  5962 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:54.504714  5962 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.505800  6067 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:54.506783  5962 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:54.506492  6067 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.499017  4598 rpcz_store.cc:276] 0709 12:24:53.184760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.184917 (+   157us) service_pool.cc:224] Handling call
0709 12:24:53.185233 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.185694 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.185720 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:53.188356 (+  2636us) log.cc:844] Serialized 26 byte log entry
0709 12:24:53.348266 (+159910us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:53.353982 (+  5716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.354059 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.460400 (+1106341us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b240011a728. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb24f9c4 00007effeb24dacb 00007effeb408dc9 00007effeb408a55 00007effeb40809d 00007effee03a10e 00007effee05188c 00007effee050bb6 00007effee0507d8 00007effee05a221
0709 12:24:54.495585 (+ 35185us) raft_consensus.cc:1685] finished
0709 12:24:54.495612 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.495837 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73,"raft.queue_time_us":546,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":5143,"spinlock_wait_cycles":5049600,"thread_start_us":143832,"threads_started":1,"wal-append.queue_time_us":162463}
I20260709 12:24:54.503415  5872 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:54.517360  5314 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:54.484928  5902 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:54.486495  5930 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.536379  4571 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:54.538604  4571 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:54.539525  6008 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:54.540062  5857 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:54.540169  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 47430.2us
I20260709 12:24:54.546046  4571 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:54.547043  6008 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:54.547544  6008 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:54.547979  6008 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:54.549917  6058 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:54.551527  6045 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:54.557070  6021 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:54.533070  5961 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:54.588248  5902 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:54.601723  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:54.608098  4753 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:54.586761  6104 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.610476  5930 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:54.610634  6104 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:54.615231  4680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 669) took 2095 ms (client timeout 1717 ms). Trace:
W20260709 12:24:54.615659  4680 rpcz_store.cc:269] 0709 12:24:52.519469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.519604 (+   135us) service_pool.cc:224] Handling call
0709 12:24:54.615005 (+2095401us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b5400085554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:24:54.615145 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3472364800}
I20260709 12:24:54.619614  6067 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:24:54.629106  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 111520us
I20260709 12:24:54.629791  4753 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:24:54.628723  6009 raft_consensus.cc:1064] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
W20260709 12:24:54.630676  6070 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.012s	sys 0.001s
I20260709 12:24:54.684746  4599 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:54.681463  6070 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.013s	sys 0.002s
W20260709 12:24:54.628986  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 132946us
I20260709 12:24:54.698912  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:54.700460  5991 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:54.683629  6104 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:54.737401  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 55330.8us
W20260709 12:24:54.741775  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.741986  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:54.743772  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:54.755365  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 312115us
W20260709 12:24:54.674098  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 146357us
I20260709 12:24:54.765873  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:54.766955  4754 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:24:54.703830  6009 consensus_queue.cc:237] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:24:54.752586  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 54975.8us
I20260709 12:24:54.846231  5965 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:54.832511  5963 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:54.769683  6004 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.089s
I20260709 12:24:54.834890  6058 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:24:54.878744  6110 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:24:54.852270  5314 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:54.883647  6113 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:54.885425  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:54.887575  6069 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:54.862337  5963 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.891491  5963 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:54.851670  5965 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.894047  5965 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:54.898634  5927 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:54.899246  5927 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.876987  6104 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.901322  6104 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.901557  5927 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:54.903610  6104 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:54.906116  6008 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:54.909679  6008 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:54.910894  4601 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:54.910866  4720 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:54.911857  5991 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:54.912233  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:54.912609  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:54.919340  4751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:24:54.920084  4751 raft_consensus.cc:606] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:54.921983  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:54.925908  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:54.930945  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:54.927469  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:54.935469  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:54.937011  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:54.857189  4599 raft_consensus.cc:2955] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:54.930845  5991 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:54.970022  6023 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:54.930492  5927 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:55.011315  6023 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.015307  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.016543  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.017941  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.018131  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:54.896896  6058 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:55.037665  6116 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:55.037999  5960 raft_consensus.cc:993] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:24:55.038437  5960 raft_consensus.cc:1081] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:24:55.038316  6116 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:55.039187  6058 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:55.042191  4678 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:55.043435  4677 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:24:55.044715  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.045130  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:54.940876  6066 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:55.047235  6066 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:54.942348  6114 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:55.023723  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.029608  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.018944  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.051221  4675 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:24:55.063016  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:54.940868  6104 consensus_queue.cc:1059] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 12:24:55.063745  5347 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:24:55.065284  4596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:24:55.067788  4596 raft_consensus.cc:606] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:55.065892  4675 raft_consensus.cc:3055] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:55.069097  4675 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:55.067467  6114 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:55.063072  6066 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:55.069868  4675 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.070739  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.070761  4675 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:55.073405  4602 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:55.063102  5927 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.078230  6114 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:55.066236  6089 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:55.079849  6089 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.082446  6089 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:55.083868  4675 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:24:55.084246  4675 pending_rounds.cc:85] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 1
I20260709 12:24:55.093909  4675 pending_rounds.cc:107] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:24:55.094230  4675 raft_consensus.cc:2889] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:55.084225  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:55.096431  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.096822  4747 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:55.101670  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:55.115389  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 1508) took 1131 ms. Trace:
I20260709 12:24:55.118681  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.116007  4756 rpcz_store.cc:276] 0709 12:24:53.984103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.984260 (+   157us) service_pool.cc:224] Handling call
0709 12:24:53.988508 (+  4248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.001485 (+ 12977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:54.001515 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:54.001559 (+    44us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:54.001620 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.115063 (+1113443us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b54000721d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:24:55.115135 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.115308 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29377792}
I20260709 12:24:55.130465  4596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:24:55.131309  4596 raft_consensus.cc:606] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Received request to transfer leadership to TS dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:24:55.134558  6094 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:24:55.137120  6023 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:55.156399  5966 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:55.157209  5966 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.159804  5966 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:55.104509  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:24:55.161532  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57132.9us
I20260709 12:24:55.166800  6114 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:55.172709  4596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:24:55.173556  4596 raft_consensus.cc:606] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:55.174211  4596 raft_consensus.cc:614] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:55.175530  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.179879  6092 raft_consensus.cc:3055] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:55.180428  6092 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:55.193516  6116 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:55.199120  6092 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.201048  6092 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:55.208392  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:55.214951  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:55.216889  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:55.195549  5927 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:55.198065  6011 raft_consensus.cc:993] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: : Instructing follower dbe960c35c84451faf4e5b5c9233fa39 to start an election
I20260709 12:24:55.216889  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:55.196571  4568 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:24:55.218709  6011 raft_consensus.cc:1081] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Signalling peer dbe960c35c84451faf4e5b5c9233fa39 to start an election
I20260709 12:24:55.220873  6011 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:55.221199  6011 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:55.221551  6011 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:55.225812  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.227414  5345 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
 from {username='slave'} at 127.0.0.1:50644
I20260709 12:24:55.227986  5345 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:55.228356  5345 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:55.229126  6011 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:55.229501  5943 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:55.230052  5943 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:55.179911  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 139283us
I20260709 12:24:55.232031  5943 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:55.233901  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:55.234913  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:55.235049  6116 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:55.238332  6011 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:24:55.241154  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.250869  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.252561  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:24:55.255985  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.268839  4597 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:55.272511  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:55.281176  5963 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.286352  4676 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:24:55.281853  5963 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.288710  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:55.289858  6073 log.cc:927] Time spent T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 1.282s	user 0.010s	sys 0.003s
I20260709 12:24:55.290305  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.291003  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1086) took 1699 ms. Trace:
I20260709 12:24:55.291388  4752 rpcz_store.cc:276] 0709 12:24:53.591549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.591756 (+   207us) service_pool.cc:224] Handling call
0709 12:24:53.603127 (+ 11371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.603455 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.603490 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:53.607580 (+  4090us) log.cc:844] Serialized 26 byte log entry
0709 12:24:53.906139 (+298559us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:53.906177 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.906251 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.941530 (+1035279us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:24:55.278077 (+336547us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b24001f94d8. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb24f9c4 00007effeb24dacb 00007effeb408dc9 00007effeb408a55 00007effeb40809d 00007effee03a10e 00007effee05188c 00007effee050bb6 00007effee0507d8 00007effee05a221
0709 12:24:55.290815 (+ 12738us) raft_consensus.cc:1685] finished
0709 12:24:55.290837 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.290949 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68493,"raft.queue_time_us":13327,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":7947,"spinlock_wait_cycles":15196288,"thread_start_us":250188,"threads_started":1,"wal-append.queue_time_us":357022}
I20260709 12:24:55.291043  4678 raft_consensus.cc:2393] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:55.289400  5963 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:55.298916  5927 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:55.299405  6079 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:55.305845  4647 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.307132  5985 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:55.307683  5985 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.308135  5985 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:55.321476  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.323926  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.327226  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.329159  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:55.329833  5963 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:55.330811  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:55.331079  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.338833  4569 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:55.339196  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.339651  6079 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:55.340505  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.341807  6107 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:55.342181  5961 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:55.342417  5963 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:55.342640  5961 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.332723  4674 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:55.343098  5961 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:55.343329  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.345180  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.345726  6065 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:55.346539  4605 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:55.347055  6011 raft_consensus.cc:993] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:24:55.347498  6136 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:55.347982  6136 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:55.348393  6136 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:55.342231  6107 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:55.349861  6107 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:55.350615  4675 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:55.353520  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:55.347510  6011 raft_consensus.cc:1081] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:24:55.354554  5964 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:55.355120  5927 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:55.357587  6065 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.358225  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.358101  6065 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.358976  5887 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:55.359333  4675 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51856
I20260709 12:24:55.359920  4675 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:55.360289  4675 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:55.360220  6136 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.363195  5963 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:55.365010  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.365759  5963 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:55.367046  5961 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.367362  5963 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:55.369040  4675 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.360122  6065 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:55.365211  6107 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.371035  4675 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:55.371701  6107 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:55.371866  5927 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:55.372462  6136 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:55.374101  5927 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:55.369583  5961 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:55.388643  5927 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:24:55.395094  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 239254us
I20260709 12:24:55.404601  4752 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:55.406149  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:55.412385  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.412990  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:55.408815  4747 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:55.410063  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:55.378088  5348 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:55.413839  4749 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:55.415297  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:55.418072  4599 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:55.419878  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:55.428861  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:55.429777  4750 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:24:55.429731  4749 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:24:55.428972  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:55.432675  4749 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:55.444693  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:55.448766  4681 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:24:55.449467  4681 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:55.449860  4681 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:55.452898  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.454109  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.455461  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:55.456871  4752 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:55.459618  4498 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 80) took 2311 ms. Trace:
I20260709 12:24:55.460081  4498 rpcz_store.cc:276] 0709 12:24:53.148522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.148716 (+   194us) service_pool.cc:224] Handling call
0709 12:24:55.459558 (+2310842us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:53.359132 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:53.460867 (+101735us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:53.460904 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:24:53.460924 (+    20us) tablet.cc:662] Decoding operations
0709 12:24:53.461384 (+   460us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:53.461455 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:53.461502 (+    47us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:53.491455 (+ 29953us) tablet.cc:701] Row locks acquired
0709 12:24:53.493424 (+  1969us) write_op.cc:260] PREPARE: finished
0709 12:24:53.508486 (+ 15062us) write_op.cc:270] Start()
0709 12:24:53.515663 (+  7177us) write_op.cc:276] Timestamp: P: 1783599893508424 usec, L: 0
0709 12:24:53.519975 (+  4312us) op_driver.cc:348] REPLICATION: starting
0709 12:24:53.606708 (+ 86733us) log.cc:844] Serialized 841 byte log entry
0709 12:24:53.988506 (+381798us) op_driver.cc:464] REPLICATION: finished
0709 12:24:54.553117 (+564611us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effeb3ac0ac 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007efff1eecef0
0709 12:24:54.779900 (+226783us) write_op.cc:301] APPLY: starting
0709 12:24:54.788102 (+  8202us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:54.835284 (+ 47182us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:54.841624 (+  6340us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:54.854466 (+ 12842us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:54.942571 (+ 88105us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:54.971846 (+ 29275us) write_op.cc:312] APPLY: finished
0709 12:24:55.040122 (+ 68276us) log.cc:844] Serialized 22 byte log entry
0709 12:24:55.316233 (+276111us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:55.316385 (+   152us) write_op.cc:454] Released schema lock
0709 12:24:55.316449 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:55.317153 (+   704us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:55.317208 (+    55us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":124440,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":764355,"apply.run_cpu_time_us":17559,"apply.run_wall_time_us":547571,"num_ops":1,"prepare.queue_time_us":102,"prepare.run_cpu_time_us":4374,"prepare.run_wall_time_us":276785,"replication_time_us":460832,"spinlock_wait_cycles":6118144,"thread_start_us":936489,"threads_started":2,"wal-append.queue_time_us":230548}]]}
W20260709 12:24:55.467003  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 43670.1us
I20260709 12:24:55.467432  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:55.469074  4678 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:24:55.469452  4678 pending_rounds.cc:85] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 1
I20260709 12:24:55.470293  4678 pending_rounds.cc:107] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:24:55.470693  4678 raft_consensus.cc:2889] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:55.475507  4496 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52818 (request call id 188) took 2116 ms. Trace:
I20260709 12:24:55.475944  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:55.477147  4571 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:55.477351  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:55.478607  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:55.454710  4679 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:24:55.486334  4679 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:24:55.486703  4679 pending_rounds.cc:85] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 2
I20260709 12:24:55.486945  4679 pending_rounds.cc:107] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:24:55.487187  4679 raft_consensus.cc:2889] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:55.476204  4496 rpcz_store.cc:276] 0709 12:24:53.359581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.359721 (+   140us) service_pool.cc:224] Handling call
0709 12:24:55.475440 (+2115719us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35778,"reported_tablets":3}
I20260709 12:24:55.498716  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:55.473541  6057 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.090s	sys 0.064s
I20260709 12:24:55.444470  6136 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:55.507186  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 63800.4us
I20260709 12:24:55.522400  5956 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:55.530066  5956 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.532313  5956 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:55.539335  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:55.515681  4681 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.502627  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:55.520450  4604 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:55.540892  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.419232  5927 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.546489  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:55.584628  4605 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:24:55.506599  6060 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:55.544281  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:24:55.516860  6057 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.091s	sys 0.064s
W20260709 12:24:55.589828  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 112763us
I20260709 12:24:55.587587  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:55.343125  5345 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:55.492762  6104 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.599949  4602 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:24:55.600142  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 105711us
I20260709 12:24:55.634661  4602 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:55.600900  4499 catalog_manager.cc:5697] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 2 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:55.637313  4499 catalog_manager.cc:5697] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 2 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:55.638912  4499 catalog_manager.cc:5697] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 2 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:55.639693  5927 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:55.640431  4499 catalog_manager.cc:5697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:24:55.481945  6107 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.103s
I20260709 12:24:55.635532  4602 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.644985  4602 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:55.647008  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 62132.7us
I20260709 12:24:55.649432  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:55.649998  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:55.649428  5966 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:55.650887  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:55.650993  4752 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 4: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:55.650990  5966 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:24:55.652086  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 48394.1us
I20260709 12:24:55.653000  6133 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:55.653410  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:55.653772  5966 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:55.650120  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:55.654567  4499 catalog_manager.cc:5697] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 3, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:55.657616  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:55.657730  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:55.658286  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:55.504138  6009 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.105s
W20260709 12:24:55.601251  6120 log.cc:927] Time spent T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:24:55.658358  4747 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:55.656699  4499 catalog_manager.cc:5697] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 2 to 3, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:55.664240  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.667400  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.672189  4673 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:24:55.688083  4602 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:24:55.640053  6010 consensus_queue.cc:1059] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 12:24:55.690548  5910 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:55.690798  5966 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:55.690989  5910 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:55.692004  5910 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.596412  5782 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:55.693892  5782 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:55.694339  5782 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:55.698541  4747 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:24:55.708431  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:55.709308  4747 raft_consensus.cc:3055] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:55.709738  4747 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:55.654964  6136 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:55.739861  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 72565.6us
I20260709 12:24:55.661603  4681 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:55.743912  4647 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.710791  4747 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.745270  4747 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:55.693403  5910 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:55.764820  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.697620  5930 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 12:24:55.772775  5943 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:55.772917  5345 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:55.660267  6104 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.774863  5985 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.776058  6104 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:55.777984  4680 raft_consensus.cc:2368] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6.
I20260709 12:24:55.695982  5964 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260709 12:24:55.605343  6004 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.155s
I20260709 12:24:55.761471  5346 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:55.760771  6136 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.789108  6136 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:55.792665  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 48887.2us
I20260709 12:24:55.793514  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:24:55.796167  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.797410  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:55.798053  4597 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:55.798741  4597 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:55.797968  5985 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:55.818424  6041 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:55.819010  6023 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:55.750061  5804 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:55.854746  5804 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:55.832003  6124 raft_consensus.cc:993] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:24:55.858405  6124 raft_consensus.cc:1081] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:24:55.832721  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:55.859452  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:55.859781  4674 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:55.799584  4597 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:55.843089  6061 raft_consensus.cc:3055] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:55.866312  6061 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:55.857318  5804 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:55.870200  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:55.876307  4602 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:55.870988  4747 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:55.871655  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:55.879415  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:55.879997  4598 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:55.883749  4674 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:55.910446  4602 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:55.799535  5782 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:55.867393  6061 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:55.909562  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:55.931617  4598 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:24:55.911700  6058 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.924008  4597 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:55.937036  6041 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:55.942806  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:24:55.949882  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 295305us
I20260709 12:24:55.942032  6058 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:55.950448  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 315314us
I20260709 12:24:55.980887  4571 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:55.976266  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:56.002350  4569 leader_election.cc:400] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6.
I20260709 12:24:56.003545  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:56.004458  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:55.978356  6143 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:55.953917  6061 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:55.950439  6041 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:55.952828  5782 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:56.020890  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 226124us
I20260709 12:24:56.022321  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:55.987179  5985 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:55.983879  5929 raft_consensus.cc:2764] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:24:55.984617  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:55.988061  6094 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:56.024847  4676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:24:56.030320  6008 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:24:56.033623  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.035846  4679 raft_consensus.cc:2393] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:56.001133  6026 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.028359  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:56.048975  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:56.025343  6060 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:56.055361  4647 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:55.964403  6136 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.038546  5943 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:56.050304  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:56.041409  4676 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:56.062513  4676 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:56.051692  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:56.063305  4747 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 2: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:24:56.053362  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:56.017729  6023 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:56.066262  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:56.033946  5986 consensus_queue.cc:1059] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:24:56.108170  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:24:56.108702  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 63950.6us
I20260709 12:24:56.046066  6143 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:56.059314  5929 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:56.059185  6065 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:56.111097  4756 raft_consensus.cc:2368] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4.
I20260709 12:24:56.101255  6023 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:56.107311  5961 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:56.116627  4644 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:56.064640  5985 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:56.068885  5988 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:56.094389  6004 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:56.059439  6066 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 12:24:56.133440  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:56.050488  6094 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.131985  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:56.094838  6067 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:24:56.112679  6060 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:56.059782  6026 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.114859  5897 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:56.462741  6143 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.495599  6058 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:24:56.565390  6061 consensus_meta.cc:287] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.537s	user 0.005s	sys 0.002s
I20260709 12:24:56.121419  6041 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.508622  6156 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:56.461337  5985 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:56.497818  6147 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.395s	user 0.000s	sys 0.003s
I20260709 12:24:56.123587  5961 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:24:56.505290  6094 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:56.517557  5963 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:56.697757  5963 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.701254  5963 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:24:56.278645  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 155653us
I20260709 12:24:56.544703  6023 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:56.671530  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 727) took 1172 ms. Trace:
I20260709 12:24:56.703886  6041 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:24:56.704907  6041 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:56.534494  5988 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:56.604076  6060 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.498699  6136 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:56.256883  4676 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.521745  4602 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:56.555382  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:24:56.713277  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 157930us
I20260709 12:24:56.676121  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:56.683145  5857 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:24:56.498337  6142 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.433s	user 0.000s	sys 0.003s
I20260709 12:24:56.716553  5857 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.718714  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 1542) took 1352 ms. Trace:
I20260709 12:24:56.530726  5930 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:56.719709  5857 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:56.695418  5961 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:24:56.515204  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 398672us
I20260709 12:24:56.720315  5930 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.690292  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:56.284795  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:24:56.534020  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 402098us
I20260709 12:24:56.681778  5782 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:56.722872  4747 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:56.723446  5782 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.724879  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:56.726438  5930 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:24:56.674978  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.719164  4675 rpcz_store.cc:276] 0709 12:24:55.365692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.407100 (+ 41408us) service_pool.cc:224] Handling call
0709 12:24:55.407385 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.419623 (+ 12238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.419654 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.420090 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.868848 (+448758us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:55.868896 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.868965 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.718130 (+849165us) raft_consensus.cc:1685] finished
0709 12:24:56.718176 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.718438 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18297,"raft.queue_time_us":11839,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":53806,"thread_start_us":364992,"threads_started":1,"wal-append.queue_time_us":543714}
W20260709 12:24:56.682096  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 139044us
I20260709 12:24:56.735695  5782 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:56.703613  4603 rpcz_store.cc:276] 0709 12:24:55.498993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.499652 (+   659us) service_pool.cc:224] Handling call
0709 12:24:55.506211 (+  6559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.541263 (+ 35052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.541293 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.541755 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.945840 (+404085us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:55.945898 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.946005 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.669081 (+723076us) raft_consensus.cc:1685] finished
0709 12:24:56.669175 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.671217 (+  2042us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88086,"raft.queue_time_us":35125,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":12231,"thread_start_us":307771,"threads_started":1,"wal-append.queue_time_us":437595}
I20260709 12:24:56.531093  6148 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:56.741230  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:56.744468  6148 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.746037  6162 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:56.747182  6162 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:56.750095  6144 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:56.755827  6143 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:56.759541  6162 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:56.756898  6143 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.761235  6148 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:56.810717  6144 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:24:56.812804  6144 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:24:56.814410  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 52267.2us
W20260709 12:24:56.823783  4597 consensus_meta.cc:287] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.877s	user 0.015s	sys 0.009s
I20260709 12:24:56.824296  4597 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:56.825894  4747 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:24:56.833714  5983 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:56.834292  5983 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:56.835927  5983 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.851269  6143 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:56.851578  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:56.865790  6162 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:56.866463  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:56.866922  6162 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:56.868158  6148 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:24:56.868902  6148 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:56.869372  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:56.871160  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:56.872421  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:56.875702  6041 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:56.838307  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 58) took 1036 ms. Trace:
I20260709 12:24:56.877315  4597 rpcz_store.cc:276] 0709 12:24:55.788578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.791133 (+  2555us) service_pool.cc:224] Handling call
0709 12:24:56.825008 (+1033875us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:56.876102  5346 raft_consensus.cc:2955] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:56.850261  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:56.882709  4603 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:56.850039  5957 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:56.884115  6162 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:56.889310  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:56.891781  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:56.885326  5957 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.898885  5957 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:56.899369  4603 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
W20260709 12:24:56.901870  6171 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.003s
I20260709 12:24:56.887787  5983 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:56.905047  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:56.907389  5961 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:56.908126  5961 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.910964  5961 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:56.913968  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:56.914669  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:56.917464  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:56.919871  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:56.922515  6151 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.891s	user 0.005s	sys 0.001s
W20260709 12:24:56.931218  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 44061.2us
I20260709 12:24:56.950016  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:56.952229  5887 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:56.946830  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:56.953437  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:56.956593  6002 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:56.957147  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 726) took 1483 ms. Trace:
I20260709 12:24:56.957365  6002 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:56.957593  4599 rpcz_store.cc:276] 0709 12:24:55.473889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.474085 (+   196us) service_pool.cc:224] Handling call
0709 12:24:55.504869 (+ 30784us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b5800002798. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff348ffc0 00007efff347a929 00007efff347a3d2 00007effec9f1bd4
0709 12:24:55.505135 (+   266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.597080 (+ 91945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.597116 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.597612 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.983252 (+385640us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:55.983290 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.983451 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.900841 (+917390us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:24:56.954396 (+ 53555us) raft_consensus.cc:1685] finished
0709 12:24:56.954426 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.956976 (+  2550us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91475,"raft.queue_time_us":92442,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":32865,"spinlock_wait_cycles":78892672,"thread_start_us":245686,"threads_started":1,"wal-append.queue_time_us":373539}
I20260709 12:24:56.959100  4672 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:56.959940  6002 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:56.973209  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:56.974922  6026 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:56.975172  5804 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:56.975785  5804 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:56.977767  6146 consensus_queue.cc:1059] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:56.977954  6026 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:56.978060  6116 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:56.978792  6026 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:56.978209  4676 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:56.981290  6026 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:56.973240  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:56.981284  6060 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:56.984194  6060 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.985848  6170 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:56.986718  6060 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:56.986624  6170 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:56.971926  6002 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:56.987844  6002 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:56.989094  6170 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:56.990845  6002 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:56.990906  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:56.981380  6144 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:56.996783  5312 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:57.000487  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.000692  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:57.001322  5348 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:57.002441  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:57.003135  4604 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:57.005673  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.006976  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.007048  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.008205  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.009738  6116 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:57.010128  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.010733  4680 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:56.976948  5804 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.015130  5312 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:24:57.019994  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.019999  4604 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:57.025005  6112 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:57.025251  6116 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:57.025738  6116 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:57.026319  6116 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:57.025756  6112 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.030531  5963 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:57.041477  5963 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.042208  4571 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:57.043814  5963 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:57.043746  4571 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:57.042761  5782 raft_consensus.cc:2955] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:57.045131  5983 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:56.992347  6144 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:57.064431  6149 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:57.016615  4676 raft_consensus.cc:2955] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:57.074185  6026 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:57.077726  5897 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:57.077764  4499 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52812 (request call id 194) took 1823 ms. Trace:
I20260709 12:24:57.084725  4496 catalog_manager.cc:5697] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 2 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:57.079260  4499 rpcz_store.cc:276] 0709 12:24:55.236737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.236877 (+   140us) service_pool.cc:224] Handling call
0709 12:24:57.054169 (+1817292us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:56.031680 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:56.032263 (+   583us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:56.032296 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:24:56.032315 (+    19us) tablet.cc:662] Decoding operations
0709 12:24:56.033167 (+   852us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:56.033246 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:56.036750 (+  3504us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:24:56.106663 (+ 69913us) tablet.cc:701] Row locks acquired
0709 12:24:56.106684 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:24:56.106834 (+   150us) write_op.cc:270] Start()
0709 12:24:56.106954 (+   120us) write_op.cc:276] Timestamp: P: 1783599896106783 usec, L: 0
0709 12:24:56.106977 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:24:56.434985 (+328008us) log.cc:844] Serialized 2463 byte log entry
0709 12:24:56.647130 (+212145us) op_driver.cc:464] REPLICATION: finished
0709 12:24:56.877716 (+230586us) write_op.cc:301] APPLY: starting
0709 12:24:56.877866 (+   150us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:56.878123 (+   257us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:56.878414 (+   291us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:56.879571 (+  1157us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:56.879998 (+   427us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:24:56.880063 (+    65us) write_op.cc:312] APPLY: finished
0709 12:24:56.880491 (+   428us) log.cc:844] Serialized 52 byte log entry
0709 12:24:56.908193 (+ 27702us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:56.908462 (+   269us) write_op.cc:454] Released schema lock
0709 12:24:56.908534 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:56.909079 (+   545us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:56.909109 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":226192,"apply.run_cpu_time_us":3943,"apply.run_wall_time_us":35449,"mutex_wait_us":2035,"num_ops":6,"prepare.queue_time_us":292354,"prepare.run_cpu_time_us":95039,"prepare.run_wall_time_us":610131,"raft.queue_time_us":1137,"raft.run_cpu_time_us":9858,"raft.run_wall_time_us":221622,"replication_time_us":539989,"spinlock_wait_cycles":110208,"thread_start_us":426426,"threads_started":4,"wal-append.queue_time_us":135143}]]}
I20260709 12:24:57.091267  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.078253  6112 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.096554  4647 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.099560  6103 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:57.101954  6144 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:57.101841  4644 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:24:57.103016  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 50532.6us
I20260709 12:24:57.104615  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.105780  4600 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 2: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:57.107280  5983 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:57.107606  5943 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:57.108094  5943 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:57.108525  5943 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:57.107923  5983 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.074924  6026 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.110224  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.111203  6026 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.112290  6057 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:57.113044  6148 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:57.113236  6057 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.078486  5897 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.116439  5897 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:57.111230  4681 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:57.120072  5983 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.131366  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:57.132129  4679 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:57.101866  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.139950  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.147106  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.149364  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.144852  6144 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:57.159539  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.159588  4677 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:57.159303  6144 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:57.167213  6144 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:57.167687  6057 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.168994  6148 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:57.172649  4571 leader_election.cc:400] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4.
I20260709 12:24:57.172991  4571 leader_election.cc:403] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:24:57.174717  5927 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4.
I20260709 12:24:57.175414  5927 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.175999  5927 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:57.178980  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.181095  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.164273  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.182919  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.185842  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.186800  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.188431  5988 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:57.189091  5988 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.191051  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.191301  5897 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:57.191452  5988 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:57.192246  5988 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:24:57.194475  6150 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 1.164s	user 0.013s	sys 0.000s
I20260709 12:24:57.195535  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.196417  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 1540) took 1765 ms. Trace:
I20260709 12:24:57.196619  4674 raft_consensus.cc:2393] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:57.196868  4750 rpcz_store.cc:276] 0709 12:24:55.430793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.431684 (+   891us) service_pool.cc:224] Handling call
0709 12:24:55.431982 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.517865 (+ 85883us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.517899 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.534545 (+ 16646us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.978633 (+444088us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:55.978675 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.978760 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:57.162088 (+1183328us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:24:57.195848 (+ 33760us) raft_consensus.cc:1685] finished
0709 12:24:57.195868 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.196138 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85478,"raft.queue_time_us":152257,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":10088,"spinlock_wait_cycles":59788032,"thread_start_us":344324,"threads_started":1,"wal-append.queue_time_us":436478}
I20260709 12:24:57.198385  4673 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:57.192095  5897 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.184545  5943 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:57.209034  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.233703  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:57.236393  4569 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:57.238055  4752 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:57.238844  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.193099  6148 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.192994  5988 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.243244  4679 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:24:57.244781  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.248594  4674 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:57.237139  6116 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:57.242866  6060 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:57.246893  6112 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.262944  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.258509  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.259163  6060 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:57.277737  5346 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:57.284392  6013 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:24:57.287819  5927 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.288060  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.289865  4645 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:57.292024  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.292960  6116 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.296121  5943 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:57.301766  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:57.301991  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.304654  4496 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 81) took 1536 ms. Trace:
I20260709 12:24:57.307655  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:24:57.277815  6060 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:57.315366  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.327324  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.299737  6144 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:57.305061  4496 rpcz_store.cc:276] 0709 12:24:55.768011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.768178 (+   167us) service_pool.cc:224] Handling call
0709 12:24:57.304493 (+1536315us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:57.158391 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:57.177382 (+ 18991us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:57.177429 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:24:57.177446 (+    17us) tablet.cc:662] Decoding operations
0709 12:24:57.177848 (+   402us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:57.177929 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:57.177955 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:57.178281 (+   326us) tablet.cc:701] Row locks acquired
0709 12:24:57.178337 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:24:57.178506 (+   169us) write_op.cc:270] Start()
0709 12:24:57.178654 (+   148us) write_op.cc:276] Timestamp: P: 1783599897178448 usec, L: 0
0709 12:24:57.178674 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:24:57.179389 (+   715us) log.cc:844] Serialized 594 byte log entry
0709 12:24:57.181910 (+  2521us) op_driver.cc:464] REPLICATION: finished
0709 12:24:57.182459 (+   549us) write_op.cc:301] APPLY: starting
0709 12:24:57.182560 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:57.182661 (+   101us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:57.182756 (+    95us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:57.183020 (+   264us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:57.183192 (+   172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:57.183236 (+    44us) write_op.cc:312] APPLY: finished
0709 12:24:57.183417 (+   181us) log.cc:844] Serialized 22 byte log entry
0709 12:24:57.183618 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:57.183704 (+    86us) write_op.cc:454] Released schema lock
0709 12:24:57.183743 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:57.184097 (+   354us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:57.184125 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":194,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":232,"apply.run_cpu_time_us":1688,"apply.run_wall_time_us":1691,"num_ops":1,"prepare.queue_time_us":62552,"prepare.run_cpu_time_us":3117,"prepare.run_wall_time_us":24946,"replication_time_us":3117}]]}
I20260709 12:24:57.330818  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:57.332355  6144 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:57.332778  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.342401  6104 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:57.343017  6104 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:57.344065  6104 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.348532  4720 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.352504  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.352963  4603 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:57.353957  4600 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:57.357879  5988 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:57.358002  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.363945  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.364077  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.365455  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.365875  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.366868  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.367425  4750 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:57.367493  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.369218  4749 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:57.369437  4747 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:57.369976  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.365424  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.374912  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:57.375699  5963 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:57.384240  4720 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.390080  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.390452  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:57.391222  4756 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:57.391512  4756 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:57.388986  5887 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.392342  4756 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.393829  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.395009  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.398036  6060 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.399518  4756 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:57.400331  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.404481  5887 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:57.406145  5897 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:57.408851  5988 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:57.404263  5943 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:24:57.415385  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:57.403815  6170 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:57.409824  5957 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:24:57.419162  6066 raft_consensus.cc:3055] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:57.419633  6066 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:57.430824  5782 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.431402  5782 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.431667  4500 catalog_manager.cc:5697] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:57.436108  6066 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.429360  4571 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:57.440191  6148 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:57.442703  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:57.444301  4720 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.434937  4500 catalog_manager.cc:5697] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:57.446812  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.449376  6148 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:57.433351  5782 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.468240  4647 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.469834  6010 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.470294  4605 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:57.470382  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:57.470975  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.472187  6060 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.475375  4571 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:57.476781  6010 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:57.484119  6066 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:57.493996  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:57.495397  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.498462  4605 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 2: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:24:57.499742  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:57.501824  4571 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:57.505823  4597 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:57.508087  6136 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:57.509210  4680 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:57.512187  4756 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:57.497671  5988 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.503571  6149 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:57.505300  6094 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.496050  5943 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:57.520777  4676 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:57.524597  4571 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:57.525153  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:24:57.558079  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 57547.2us
I20260709 12:24:57.527408  4719 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:57.527351  5701 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:57.538057  4500 catalog_manager.cc:5697] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:57.563761  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:24:57.565474  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 64883.2us
I20260709 12:24:57.566262  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.565739  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:57.570048  5910 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:57.565186  5927 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.565117  6170 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:24:57.570799  5927 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:57.572434  5887 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:57.573100  5927 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:57.573000  5887 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.574765  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:24:57.574862  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 47624us
I20260709 12:24:57.575525  5701 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.575626  6176 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:57.576201  5701 raft_consensus.cc:3055] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:57.576542  5701 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:57.578188  4751 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:24:57.571041  5943 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:57.579066  5943 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.574997  5897 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:57.580137  5897 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.581115  5943 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.581822  5986 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:57.581192  4500 catalog_manager.cc:5697] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:57.582124  5897 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:57.583043  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.583161  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:57.576205  6176 raft_consensus.cc:697] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:24:57.583652  4754 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:24:57.584290  6176 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:57.577433  5701 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:57.587750  5701 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:57.588371  4500 catalog_manager.cc:5697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:57.590767  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 86733.6us
I20260709 12:24:57.591647  6002 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:57.592214  5988 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:57.560560  5957 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:57.598028  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.598662  4676 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:57.599470  4647 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.601226  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.602459  4756 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:24:57.602777  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:57.578193  6192 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:24:57.593592  5957 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.606742  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.607415  4599 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:57.607759  5957 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.613214  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:57.614470  4647 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.619001  6149 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:57.618983  6011 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.619606  6149 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:57.619998  6149 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:57.623202  4645 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:57.626498  5985 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:57.628875  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:57.630093  5956 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:57.630535  5956 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:57.630908  5956 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:57.631719  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:57.632867  6060 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:57.633242  6060 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:57.633630  6060 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:57.626624  6002 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:57.635381  4647 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.640455  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.646217  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.647202  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.650907  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.618927  6148 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.659186  6112 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:57.655520  4756 raft_consensus.cc:2955] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:57.662847  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.668495  4604 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:57.644393  6104 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:57.650223  6002 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:57.651685  5985 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:57.681308  4753 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:57.682204  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.661041  5887 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:57.677340  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.671661  5943 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:24:57.679687  6002 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:57.761457  5956 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.764686  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:57.767002  5956 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.678519  6112 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 12:24:57.729840  6156 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:57.748257  4597 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:57.729177  6023 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:57.729794  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:24:57.777032  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 117027us
I20260709 12:24:57.780056  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:57.787330  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:57.772809  6199 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.091s	user 0.002s	sys 0.000s
I20260709 12:24:57.792531  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:57.789309  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.793150  4748 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:57.794849  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.796041  5701 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.814385  4672 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:57.817867  4602 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:57.765398  6060 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.824436  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.833696  6103 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.834695  4496 catalog_manager.cc:5697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:57.839756  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:57.844619  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:57.846921  4748 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:24:57.848158  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:24:57.849753  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 60605.1us
I20260709 12:24:57.851707  4647 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:57.856210  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:57.859483  6066 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:57.860072  6066 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:57.861048  6066 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.863601  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.864043  4751 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:57.864369  4752 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 3: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:57.865923  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:57.866741  4755 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 2: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:57.871692  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.849380  6103 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:57.855273  5897 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.875636  6103 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:24:57.876154  6103 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:57.869792  5963 consensus_queue.cc:1059] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:24:57.871066  6104 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:57.870904  6205 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:57.881085  6149 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:57.890697  5963 raft_consensus.cc:3055] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:57.906227  6013 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:57.915971  6060 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.895432  5963 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:57.888831  6136 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:57.927654  5897 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:57.928270  6103 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:57.935195  6200 log.cc:927] Time spent T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.176s	user 0.002s	sys 0.002s
I20260709 12:24:57.937633  6103 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.938254  6103 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.938683  6103 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.947892  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:57.948513  4754 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:57.944867  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:24:57.974690  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 73985.8us
I20260709 12:24:57.961711  5963 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:57.984838  5963 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:57.975070  6103 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:57.999743  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:57.980196  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:58.020908  5701 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:58.023407  4601 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:24:58.031369  5348 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:58.036645  4645 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:58.041841  6149 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:58.002676  6002 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:57.985184  5897 consensus_queue.cc:1175] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3a5aeedd11fe49938079fe86cb01232c" responder_term: 3 status { last_received { term: 3 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 3 index: 2 } }
I20260709 12:24:58.059468  4754 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:58.064803  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:58.065709  4603 raft_consensus.cc:2368] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:58.058879  6104 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:58.085898  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 81726us
I20260709 12:24:58.086688  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:58.088004  4645 leader_election.cc:400] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:58.088313  4645 leader_election.cc:403] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:58.037786  5887 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:24:58.086910  6104 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.092221  5897 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.101845  6103 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:58.091239  4601 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:58.086369  6002 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:58.116041  6104 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:58.146412  4605 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:24:58.147955  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:58.153625  4600 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:58.154170  6213 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:58.145222  6002 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:24:58.155241  6002 consensus_queue.cc:1243] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:24:58.155905  6002 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:58.160322  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:24:58.163800  6172 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:58.156291  6002 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:58.153062  6112 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:58.188547  6112 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:58.128785  5897 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:58.185647  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:58.195333  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:58.196405  4754 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 3: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:24:58.196074  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:58.198472  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:58.199491  4751 raft_consensus.cc:2368] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:58.199064  4747 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:24:58.213713  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:58.214342  4674 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:58.168654  6002 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:58.225065  4752 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:58.226730  4603 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:58.227221  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:58.202023  6013 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:24:58.220474  6208 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:24:58.235960  4601 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:58.236548  6013 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:58.237848  5983 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:58.238422  5983 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:58.239128  6013 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:58.239426  5983 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:58.241539  6112 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:58.265110  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:58.270663  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 57916us
W20260709 12:24:58.280268  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 62172.8us
I20260709 12:24:58.267429  6103 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
I20260709 12:24:58.266453  6104 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:58.306196  5897 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:58.309175  4675 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
W20260709 12:24:58.314049  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 58836.7us
I20260709 12:24:58.320772  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:24:58.324594  6218 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:58.327229  6104 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:58.275327  6156 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:24:58.281776  6172 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:24:58.329409  6172 raft_consensus.cc:3055] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:58.329876  6172 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:24:58.330744  6172 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:58.331426  6220 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 12:24:58.332018  6172 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:58.333878  4500 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52808 (request call id 181) took 2889 ms. Trace:
I20260709 12:24:58.334246  4500 rpcz_store.cc:276] 0709 12:24:55.444044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.444532 (+   488us) service_pool.cc:224] Handling call
0709 12:24:58.333803 (+2889271us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:57.604940 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:57.605478 (+   538us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:57.605510 (+    32us) write_op.cc:435] Acquired schema lock
0709 12:24:57.605523 (+    13us) tablet.cc:662] Decoding operations
0709 12:24:57.608876 (+  3353us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:57.608932 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:57.608997 (+    65us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:24:57.609536 (+   539us) tablet.cc:701] Row locks acquired
0709 12:24:57.609555 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:24:57.609713 (+   158us) write_op.cc:270] Start()
0709 12:24:57.609826 (+   113us) write_op.cc:276] Timestamp: P: 1783599897609671 usec, L: 0
0709 12:24:57.609849 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:24:57.610530 (+   681us) log.cc:844] Serialized 2079 byte log entry
0709 12:24:57.624408 (+ 13878us) op_driver.cc:464] REPLICATION: finished
0709 12:24:57.631090 (+  6682us) write_op.cc:301] APPLY: starting
0709 12:24:57.631175 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:57.631356 (+   181us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:57.631462 (+   106us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:57.632166 (+   704us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:57.632350 (+   184us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:24:57.632396 (+    46us) write_op.cc:312] APPLY: finished
0709 12:24:57.632625 (+   229us) log.cc:844] Serialized 47 byte log entry
0709 12:24:57.632787 (+   162us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:57.632910 (+   123us) write_op.cc:454] Released schema lock
0709 12:24:57.632952 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:57.633316 (+   364us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:57.633371 (+    55us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3228,"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":2418,"apply.run_cpu_time_us":2183,"apply.run_wall_time_us":2184,"num_ops":5,"prepare.queue_time_us":1012,"prepare.run_cpu_time_us":3202,"prepare.run_wall_time_us":7757,"replication_time_us":6000}]]}
I20260709 12:24:58.336238  4674 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:24:58.346717  4673 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:58.343034  4496 catalog_manager.cc:5697] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 2 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:58.353955  4499 catalog_manager.cc:5697] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 3, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:58.355868  4499 catalog_manager.cc:5697] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:58.360073  4499 catalog_manager.cc:5697] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 2 to 3, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:58.373658  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:58.367714  4499 catalog_manager.cc:5697] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 4 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:58.405346  6221 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.079s	user 0.004s	sys 0.001s
W20260709 12:24:58.420970  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 146846us
I20260709 12:24:58.425348  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:24:58.442339  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 120478us
I20260709 12:24:58.449868  4647 leader_election.cc:400] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3.
W20260709 12:24:58.451993  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 136759us
I20260709 12:24:58.455052  4756 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:58.461625  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:58.462428  4756 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:58.474380  4569 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:24:58.475852  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:58.478291  6011 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:58.479344  6011 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:58.479960  6011 raft_consensus.cc:697] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:58.480857  6011 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:58.494804  6112 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:58.496255  6002 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:58.507201  4756 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:24:58.518184  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:24:58.523152  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:58.556813  5701 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:58.568655  6116 raft_consensus.cc:2955] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:58.573947  4673 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:58.577271  4605 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:24:58.578231  4605 raft_consensus.cc:606] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Received request to transfer leadership to TS dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:58.632256  4485 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3e20c8bae3534224bb7a82db42cab1a5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:58.643167  6149 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:24:58.613560  6240 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:24:58.655102  6234 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:58.655799  6234 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:58.656736  6234 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:58.687305  4602 raft_consensus.cc:2955] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:58.691509  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:58.692239  4599 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:58.692543  4748 raft_consensus.cc:2955] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:58.697624  6149 raft_consensus.cc:993] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3: : Instructing follower dbe960c35c84451faf4e5b5c9233fa39 to start an election
I20260709 12:24:58.698143  6149 raft_consensus.cc:1081] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Signalling peer dbe960c35c84451faf4e5b5c9233fa39 to start an election
I20260709 12:24:58.700409  4755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:24:58.701045  4755 raft_consensus.cc:606] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:58.705853  4756 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:58.709813  5348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
 from {username='slave'} at 127.0.0.1:50644
I20260709 12:24:58.710497  5348 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:58.710855  5348 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:58.727264  4602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:24:58.728726  4756 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:58.729434  4602 raft_consensus.cc:606] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:24:58.705814  4599 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:24:58.736533  5983 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:58.736770  6023 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:58.753090  4601 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:58.760035  5348 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:58.785352  5348 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:58.784716  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:58.790885  4755 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:58.808538  4755 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:58.810463  4601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:24:58.816390  4601 raft_consensus.cc:606] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:24:58.803608  5985 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:24:58.819025  5314 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:24:58.819631  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:58.820197  4599 raft_consensus.cc:3055] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:58.820465  4599 raft_consensus.cc:740] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:58.843230  4675 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:58.849493  6008 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:58.849922  4601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8e62ff81eb34853811314530131292b"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:24:58.850493  4601 raft_consensus.cc:606] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:58.850870  4601 raft_consensus.cc:614] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:58.852320  6192 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:58.853103  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:58.858095  6192 raft_consensus.cc:697] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:24:58.844319  4599 consensus_queue.cc:260] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:58.863332  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:58.896963  6192 consensus_queue.cc:237] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:58.899492  4599 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:58.912597  6215 raft_consensus.cc:993] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:24:58.913225  6215 raft_consensus.cc:1081] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:24:58.921052  4673 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:24:58.921900  4673 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:58.922341  4673 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:58.951704  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 132768us
I20260709 12:24:58.951761  4599 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:24:58.986176  4755 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:59.002043  5348 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:24:59.002450  5348 pending_rounds.cc:85] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 4
I20260709 12:24:59.002692  5348 pending_rounds.cc:107] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:24:59.011621  5348 raft_consensus.cc:2889] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:59.003209  6066 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:24:59.017462  6060 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:24:59.036339  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:59.057405  4496 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 82) took 1649 ms. Trace:
I20260709 12:24:59.057889  4496 rpcz_store.cc:276] 0709 12:24:57.408642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.420499 (+ 11857us) service_pool.cc:224] Handling call
0709 12:24:57.472808 (+ 52309us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee420090 00007effee4bef3a 00007efff4274e9c 00007efff437304e 00007efff28342a4 00007efff2834232 00007efff2834161 00007efff28340dc 00007efff2833392 00007effec7f148c 00007effec7f08c6
0709 12:24:59.057337 (+1584529us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:58.591745 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:58.592290 (+   545us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:58.592320 (+    30us) write_op.cc:435] Acquired schema lock
0709 12:24:58.592337 (+    17us) tablet.cc:662] Decoding operations
0709 12:24:58.592724 (+   387us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:58.592777 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:58.592804 (+    27us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:24:58.593119 (+   315us) tablet.cc:701] Row locks acquired
0709 12:24:58.593141 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:24:58.595071 (+  1930us) write_op.cc:270] Start()
0709 12:24:58.617872 (+ 22801us) write_op.cc:276] Timestamp: P: 1783599898593237 usec, L: 0
0709 12:24:58.619090 (+  1218us) op_driver.cc:348] REPLICATION: starting
0709 12:24:58.653094 (+ 34004us) log.cc:844] Serialized 975 byte log entry
0709 12:24:58.865232 (+212138us) op_driver.cc:464] REPLICATION: finished
0709 12:24:59.023084 (+157852us) write_op.cc:301] APPLY: starting
0709 12:24:59.023174 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:59.023295 (+   121us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:59.023401 (+   106us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:59.045137 (+ 21736us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:59.045301 (+   164us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:24:59.045344 (+    43us) write_op.cc:312] APPLY: finished
0709 12:24:59.045553 (+   209us) log.cc:844] Serialized 29 byte log entry
0709 12:24:59.045743 (+   190us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:59.045821 (+    78us) write_op.cc:454] Released schema lock
0709 12:24:59.045864 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:59.046253 (+   389us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:59.046280 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":695251,"reported_tablets":2,"spinlock_wait_cycles":55465856,"child_traces":[["op",{"apply.queue_time_us":157579,"apply.run_cpu_time_us":2058,"apply.run_wall_time_us":23149,"num_ops":2,"prepare.queue_time_us":239116,"prepare.run_cpu_time_us":5755,"prepare.run_wall_time_us":61645,"replication_time_us":193948,"thread_start_us":331178,"threads_started":2}]]}
I20260709 12:24:59.074620  4673 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:59.083026  4673 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:59.090188  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:59.090826  4601 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:59.091544  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:24:59.092028  4752 raft_consensus.cc:3055] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:59.092263  4752 raft_consensus.cc:740] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:59.098636  4498 catalog_manager.cc:5697] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 3 to 4, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 4 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:59.111758  4498 catalog_manager.cc:5697] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 2, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 2 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:59.118381  4599 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:59.107417  4752 consensus_queue.cc:260] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:59.130096  4498 catalog_manager.cc:5697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:24:59.136775  4673 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:59.138659  4752 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:59.124230  6023 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:24:59.143800  4499 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52818 (request call id 189) took 1702 ms. Trace:
I20260709 12:24:59.144644  4601 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:59.146736  4645 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:24:59.144217  4499 rpcz_store.cc:276] 0709 12:24:57.441708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.443107 (+  1399us) service_pool.cc:224] Handling call
0709 12:24:57.465489 (+ 22382us) spinlock_profiling.cc:243] Waited 7.38 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee420090 00007effee4bef3a 00007efff4274e9c 00007efff437304e 00007efff28342a4 00007efff2834232 00007efff2834161 00007efff28340dc 00007efff2833392 00007effec7f148c 00007effec7f08c6
0709 12:24:59.143698 (+1678209us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:58.659116 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:58.659678 (+   562us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:58.659711 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:24:58.659729 (+    18us) tablet.cc:662] Decoding operations
0709 12:24:58.660273 (+   544us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:58.660323 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:58.660348 (+    25us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:24:58.660749 (+   401us) tablet.cc:701] Row locks acquired
0709 12:24:58.660766 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:24:58.660887 (+   121us) write_op.cc:270] Start()
0709 12:24:58.660994 (+   107us) write_op.cc:276] Timestamp: P: 1783599898660847 usec, L: 0
0709 12:24:58.661013 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:24:58.661603 (+   590us) log.cc:844] Serialized 1694 byte log entry
0709 12:24:59.012876 (+351273us) op_driver.cc:464] REPLICATION: finished
0709 12:24:59.046647 (+ 33771us) write_op.cc:301] APPLY: starting
0709 12:24:59.046717 (+    70us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:59.046862 (+   145us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:59.046962 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:59.047439 (+   477us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:59.047600 (+   161us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:24:59.047645 (+    45us) write_op.cc:312] APPLY: finished
0709 12:24:59.047858 (+   213us) log.cc:844] Serialized 41 byte log entry
0709 12:24:59.048005 (+   147us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:59.048089 (+    84us) write_op.cc:454] Released schema lock
0709 12:24:59.048126 (+    37us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":30211,"reported_tablets":17,"spinlock_wait_cycles":16966400,"child_traces":[["op",{"apply.queue_time_us":30357,"apply.run_cpu_time_us":1724,"apply.run_wall_time_us":1724,"num_ops":4,"prepare.queue_time_us":233213,"prepare.run_cpu_time_us":2796,"prepare.run_wall_time_us":8301,"replication_time_us":338810,"thread_start_us":99651,"threads_started":1}]]}
I20260709 12:24:59.153846  5983 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:24:59.151787  4498 catalog_manager.cc:5697] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 6, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 6 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:59.152927  6023 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:59.178913  4752 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:24:59.174099  4498 catalog_manager.cc:5697] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 2 to 3, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:59.170315  6191 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.181041  6023 raft_consensus.cc:697] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:24:59.189049  6191 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:59.205539  6191 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:59.193092  6023 consensus_queue.cc:237] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:59.207448  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:59.215883  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:59.216678  4601 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:24:59.223021  6233 raft_consensus.cc:993] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:24:59.223356  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:59.224468  6215 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:59.224834  6215 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.225144  6215 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:59.223676  6233 raft_consensus.cc:1081] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:24:59.231637  4752 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:24:59.232299  4752 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:59.232721  4752 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:59.232903  6215 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:59.240362  4752 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:59.242017  4752 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:59.243207  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:59.243896  4673 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:59.245150  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:59.245812  4601 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:59.246126  4601 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:59.254601  4673 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:24:59.254098  4601 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:59.261554  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:59.267057  4601 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:59.273284  4752 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:59.275302  6191 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:59.275929  6191 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:59.276710  6002 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:59.277149  6191 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:59.284349  4601 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
W20260709 12:24:59.297981  6258 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:24:59.308991  6136 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.309655  6136 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:59.312294  6136 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:59.313225  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:24:59.313997  4672 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:24:59.314038  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:59.315004  4597 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:59.315279  4720 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:59.316286  6136 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:59.316803  6136 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.317214  6136 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:59.318068  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:59.318887  5346 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:24:59.325143  6227 raft_consensus.cc:993] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:24:59.326525  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:59.341336  6215 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:59.341668  6066 raft_consensus.cc:1081] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:24:59.348048  4597 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:24:59.348413  4755 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:59.351164  6116 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:59.355762  4748 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:24:59.356438  4748 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:59.356781  4748 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:59.370535  4748 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:59.373950  4748 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:24:59.391994  6136 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:59.394529  6136 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:59.395193  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:59.395746  4675 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:59.395952  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:24:59.396718  6148 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:59.397192  6148 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:59.386935  6112 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:24:59.398041  6148 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:59.399739  6112 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:59.402168  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:24:59.402844  4597 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:59.403182  4597 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:24:59.403534  4748 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:59.404735  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:24:59.405328  5348 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:59.405757  6112 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:24:59.403978  4597 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:24:59.406991  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:59.407680  4597 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:59.409150  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:24:59.402226  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:59.409926  4673 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:59.420239  4675 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:24:59.426232  4601 raft_consensus.cc:2368] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5.
I20260709 12:24:59.427362  4673 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:24:59.426057  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:59.432251  4645 leader_election.cc:400] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5.
I20260709 12:24:59.432611  4645 leader_election.cc:403] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:59.438122  4496 catalog_manager.cc:5697] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 3 to 5, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New cstate: current_term: 2 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:59.437341  4498 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52812 (request call id 195) took 1333 ms. Trace:
I20260709 12:24:59.443565  4720 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:24:59.445248  5701 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:59.444972  4496 catalog_manager.cc:5697] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 2 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:59.449206  4601 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:59.450075  5701 raft_consensus.cc:697] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:59.450889  5701 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:59.442811  6136 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:59.465190  5348 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:24:59.446764  5957 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:24:59.467805  6116 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:59.440330  4498 rpcz_store.cc:276] 0709 12:24:58.103772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.104058 (+   286us) service_pool.cc:224] Handling call
0709 12:24:59.437277 (+1333219us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:59.310033 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:59.310892 (+   859us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:59.331993 (+ 21101us) write_op.cc:435] Acquired schema lock
0709 12:24:59.342554 (+ 10561us) tablet.cc:662] Decoding operations
0709 12:24:59.343856 (+  1302us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:59.343939 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:59.343967 (+    28us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:24:59.344668 (+   701us) tablet.cc:701] Row locks acquired
0709 12:24:59.352424 (+  7756us) write_op.cc:260] PREPARE: finished
0709 12:24:59.366540 (+ 14116us) write_op.cc:270] Start()
0709 12:24:59.366654 (+   114us) write_op.cc:276] Timestamp: P: 1783599899366492 usec, L: 0
0709 12:24:59.366675 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:24:59.367308 (+   633us) log.cc:844] Serialized 2079 byte log entry
0709 12:24:59.384170 (+ 16862us) op_driver.cc:464] REPLICATION: finished
0709 12:24:59.384902 (+   732us) write_op.cc:301] APPLY: starting
0709 12:24:59.384987 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:59.385168 (+   181us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:59.385303 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:59.395505 (+ 10202us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:59.395683 (+   178us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:24:59.395725 (+    42us) write_op.cc:312] APPLY: finished
0709 12:24:59.395945 (+   220us) log.cc:844] Serialized 47 byte log entry
0709 12:24:59.396128 (+   183us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:59.396225 (+    97us) write_op.cc:454] Released schema lock
0709 12:24:59.396265 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:59.396613 (+   348us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:59.396641 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":247858,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":456,"apply.run_cpu_time_us":2340,"apply.run_wall_time_us":11682,"num_ops":5,"prepare.queue_time_us":61313,"prepare.run_cpu_time_us":3853,"prepare.run_wall_time_us":57592,"replication_time_us":17317,"thread_start_us":83967,"threads_started":1}]]}
I20260709 12:24:59.491248  6136 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:24:59.501708  6136 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:24:59.414232  6227 consensus_queue.cc:1175] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49277a7653524787a715232ab908bbc6" responder_term: 4 status { last_received { term: 3 index: 2 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:59.526230  4597 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:24:59.530800  4499 catalog_manager.cc:5697] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 4 to 5, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 5 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:24:59.533042  4499 catalog_manager.cc:5697] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 2 to 3, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:24:59.536603  4665 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e20c8bae3534224bb7a82db42cab1a5 with delete_type TABLET_DATA_TOMBSTONED (TS 3a5aeedd11fe49938079fe86cb01232c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51854
I20260709 12:24:59.535312  4499 catalog_manager.cc:5697] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:24:59.543371  5957 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5.
I20260709 12:24:59.546857  4496 catalog_manager.cc:5697] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 1 to 3, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 3 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:59.565140  5957 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:24:59.565837  5957 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:59.579681  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:24:59.584123  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:24:59.585618  4644 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:24:59.586570  6002 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:59.587855  5957 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:24:59.591279  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:24:59.606666  6272 tablet_replica.cc:333] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:24:59.610934  4755 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:59.613538  5957 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:59.619709  6272 raft_consensus.cc:2243] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:59.620627  6272 raft_consensus.cc:2272] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:59.623677  6272 ts_tablet_manager.cc:1916] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:59.651124  4498 catalog_manager.cc:5697] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 3 to 5, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 5 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:59.654031  4498 catalog_manager.cc:5697] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 2, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 2 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:59.673477  6272 ts_tablet_manager.cc:1929] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:24:59.674163  6272 log.cc:1199] T 3e20c8bae3534224bb7a82db42cab1a5 P 3a5aeedd11fe49938079fe86cb01232c: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root/wals/3e20c8bae3534224bb7a82db42cab1a5
I20260709 12:24:59.679054  4486 catalog_manager.cc:5028] TS 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): tablet 3e20c8bae3534224bb7a82db42cab1a5 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:24:59.687011  4597 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:59.688688  6224 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:59.719621  4499 catalog_manager.cc:5697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 3 to 4, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 4 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:59.708925  4597 consensus_queue.cc:237] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:24:59.723366  4499 catalog_manager.cc:5697] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:59.724722  4602 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:59.727699  4499 catalog_manager.cc:5697] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:59.730455  5701 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:59.730088  4499 catalog_manager.cc:5697] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 3 to 4, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 4 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:59.807938  5346 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:59.812671  6233 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:59.816566  4755 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:59.819097  6010 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:59.827129  4673 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:59.828823  4601 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:59.831450  6209 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:59.831780  6136 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:59.840081  6227 raft_consensus.cc:2955] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:59.860472  5346 raft_consensus.cc:2955] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:59.878404  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 34a0b6acf7c6497e8ef8ffd63b4127d6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:59.885409  4676 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:59.897261  4675 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:59.905937  4498 catalog_manager.cc:5697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:59.916548  6148 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:59.918396  6148 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:59.925750  4755 raft_consensus.cc:2955] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:24:59.957625  4601 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:59.974490  5346 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:59.976701  6209 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:59.977845  5701 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:59.998155  4675 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:00.000406  5701 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:00.031322  4665 tablet_service.cc:1558] Processing DeleteTablet for tablet 34a0b6acf7c6497e8ef8ffd63b4127d6 with delete_type TABLET_DATA_TOMBSTONED (TS 3a5aeedd11fe49938079fe86cb01232c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51854
I20260709 12:25:00.032441  6272 tablet_replica.cc:333] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:25:00.033635  6272 raft_consensus.cc:2243] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:00.037029  6272 raft_consensus.cc:2272] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:00.043402  6272 ts_tablet_manager.cc:1916] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:00.098608  6272 ts_tablet_manager.cc:1929] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:00.099056  6272 log.cc:1199] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 3a5aeedd11fe49938079fe86cb01232c: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root/wals/34a0b6acf7c6497e8ef8ffd63b4127d6
I20260709 12:25:00.100704  4486 catalog_manager.cc:5028] TS 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): tablet 34a0b6acf7c6497e8ef8ffd63b4127d6 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:25:00.287034  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:00.389833  4755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:00.390529  4755 raft_consensus.cc:606] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:00.523025  4676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:00.523672  4676 raft_consensus.cc:606] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:00.545245  4601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:25:00.547888  4601 raft_consensus.cc:606] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:00.555780  6224 raft_consensus.cc:993] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:00.556516  6224 raft_consensus.cc:1081] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:00.566799  4756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43876
I20260709 12:25:00.567890  4756 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:00.568418  4756 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:00.565629  4601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:25:00.569540  4601 raft_consensus.cc:606] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:00.571209  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:00.571794  6136 raft_consensus.cc:993] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:00.572237  6136 raft_consensus.cc:1081] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:00.573019  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:00.573814  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:00.574095  4673 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:25:00.574753  4673 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:00.575147  4673 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:00.581071  4756 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:00.590895  4673 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:00.591065  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:00.596498  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:00.605001  4673 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:00.606397  4676 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:00.607024  4676 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:00.607890  4676 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:00.608964  4676 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:00.609609  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:00.610280  4756 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:00.610396  5346 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:00.613837  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:00.621948  4755 raft_consensus.cc:3055] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:00.622273  4755 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:00.623027  4755 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:00.624295  4755 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:00.629885  5346 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:25:00.631572  4644 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:00.632473  6224 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:00.632937  6224 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:00.633818  6224 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:00.610775  4601 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:00.665489  4499 catalog_manager.cc:5697] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 2, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 2 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:00.686194  4755 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:25:00.690815  4601 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:00.692349  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:00.699000  5701 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:00.699635  5701 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:00.700534  5701 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:00.744114  4676 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:00.747103  4500 catalog_manager.cc:5697] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 4 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:00.906827  6233 raft_consensus.cc:993] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:00.907598  6233 raft_consensus.cc:1081] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:00.911067  4755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:25:00.911718  4755 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:00.912122  4755 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:00.942219  4755 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:00.944294  4755 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:00.944828  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:00.945438  4601 raft_consensus.cc:3055] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:00.945756  4601 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:00.946499  4601 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:00.947575  4601 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:00.948798  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:00.949373  5346 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:00.955893  5346 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:00.958258  4719 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:00.959139  6136 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:00.959651  6136 raft_consensus.cc:697] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:00.960570  6136 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:00.958787  4601 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:00.996798  4500 catalog_manager.cc:5697] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:01.024626  5857 raft_consensus.cc:993] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:01.025389  5857 raft_consensus.cc:1081] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:01.027642  4756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:25:01.028224  4756 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:01.028769  4756 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:01.034087  6294 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:25:01.040665  4756 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:01.067047  4756 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:01.068109  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:01.073326  4601 raft_consensus.cc:3055] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:01.076700  4601 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:01.077667  4601 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:01.078887  4601 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:01.079654  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:01.080269  4675 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:01.087265  4675 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:01.096503  4601 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:01.107115  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:01.108222  5701 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:01.108705  5701 raft_consensus.cc:697] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:01.109705  5701 consensus_queue.cc:237] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:01.162973  4500 catalog_manager.cc:5697] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:01.193849  5346 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:01.196794  5957 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:01.209908  4755 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:01.212838  6002 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:01.310397  4675 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:01.312546  5701 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:01.338239  4601 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:01.339973  5701 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:01.407438  5346 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:01.409710  5701 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:01.496379  4601 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:01.497838  5963 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:01.576471  4602 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:01.579329  5963 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:01.580878  4675 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:01.598093  6136 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:01.657851  4602 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:01.688570  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:01.739580  4748 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:01.743815  4672 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:25:01.746181  4568 consensus_peers.cc:597] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f173b0d5264e4eff8b1704becc555780. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:01.747574  5857 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:01.749873  5857 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:01.769528  6233 raft_consensus.cc:2955] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:01.784574  4748 raft_consensus.cc:2955] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:01.790699  4672 raft_consensus.cc:2955] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:01.803226  4602 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:01.811218  4499 catalog_manager.cc:5697] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 2 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:01.812659  4676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:01.813238  4676 raft_consensus.cc:606] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:25:01.829830  4748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:01.830462  4748 raft_consensus.cc:606] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:01.840060  4748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:01.840726  4748 raft_consensus.cc:606] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:01.846031  4597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:25:01.846642  4597 raft_consensus.cc:606] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:01.848351  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:01.850361  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:01.851262  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:01.868809  4748 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:01.870458  6010 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:25:01.876507  4568 consensus_peers.cc:597] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23fa1ebbcb7f4a7e9bcf064d4198f48e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:01.882863  4676 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:01.885340  5857 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:01.890749  6233 raft_consensus.cc:993] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:01.891506  6233 raft_consensus.cc:1081] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:01.894393  4755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:25:01.894968  4755 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:01.895313  4755 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:01.905213  4755 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:01.910142  4755 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:01.911159  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:01.913968  4597 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:01.914328  4597 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:01.915078  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:01.915681  4675 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:01.915580  4597 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:01.916854  4597 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:01.937979  4675 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:01.939719  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:01.947161  5701 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:01.947708  5701 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:01.948988  4597 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:01.949445  5701 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:01.966295  6233 raft_consensus.cc:2955] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:01.968124  4676 raft_consensus.cc:2955] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:01.993075  4496 catalog_manager.cc:5697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index -1 to 4, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:02.003592  4748 consensus_queue.cc:237] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:02.022294  4500 catalog_manager.cc:5697] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 2 to 3, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:02.025918  4752 raft_consensus.cc:2955] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
W20260709 12:25:02.048440  4719 consensus_peers.cc:597] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 718b286f0ba24a90af47c49ec0031b11. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:02.049705  4602 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:02.051525  4676 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:02.051427  5701 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:02.056020  6315 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:02.092870  4672 raft_consensus.cc:2955] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:02.096310  6136 raft_consensus.cc:993] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:02.096903  6327 raft_consensus.cc:1081] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:02.085389  5963 raft_consensus.cc:2955] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:02.099572  4673 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:25:02.100144  4673 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:02.100509  4673 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:02.108625  4673 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:02.139851  6136 raft_consensus.cc:993] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:02.140524  6327 raft_consensus.cc:1081] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:02.124521  4496 catalog_manager.cc:5697] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:02.145824  4676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:25:02.146553  4676 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:02.146853  6224 raft_consensus.cc:993] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:02.146981  4676 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:02.147480  6224 raft_consensus.cc:1081] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:02.149639  4597 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60032
I20260709 12:25:02.150235  4597 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:02.150616  4597 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:02.163200  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:02.163905  4601 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:02.166308  4673 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:02.179965  4601 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:02.183058  4645 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:02.184273  6002 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:02.184782  6002 raft_consensus.cc:697] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:02.185633  6002 consensus_queue.cc:237] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:02.201138  4748 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:02.225302  4677 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:02.228502  4599 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:02.230370  5963 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:02.250730  6327 raft_consensus.cc:2955] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:02.261664  4676 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:02.264222  4676 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:02.266409  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:02.267206  5346 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:02.269423  5701 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:02.273442  4603 raft_consensus.cc:2955] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:02.303846  4500 catalog_manager.cc:5697] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 3 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.316617  4597 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:02.392113  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:02.392937  4676 raft_consensus.cc:3055] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:02.393272  4676 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:02.397174  4603 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:02.376163  6332 ts_tablet_manager.cc:933] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:25:02.399223  5963 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:02.400230  6327 raft_consensus.cc:3055] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:02.400583  6327 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:02.401400  6327 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:02.402695  6327 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:02.374048  6331 ts_tablet_manager.cc:933] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:25:02.415872  6332 tablet_copy_client.cc:323] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.193:45077
I20260709 12:25:02.428385  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:02.436188  4748 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:02.395043  4676 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:02.478389  4676 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:02.457290  6331 tablet_copy_client.cc:323] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.193:45077
I20260709 12:25:02.487859  5346 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:02.471072  4615 tablet_copy_service.cc:140] P 5c3201048f8346009aefca07428828f3: Received BeginTabletCopySession request for tablet 23fa1ebbcb7f4a7e9bcf064d4198f48e from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:53448)
I20260709 12:25:02.543649  4615 tablet_copy_service.cc:161] P 5c3201048f8346009aefca07428828f3: Beginning new tablet copy session on tablet 23fa1ebbcb7f4a7e9bcf064d4198f48e from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:53448: session id = dbe960c35c84451faf4e5b5c9233fa39-23fa1ebbcb7f4a7e9bcf064d4198f48e
I20260709 12:25:02.554550  4615 tablet_copy_source_session.cc:215] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:02.538441  4601 raft_consensus.cc:2955] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:02.458737  4597 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:02.565445  4614 tablet_copy_service.cc:140] P 5c3201048f8346009aefca07428828f3: Received BeginTabletCopySession request for tablet f173b0d5264e4eff8b1704becc555780 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:53448)
I20260709 12:25:02.612721  4614 tablet_copy_service.cc:161] P 5c3201048f8346009aefca07428828f3: Beginning new tablet copy session on tablet f173b0d5264e4eff8b1704becc555780 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:53448: session id = dbe960c35c84451faf4e5b5c9233fa39-f173b0d5264e4eff8b1704becc555780
I20260709 12:25:02.622851  4614 tablet_copy_source_session.cc:215] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:02.650380  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:02.651042  4755 raft_consensus.cc:3055] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:02.651386  4755 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:02.585491  4644 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:02.653016  4676 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:02.594161  4673 raft_consensus.cc:2955] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
W20260709 12:25:02.682324  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 96908.3us
I20260709 12:25:02.673470  4755 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:02.704954  4569 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:02.707492  4500 catalog_manager.cc:5697] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 2 to 3, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.712733  4755 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:02.706010  6319 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:02.718199  6319 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:02.719192  6319 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:02.651196  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 136749us
I20260709 12:25:02.721941  5857 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:02.722543  5857 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:02.723359  5857 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:02.754365  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 55363us
I20260709 12:25:02.758657  4601 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:02.764173  4755 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
W20260709 12:25:02.788205  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 59340.5us
I20260709 12:25:02.793215  5701 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:02.795261  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 64940.7us
W20260709 12:25:02.804108  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 61140.7us
W20260709 12:25:02.814343  4719 consensus_peers.cc:597] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b8e62ff81eb34853811314530131292b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:02.820742  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:02.821349  4756 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:02.798853  6332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23fa1ebbcb7f4a7e9bcf064d4198f48e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:02.837685  4601 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:02.837685  4752 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:25:02.814163  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 93000.6us
I20260709 12:25:02.847498  4673 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:02.850817  5957 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:02.862099  6002 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:02.868242  6331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f173b0d5264e4eff8b1704becc555780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:02.875209  4756 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:02.887681  5963 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:02.899880  6345 ts_tablet_manager.cc:933] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:02.926461  6332 tablet_copy_client.cc:806] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:02.927156  6332 tablet_copy_client.cc:670] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:02.930505  6345 tablet_copy_client.cc:323] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.195:41049
I20260709 12:25:02.932941  4766 tablet_copy_service.cc:140] P 49277a7653524787a715232ab908bbc6: Received BeginTabletCopySession request for tablet 718b286f0ba24a90af47c49ec0031b11 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:42632)
I20260709 12:25:02.933580  4766 tablet_copy_service.cc:161] P 49277a7653524787a715232ab908bbc6: Beginning new tablet copy session on tablet 718b286f0ba24a90af47c49ec0031b11 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:42632: session id = dbe960c35c84451faf4e5b5c9233fa39-718b286f0ba24a90af47c49ec0031b11
I20260709 12:25:02.936445  4500 catalog_manager.cc:5697] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 2 to 3, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.947839  4766 tablet_copy_source_session.cc:215] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:02.953744  6345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 718b286f0ba24a90af47c49ec0031b11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:02.983229  6345 tablet_copy_client.cc:806] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:02.983829  6345 tablet_copy_client.cc:670] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
W20260709 12:25:02.986636  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 107489us
I20260709 12:25:02.987847  6345 tablet_copy_client.cc:538] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:02.989705  6331 tablet_copy_client.cc:806] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:02.990386  6331 tablet_copy_client.cc:670] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:02.993423  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:03.017148  6345 tablet_bootstrap.cc:492] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
W20260709 12:25:03.022411  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 159261us
I20260709 12:25:03.067891  4496 catalog_manager.cc:5697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:03.089277  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 55729.1us
I20260709 12:25:03.131604  6345 tablet_bootstrap.cc:492] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:03.132684  6345 tablet_bootstrap.cc:492] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
W20260709 12:25:03.135540  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 47528.8us
W20260709 12:25:03.136132  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 46184.1us
I20260709 12:25:03.137157  6345 ts_tablet_manager.cc:1403] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.120s	user 0.033s	sys 0.004s
I20260709 12:25:03.153541  6332 tablet_copy_client.cc:538] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:03.166553  6331 tablet_copy_client.cc:538] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:03.171012  5346 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:03.173678  6346 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:03.177146  6367 ts_tablet_manager.cc:933] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:03.167210  6345 raft_consensus.cc:359] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:03.178558  6345 raft_consensus.cc:740] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:25:03.179323  6345 consensus_queue.cc:260] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:03.199291  6345 ts_tablet_manager.cc:1434] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.000s
I20260709 12:25:03.201107  4766 tablet_copy_service.cc:342] P 49277a7653524787a715232ab908bbc6: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-718b286f0ba24a90af47c49ec0031b11 received from {username='slave'} at 127.0.0.1:42632
I20260709 12:25:03.201627  4766 tablet_copy_service.cc:434] P 49277a7653524787a715232ab908bbc6: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-718b286f0ba24a90af47c49ec0031b11 on tablet 718b286f0ba24a90af47c49ec0031b11 with peer dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:03.216889  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 75475.5us
I20260709 12:25:03.239975  4755 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:03.246771  6010 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:03.257412  4755 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:03.260717  6346 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:03.261258  6332 tablet_bootstrap.cc:492] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:25:03.317400  4676 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:03.320379  5857 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:03.322650  6331 tablet_bootstrap.cc:492] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:25:03.352752  4748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:03.353425  4748 raft_consensus.cc:606] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:03.360513  6367 tablet_copy_client.cc:323] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.195:41049
I20260709 12:25:03.362615  4766 tablet_copy_service.cc:140] P 49277a7653524787a715232ab908bbc6: Received BeginTabletCopySession request for tablet b8e62ff81eb34853811314530131292b from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:42632)
I20260709 12:25:03.363161  4766 tablet_copy_service.cc:161] P 49277a7653524787a715232ab908bbc6: Beginning new tablet copy session on tablet b8e62ff81eb34853811314530131292b from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:42632: session id = dbe960c35c84451faf4e5b5c9233fa39-b8e62ff81eb34853811314530131292b
I20260709 12:25:03.371335  4766 tablet_copy_source_session.cc:215] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:03.375000  6367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8e62ff81eb34853811314530131292b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:03.425800  4601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:25:03.426512  4601 raft_consensus.cc:606] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:03.431344  4673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:03.431962  4673 raft_consensus.cc:606] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:25:03.441664  4673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:03.442286  4673 raft_consensus.cc:606] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:25:03.449558  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:03.454396  6346 raft_consensus.cc:993] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:03.454921  6346 raft_consensus.cc:1081] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:03.457703  4601 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60032
I20260709 12:25:03.458324  4601 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:03.458936  4601 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:03.459398  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:03.460239  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:03.476503  4601 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:03.480382  5963 raft_consensus.cc:993] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:03.480654  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:03.481360  4673 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:03.481683  4673 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:03.482369  4673 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:03.483389  4673 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:03.480887  5963 raft_consensus.cc:1081] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:03.495082  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:03.495692  4756 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:03.498606  4601 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:03.503947  4756 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:03.506157  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:03.507417  6352 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:03.507989  6352 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:03.508857  6352 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:03.496738  4676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:25:03.513597  4676 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:03.547117  4676 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:03.508855  6367 tablet_copy_client.cc:806] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:03.539283  4673 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:03.566083  6367 tablet_copy_client.cc:670] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:03.580261  6367 tablet_copy_client.cc:538] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:03.604534  4499 catalog_manager.cc:5697] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:03.616870  4676 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:03.647575  4676 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:03.648772  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:03.649516  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:03.649381  4748 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:03.650167  5345 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:03.650391  4748 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:03.650301  6331 tablet_bootstrap.cc:492] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:03.651331  4748 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:03.652581  4748 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:03.660554  5345 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:03.662091  4644 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:03.663230  6346 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:03.663714  6346 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:03.664637  6346 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:03.665499  6367 tablet_bootstrap.cc:492] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:25:03.652604  6331 tablet_bootstrap.cc:492] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:25:03.666893  6331 ts_tablet_manager.cc:1403] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.345s	user 0.023s	sys 0.007s
I20260709 12:25:03.658401  6332 tablet_bootstrap.cc:492] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:03.675366  6331 raft_consensus.cc:359] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:03.676360  6331 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:25:03.687764  6331 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:03.668304  6332 tablet_bootstrap.cc:492] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:25:03.679900  5345 raft_consensus.cc:1217] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:25:03.689649  6319 raft_consensus.cc:993] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:03.696714  6319 raft_consensus.cc:1081] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:03.694998  6332 ts_tablet_manager.cc:1403] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.434s	user 0.035s	sys 0.011s
I20260709 12:25:03.698527  4601 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60032
I20260709 12:25:03.699054  4601 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:03.699414  4601 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:03.712209  6331 ts_tablet_manager.cc:1434] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.001s
I20260709 12:25:03.661294  4748 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:03.716609  6332 raft_consensus.cc:359] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:03.721251  6332 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:25:03.721772  4614 tablet_copy_service.cc:342] P 5c3201048f8346009aefca07428828f3: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-f173b0d5264e4eff8b1704becc555780 received from {username='slave'} at 127.0.0.1:53448
I20260709 12:25:03.740749  4614 tablet_copy_service.cc:434] P 5c3201048f8346009aefca07428828f3: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-f173b0d5264e4eff8b1704becc555780 on tablet f173b0d5264e4eff8b1704becc555780 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:03.709877  4601 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:03.748167  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:03.749018  4676 raft_consensus.cc:3055] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:03.749330  4676 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:03.750151  4676 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:03.751142  4676 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:03.739768  6332 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:03.758881  6352 raft_consensus.cc:993] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:03.754470  4496 catalog_manager.cc:5697] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 5, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 5 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:03.761996  6352 raft_consensus.cc:1081] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:03.763945  4752 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:25:03.764469  4752 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:03.764810  4752 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.765304  6332 ts_tablet_manager.cc:1434] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.067s	user 0.004s	sys 0.003s
I20260709 12:25:03.771574  4614 tablet_copy_service.cc:342] P 5c3201048f8346009aefca07428828f3: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-23fa1ebbcb7f4a7e9bcf064d4198f48e received from {username='slave'} at 127.0.0.1:53448
I20260709 12:25:03.772054  4614 tablet_copy_service.cc:434] P 5c3201048f8346009aefca07428828f3: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-23fa1ebbcb7f4a7e9bcf064d4198f48e on tablet 23fa1ebbcb7f4a7e9bcf064d4198f48e with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:03.793946  4752 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:03.799778  4752 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:03.803364  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:03.810312  4597 raft_consensus.cc:3055] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:03.812430  4597 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:03.813798  4597 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:03.815382  4597 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.819337  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:03.905967  4672 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:03.911482  4601 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:03.924516  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:03.925261  4752 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:03.930114  4676 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:03.931408  4569 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:03.936537  4597 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:03.948006  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:03.940004  6352 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:03.959893  6327 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:03.965643  5345 raft_consensus.cc:1217] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:25:03.970232  6327 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:03.971179  6327 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:03.961297  6352 raft_consensus.cc:697] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:03.988395  6352 consensus_queue.cc:237] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:03.997957  4673 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:04.000564  6010 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:04.006814  4496 catalog_manager.cc:5697] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:04.028359  4748 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:04.030033  6352 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:04.043354  4672 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:04.044333  4752 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:04.060395  4496 catalog_manager.cc:5697] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:04.076509  5963 raft_consensus.cc:1064] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:04.079537  5963 consensus_queue.cc:237] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:04.082037  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 162805us
I20260709 12:25:04.081712  6367 tablet_bootstrap.cc:492] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:04.095880  6367 tablet_bootstrap.cc:492] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:25:04.104749  6367 ts_tablet_manager.cc:1403] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.440s	user 0.051s	sys 0.033s
I20260709 12:25:04.147344  5345 raft_consensus.cc:1217] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4]   Dedup: 4.4->[]
I20260709 12:25:04.148280  6367 raft_consensus.cc:359] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:04.168913  6367 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:25:04.169735  6367 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:04.204007  4676 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:04.205229  5345 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:04.206638  6367 ts_tablet_manager.cc:1434] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.098s	user 0.008s	sys 0.000s
I20260709 12:25:04.207069  5963 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:04.211935  5963 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:04.219000  4597 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:04.219655  4766 tablet_copy_service.cc:342] P 49277a7653524787a715232ab908bbc6: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-b8e62ff81eb34853811314530131292b received from {username='slave'} at 127.0.0.1:42632
I20260709 12:25:04.220130  4766 tablet_copy_service.cc:434] P 49277a7653524787a715232ab908bbc6: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-b8e62ff81eb34853811314530131292b on tablet b8e62ff81eb34853811314530131292b with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:04.220826  5963 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:04.278645  5345 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:04.282050  6346 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:04.283746  6349 raft_consensus.cc:2955] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:04.288626  4672 raft_consensus.cc:2955] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:04.290625  4597 raft_consensus.cc:2955] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:04.297849  6010 raft_consensus.cc:1064] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:04.303843  6010 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:04.323378  4756 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:04.331110  4496 catalog_manager.cc:5697] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 reported cstate change: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:04.331398  6383 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:04.389045  5348 raft_consensus.cc:2955] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:04.443492  4756 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:04.445902  6352 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:04.446491  5348 raft_consensus.cc:1217] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:25:04.457583  4748 consensus_queue.cc:237] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:04.477808  5348 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:04.479028  4672 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:04.479404  5857 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:04.481279  5857 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:04.503003  4597 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:04.505551  6315 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:04.524833  4676 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:04.529871  6327 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:04.555850  4755 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:04.582691  5345 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:04.614399  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:04.632522  6407 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:04.650004  6315 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:25:04.651125  5348 raft_consensus.cc:2955] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:04.650094  6409 raft_consensus.cc:2955] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:04.671238  4672 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:04.715649  6010 raft_consensus.cc:1064] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
W20260709 12:25:04.721704  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 115345us
I20260709 12:25:04.737558  6010 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:04.742129  5963 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:04.746362  4500 catalog_manager.cc:5697] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:04.781754  4672 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:25:04.803845  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49927us
W20260709 12:25:04.816646  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 46977.2us
W20260709 12:25:04.883213  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 78043.1us
W20260709 12:25:04.897316  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 91917.9us
I20260709 12:25:04.910439  4672 raft_consensus.cc:2955] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:04.917970  5348 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:25:04.937216  4755 raft_consensus.cc:2955] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:04.951035  4748 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:25:04.948475  6419 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:25:04.868434  6408 raft_consensus.cc:2955] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:04.967252  6420 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:25:05.012732  4597 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:05.041800  4756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:05.042846  4756 raft_consensus.cc:606] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:05.071494  5348 raft_consensus.cc:2955] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:05.102073  6426 raft_consensus.cc:1064] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:05.109225  4676 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:25:05.112748  6411 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:05.127682  6426 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:05.134706  4676 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:05.132751  5345 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:25:05.212726  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 154728us
I20260709 12:25:05.215533  5348 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 12:25:05.232714  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 265829us
I20260709 12:25:05.229434  4500 catalog_manager.cc:5697] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 4 to 5, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:25:05.210022  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 152215us
W20260709 12:25:05.251006  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.207942  6419 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:25:05.219949  6315 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:25:05.317166  6440 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:25:05.328419  6415 raft_consensus.cc:993] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
W20260709 12:25:05.339126  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 109544us
I20260709 12:25:05.346751  6405 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:05.352387  6415 raft_consensus.cc:1081] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
W20260709 12:25:05.361043  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 96119us
W20260709 12:25:05.407377  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 67322us
I20260709 12:25:05.431334  4589 tablet_service.cc:1558] Processing DeleteTablet for tablet 718b286f0ba24a90af47c49ec0031b11 with delete_type TABLET_DATA_TOMBSTONED (TS 5c3201048f8346009aefca07428828f3 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60018
I20260709 12:25:05.431411  4601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:25:05.432283  4601 raft_consensus.cc:606] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:05.432929  4679 raft_consensus.cc:2955] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:05.443751  4488 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 718b286f0ba24a90af47c49ec0031b11 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:05.447403  4602 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:05.471423  4676 raft_consensus.cc:2955] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:05.472357  4679 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:25:05.472894  4679 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:05.473237  4679 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:05.475850  4672 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:05.494369  6452 tablet_replica.cc:333] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:25:05.495370  4679 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:05.495811  6452 raft_consensus.cc:2243] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:05.496935  6452 raft_consensus.cc:2272] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:05.500262  5345 raft_consensus.cc:2955] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:05.505338  4679 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:05.455452  6410 raft_consensus.cc:2955] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:05.514031  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:05.514627  4749 raft_consensus.cc:3055] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:05.514910  4749 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:05.515606  4749 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:05.516501  4749 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:05.508965  6452 ts_tablet_manager.cc:1916] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:05.521667  6422 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:05.524677  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:25:05.529657  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:05.530365  4601 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:05.541169  6427 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:05.550063  4749 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:05.554193  4647 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:05.556110  6416 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:05.556681  6416 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:05.558983  6416 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:05.558806  6409 raft_consensus.cc:2955] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:05.561666  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:05.564644  4601 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:05.563431  4496 catalog_manager.cc:5697] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 4 to 5, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New cstate: current_term: 2 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:05.567602  4673 raft_consensus.cc:2955] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:25:05.585006  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 321582us
I20260709 12:25:05.587319  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:05.595458  6407 raft_consensus.cc:993] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:05.596010  6407 raft_consensus.cc:1081] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:05.598446  5348 raft_consensus.cc:2955] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:05.618610  4748 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:25:05.619227  4748 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:05.619735  4748 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:05.622416  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f173b0d5264e4eff8b1704becc555780 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:05.654397  4748 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:05.657008  4749 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:05.659904  4748 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:05.677130  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:05.677810  4676 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:05.678750  4737 tablet_service.cc:1558] Processing DeleteTablet for tablet f173b0d5264e4eff8b1704becc555780 with delete_type TABLET_DATA_TOMBSTONED (TS 49277a7653524787a715232ab908bbc6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43852
I20260709 12:25:05.684715  6411 consensus_queue.cc:1059] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:05.685948  6411 raft_consensus.cc:3055] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:05.686338  6411 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:05.687178  6411 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:05.694294  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:05.726569  4676 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:05.727800  6411 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:05.809440  4500 catalog_manager.cc:5697] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 5 to 6, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 6 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:05.811898  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 48612.3us
I20260709 12:25:05.813585  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:05.814852  4496 catalog_manager.cc:5697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 reported cstate change: config changed from index 4 to 5, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:05.839859  6349 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:05.854883  6349 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:05.855849  6349 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:05.861472  6415 raft_consensus.cc:2955] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:05.868880  5348 raft_consensus.cc:2955] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:05.870898  6452 ts_tablet_manager.cc:1929] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:25:05.871345  6452 log.cc:1199] T 718b286f0ba24a90af47c49ec0031b11 P 5c3201048f8346009aefca07428828f3: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-0-root/wals/718b286f0ba24a90af47c49ec0031b11
I20260709 12:25:05.889130  4679 raft_consensus.cc:2955] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:05.913746  4486 catalog_manager.cc:5028] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): tablet 718b286f0ba24a90af47c49ec0031b11 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:25:05.909291  4756 raft_consensus.cc:2955] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:05.953140  4602 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:05.954722  6457 tablet_replica.cc:333] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:25:05.965698  4605 raft_consensus.cc:2955] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:05.980597  6002 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:25:06.060020  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 60919.1us
W20260709 12:25:05.997195  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47797.6us
I20260709 12:25:06.139006  6457 raft_consensus.cc:2243] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:06.139824  6457 raft_consensus.cc:2272] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:06.184653  6457 ts_tablet_manager.cc:1916] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:25:06.186203  6456 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:25:06.195686  4500 catalog_manager.cc:5697] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:06.202551  4597 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:06.207597  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.224094  4601 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:06.228353  5345 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:06.235575  4676 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:06.243650  6010 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:06.245108  6010 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:06.301108  4500 catalog_manager.cc:5697] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 3 to 4, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 4 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:06.305145  4673 raft_consensus.cc:2955] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:06.302922  6405 raft_consensus.cc:2955] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:06.368961  4756 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:06.350044  5345 raft_consensus.cc:2955] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:06.378656  4496 catalog_manager.cc:5697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 5 to 6, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:06.402400  4748 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:06.410295  4737 tablet_service.cc:1558] Processing DeleteTablet for tablet 23fa1ebbcb7f4a7e9bcf064d4198f48e with delete_type TABLET_DATA_TOMBSTONED (TS 49277a7653524787a715232ab908bbc6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43852
I20260709 12:25:06.427548  4673 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:06.420418  6437 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:25:06.430994  5344 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:06.446579  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 23fa1ebbcb7f4a7e9bcf064d4198f48e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:06.447821  6457 ts_tablet_manager.cc:1929] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:25:06.448369  6457 log.cc:1199] T f173b0d5264e4eff8b1704becc555780 P 49277a7653524787a715232ab908bbc6: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root/wals/f173b0d5264e4eff8b1704becc555780
I20260709 12:25:06.451545  6457 tablet_replica.cc:333] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:25:06.452708  6457 raft_consensus.cc:2243] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:06.458081  6415 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:06.466158  6450 raft_consensus.cc:2955] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:25:06.472625  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 44067.6us
I20260709 12:25:06.474603  4601 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:06.480736  5345 raft_consensus.cc:2955] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:06.480036  6457 raft_consensus.cc:2272] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:06.494093  4673 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:06.497386  4496 catalog_manager.cc:5697] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 4 to 5, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New cstate: current_term: 3 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:06.503522  4488 catalog_manager.cc:5028] TS 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): tablet f173b0d5264e4eff8b1704becc555780 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:25:06.516264  6457 ts_tablet_manager.cc:1916] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:06.527602  4589 tablet_service.cc:1558] Processing DeleteTablet for tablet b8e62ff81eb34853811314530131292b with delete_type TABLET_DATA_TOMBSTONED (TS 5c3201048f8346009aefca07428828f3 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60018
I20260709 12:25:06.542572  6450 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:06.546555  6472 tablet_replica.cc:333] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:25:06.546339  6450 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:06.549182  4488 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b8e62ff81eb34853811314530131292b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:06.558220  6472 raft_consensus.cc:2243] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:06.559973  6472 raft_consensus.cc:2272] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:06.563621  6415 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:06.564863  4679 raft_consensus.cc:2955] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:06.577275  6472 ts_tablet_manager.cc:1916] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:06.600672  6457 ts_tablet_manager.cc:1929] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260709 12:25:06.604643  6457 log.cc:1199] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 49277a7653524787a715232ab908bbc6: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root/wals/23fa1ebbcb7f4a7e9bcf064d4198f48e
I20260709 12:25:06.606998  4488 catalog_manager.cc:5028] TS 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): tablet 23fa1ebbcb7f4a7e9bcf064d4198f48e (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:25:06.619925  6472 ts_tablet_manager.cc:1929] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:25:06.620500  6472 log.cc:1199] T b8e62ff81eb34853811314530131292b P 5c3201048f8346009aefca07428828f3: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-0-root/wals/b8e62ff81eb34853811314530131292b
I20260709 12:25:06.636040  4486 catalog_manager.cc:5028] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): tablet b8e62ff81eb34853811314530131292b (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:25:06.866261  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:07.103302  4679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:07.104542  4679 raft_consensus.cc:606] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:25:07.113492  4748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:07.114292  4748 raft_consensus.cc:606] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:07.119681  4748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:07.120610  4748 raft_consensus.cc:606] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:07.128741  4597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:25:07.129494  4597 raft_consensus.cc:606] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:07.133620  4679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:07.134284  4679 raft_consensus.cc:606] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Received request to transfer leadership to TS dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:07.135573  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:07.136790  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:07.137727  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:07.275941  6421 raft_consensus.cc:993] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower dbe960c35c84451faf4e5b5c9233fa39 to start an election
I20260709 12:25:07.276746  6346 raft_consensus.cc:1081] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Signalling peer dbe960c35c84451faf4e5b5c9233fa39 to start an election
I20260709 12:25:07.279155  5344 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
 from {username='slave'} at 127.0.0.1:50638
I20260709 12:25:07.279979  5344 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:07.280496  5344 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:07.296886  5344 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:07.300616  5344 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:07.301863  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:07.302213  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:07.302724  4673 raft_consensus.cc:3055] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:07.302938  4601 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:07.303112  4673 raft_consensus.cc:740] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:07.304183  4673 consensus_queue.cc:260] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:07.306087  4673 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:07.314749  4601 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:25:07.316442  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:07.317190  4673 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:25:07.317602  6459 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:07.318192  6459 raft_consensus.cc:697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:07.319231  6459 consensus_queue.cc:237] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:07.341724  4498 catalog_manager.cc:5697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 1 to 2, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 2 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:07.345983  6414 raft_consensus.cc:993] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:07.346500  6414 raft_consensus.cc:1081] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:07.347937  4679 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:25:07.348412  4679 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:07.348726  4679 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:07.358335  4679 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:07.361263  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:07.361938  4601 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:07.367432  6346 raft_consensus.cc:993] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:07.367776  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:07.368393  4748 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:07.367937  6346 raft_consensus.cc:1081] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:07.368785  4748 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:07.369748  4748 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:07.370959  4748 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:07.370887  4679 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:07.373538  4597 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60032
I20260709 12:25:07.374094  4597 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:07.374441  4597 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.384267  4597 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:07.386585  4601 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:07.387182  4597 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:07.388005  4645 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:07.388266  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:07.388828  4679 raft_consensus.cc:3055] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:07.389173  4679 raft_consensus.cc:740] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:07.390035  4679 consensus_queue.cc:260] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:07.390779  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:07.391059  4679 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.391366  4756 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.394304  6346 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:07.394858  6346 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:07.395809  6346 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:07.405740  4748 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:07.410367  4756 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:07.411950  4571 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:07.412993  5857 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:07.413491  5857 raft_consensus.cc:697] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:07.414341  5857 consensus_queue.cc:237] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:07.444134  4679 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:07.455189  6414 raft_consensus.cc:993] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:07.455682  6414 raft_consensus.cc:1081] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:07.459403  4672 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:25:07.459904  4672 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:07.460242  4672 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.462208  4498 catalog_manager.cc:5697] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 2 to 3, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 3 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:07.469259  4672 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:07.472069  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:07.472630  4756 raft_consensus.cc:3055] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:07.473142  4672 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:07.472929  4756 raft_consensus.cc:740] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:07.474936  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:07.475540  5344 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.474953  4756 consensus_queue.cc:260] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:07.477552  4756 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:07.472890  4496 catalog_manager.cc:5697] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 5, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 5 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:07.496321  4756 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:07.498062  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:07.500771  6346 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:07.501497  6346 raft_consensus.cc:697] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:07.502470  6346 consensus_queue.cc:237] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:07.500710  5344 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:07.507027  5857 raft_consensus.cc:993] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:07.507716  6411 raft_consensus.cc:1081] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:07.509738  4756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:25:07.510262  4756 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:07.510593  4756 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:07.517086  4756 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:07.518931  4756 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:07.519882  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:07.519984  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:07.520560  4601 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:07.520646  4672 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:07.520987  4601 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:07.520792  4496 catalog_manager.cc:5697] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 2 to 3, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:07.521840  4601 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:07.528010  4672 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:07.523337  4601 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:07.530015  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:07.537156  4601 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:07.538337  6415 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:07.555631  6415 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:07.556664  6415 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:07.578245  4496 catalog_manager.cc:5697] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 5 to 6, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 6 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:07.635769  4679 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:07.656399  4601 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:07.656569  4756 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:25:07.657341  4644 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ea533a99927c40d48b894cf00a75301d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:07.658185  6436 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:07.659056  6346 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:07.687839  6436 raft_consensus.cc:2955] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:07.693817  4756 raft_consensus.cc:2955] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:07.697543  4601 raft_consensus.cc:2955] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:07.712105  4597 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:07.721541  4496 catalog_manager.cc:5697] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 2 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:07.723062  4605 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:07.732517  6428 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:07.777972  4679 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:07.781926  4601 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:07.788175  6421 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:07.788158  4756 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:25:07.793274  4568 consensus_peers.cc:597] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70fb561fb8b446969622ee2193d90917. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:07.794556  5857 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:07.797036  4679 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:07.801599  5857 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:07.802436  4756 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:07.808063  5857 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:07.819630  6491 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:25:07.844105  6411 raft_consensus.cc:2955] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:07.850788  4748 raft_consensus.cc:2955] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:07.895592  4499 catalog_manager.cc:5697] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:07.900120  4748 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:07.901052  4672 raft_consensus.cc:2955] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:07.906597  6002 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:07.951094  4673 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:07.963780  4748 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:07.970037  5857 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:07.973038  4597 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:25:07.982957  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 52603us
I20260709 12:25:07.989223  5344 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:07.991261  6436 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:07.992060  6450 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:08.004446  4673 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:08.006330  6414 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:25:08.017719  4719 consensus_peers.cc:597] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 559316f720c74aa1b50405e2ed791f6a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:08.033541  4755 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:08.036844  6346 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:08.068960  4673 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:08.072485  6415 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:08.119292  4605 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:08.123100  6415 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:08.155334  6415 raft_consensus.cc:2955] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:08.170327  4676 raft_consensus.cc:2955] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:08.184175  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:08.226327  4605 raft_consensus.cc:2955] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:08.236418  4748 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:08.255297  4499 catalog_manager.cc:5697] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:08.287370  6512 ts_tablet_manager.cc:933] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:08.298614  6514 ts_tablet_manager.cc:933] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:25:08.300124  4672 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:08.301339  4597 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:08.301860  6467 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:08.307147  6467 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:25:08.311148  4719 consensus_peers.cc:597] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f682ea56c3b4f2bb6e9413f3d98c6fd. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:08.323693  6512 tablet_copy_client.cc:323] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.194:42599
I20260709 12:25:08.323606  6450 raft_consensus.cc:2955] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:08.325862  4691 tablet_copy_service.cc:140] P 3a5aeedd11fe49938079fe86cb01232c: Received BeginTabletCopySession request for tablet ea533a99927c40d48b894cf00a75301d from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:48198)
I20260709 12:25:08.326440  4691 tablet_copy_service.cc:161] P 3a5aeedd11fe49938079fe86cb01232c: Beginning new tablet copy session on tablet ea533a99927c40d48b894cf00a75301d from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:48198: session id = dbe960c35c84451faf4e5b5c9233fa39-ea533a99927c40d48b894cf00a75301d
I20260709 12:25:08.335086  4691 tablet_copy_source_session.cc:215] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:08.339053  6512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea533a99927c40d48b894cf00a75301d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:08.340107  4605 raft_consensus.cc:2955] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:08.359515  6512 tablet_copy_client.cc:806] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:08.360366  6512 tablet_copy_client.cc:670] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:08.365010  6512 tablet_copy_client.cc:538] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:08.369693  6514 tablet_copy_client.cc:323] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.193:45077
I20260709 12:25:08.384018  6512 tablet_bootstrap.cc:492] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:25:08.388604  4498 catalog_manager.cc:5697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:08.397010  4614 tablet_copy_service.cc:140] P 5c3201048f8346009aefca07428828f3: Received BeginTabletCopySession request for tablet 70fb561fb8b446969622ee2193d90917 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:53448)
I20260709 12:25:08.397701  4614 tablet_copy_service.cc:161] P 5c3201048f8346009aefca07428828f3: Beginning new tablet copy session on tablet 70fb561fb8b446969622ee2193d90917 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:53448: session id = dbe960c35c84451faf4e5b5c9233fa39-70fb561fb8b446969622ee2193d90917
I20260709 12:25:08.406602  4614 tablet_copy_source_session.cc:215] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:08.420974  6514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70fb561fb8b446969622ee2193d90917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:08.426244  4676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:08.426872  4676 raft_consensus.cc:606] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:08.432790  4672 raft_consensus.cc:2955] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:08.438298  6514 tablet_copy_client.cc:806] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:08.440006  6514 tablet_copy_client.cc:670] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:08.446568  4748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:08.447194  4748 raft_consensus.cc:606] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:08.450579  4605 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:25:08.451334  4605 raft_consensus.cc:606] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:08.451870  6514 tablet_copy_client.cc:538] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:08.452920  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:25:08.454738  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:08.455487  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:08.481375  6519 ts_tablet_manager.cc:933] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:08.487454  6514 tablet_bootstrap.cc:492] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:25:08.496204  6512 tablet_bootstrap.cc:492] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:08.497277  6512 tablet_bootstrap.cc:492] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:25:08.497985  6519 tablet_copy_client.cc:323] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.195:41049
I20260709 12:25:08.498382  6512 ts_tablet_manager.cc:1403] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.115s	user 0.033s	sys 0.004s
I20260709 12:25:08.499961  4766 tablet_copy_service.cc:140] P 49277a7653524787a715232ab908bbc6: Received BeginTabletCopySession request for tablet 559316f720c74aa1b50405e2ed791f6a from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:42632)
I20260709 12:25:08.500559  4766 tablet_copy_service.cc:161] P 49277a7653524787a715232ab908bbc6: Beginning new tablet copy session on tablet 559316f720c74aa1b50405e2ed791f6a from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:42632: session id = dbe960c35c84451faf4e5b5c9233fa39-559316f720c74aa1b50405e2ed791f6a
I20260709 12:25:08.502010  6512 raft_consensus.cc:359] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:08.502904  6512 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:25:08.503578  6512 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:08.507356  6512 ts_tablet_manager.cc:1434] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:25:08.510133  4691 tablet_copy_service.cc:342] P 3a5aeedd11fe49938079fe86cb01232c: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-ea533a99927c40d48b894cf00a75301d received from {username='slave'} at 127.0.0.1:48198
I20260709 12:25:08.510576  4691 tablet_copy_service.cc:434] P 3a5aeedd11fe49938079fe86cb01232c: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-ea533a99927c40d48b894cf00a75301d on tablet ea533a99927c40d48b894cf00a75301d with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:08.515332  4766 tablet_copy_source_session.cc:215] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:08.525856  6519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559316f720c74aa1b50405e2ed791f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:08.551359  6519 tablet_copy_client.cc:806] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:08.552225  6519 tablet_copy_client.cc:670] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:08.561714  6519 tablet_copy_client.cc:538] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:08.571058  6519 tablet_bootstrap.cc:492] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:25:08.601303  6519 tablet_bootstrap.cc:492] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:08.602592  6519 tablet_bootstrap.cc:492] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:25:08.603266  6519 ts_tablet_manager.cc:1403] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.020s
I20260709 12:25:08.606287  6519 raft_consensus.cc:359] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:08.607134  6519 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:25:08.607774  6519 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:08.615106  6519 ts_tablet_manager.cc:1434] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 12:25:08.616816  4766 tablet_copy_service.cc:342] P 49277a7653524787a715232ab908bbc6: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-559316f720c74aa1b50405e2ed791f6a received from {username='slave'} at 127.0.0.1:42632
I20260709 12:25:08.617285  4766 tablet_copy_service.cc:434] P 49277a7653524787a715232ab908bbc6: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-559316f720c74aa1b50405e2ed791f6a on tablet 559316f720c74aa1b50405e2ed791f6a with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:08.633474  6514 tablet_bootstrap.cc:492] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:08.634572  6514 tablet_bootstrap.cc:492] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:25:08.635335  6514 ts_tablet_manager.cc:1403] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.148s	user 0.020s	sys 0.010s
I20260709 12:25:08.638584  6514 raft_consensus.cc:359] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:08.639501  6514 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:25:08.640219  6514 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:08.643131  6514 ts_tablet_manager.cc:1434] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:25:08.644927  4614 tablet_copy_service.cc:342] P 5c3201048f8346009aefca07428828f3: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-70fb561fb8b446969622ee2193d90917 received from {username='slave'} at 127.0.0.1:53448
I20260709 12:25:08.645380  4614 tablet_copy_service.cc:434] P 5c3201048f8346009aefca07428828f3: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-70fb561fb8b446969622ee2193d90917 on tablet 70fb561fb8b446969622ee2193d90917 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:08.729688  5345 raft_consensus.cc:1217] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:25:08.762135  5345 raft_consensus.cc:1217] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:08.775208  6346 raft_consensus.cc:993] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:08.775720  6346 raft_consensus.cc:1081] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:08.780902  4753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43876
I20260709 12:25:08.781495  4753 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:08.781874  4753 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:08.789551  4753 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:08.791604  4753 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:08.792641  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:08.793243  4597 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:08.793175  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:08.793941  4676 raft_consensus.cc:3055] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:08.794229  4676 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:08.794960  4676 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:08.796471  4676 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:08.807819  4597 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:08.814033  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:08.815369  6415 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:08.816586  6415 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:08.817528  6415 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:08.819861  5857 raft_consensus.cc:993] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:08.822404  5857 raft_consensus.cc:1081] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:08.827531  4753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:25:08.828063  4753 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:08.828387  4753 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:08.809648  4676 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:08.854964  6514 ts_tablet_manager.cc:933] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:08.856309  6467 raft_consensus.cc:993] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:08.856930  6414 raft_consensus.cc:1081] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:08.858855  4672 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:25:08.859438  4672 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:08.859822  4672 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:08.863803  4753 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:08.868433  4753 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:08.869237  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:08.870743  4605 raft_consensus.cc:3055] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:08.871762  4605 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:08.871342  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:08.872833  4676 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:08.875361  4499 catalog_manager.cc:5697] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 4 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:08.873464  4605 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:08.889519  4605 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:08.891135  6514 tablet_copy_client.cc:323] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.195:41049
I20260709 12:25:08.892902  4766 tablet_copy_service.cc:140] P 49277a7653524787a715232ab908bbc6: Received BeginTabletCopySession request for tablet 4f682ea56c3b4f2bb6e9413f3d98c6fd from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:42632)
I20260709 12:25:08.893479  4766 tablet_copy_service.cc:161] P 49277a7653524787a715232ab908bbc6: Beginning new tablet copy session on tablet 4f682ea56c3b4f2bb6e9413f3d98c6fd from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:42632: session id = dbe960c35c84451faf4e5b5c9233fa39-4f682ea56c3b4f2bb6e9413f3d98c6fd
I20260709 12:25:08.902269  4766 tablet_copy_source_session.cc:215] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:08.908706  4672 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:08.909791  6514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f682ea56c3b4f2bb6e9413f3d98c6fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:08.911520  4676 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:08.912739  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:08.913537  6450 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:08.915413  6450 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:08.916419  6450 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:08.921407  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:08.922091  4597 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:08.926087  4672 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:08.929265  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:08.930485  4748 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:08.930879  4748 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:08.931721  4748 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:08.932894  4748 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:08.939762  5345 raft_consensus.cc:1217] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:25:08.915385  4605 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:09.007592  4748 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:09.013588  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:09.016355  6346 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:09.016920  6346 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:08.992408  4496 catalog_manager.cc:5697] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.017844  6346 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:08.993381  4597 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:09.053937  4499 catalog_manager.cc:5697] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 5, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 5 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:09.064307  6514 tablet_copy_client.cc:806] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:09.065085  6514 tablet_copy_client.cc:670] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:09.070896  6514 tablet_copy_client.cc:538] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:09.095299  6514 tablet_bootstrap.cc:492] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:25:09.134016  6514 tablet_bootstrap.cc:492] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:09.135200  6514 tablet_bootstrap.cc:492] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:25:09.136032  6514 ts_tablet_manager.cc:1403] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.041s	user 0.023s	sys 0.008s
I20260709 12:25:09.140641  6514 raft_consensus.cc:359] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:09.141667  6514 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:25:09.142477  6502 raft_consensus.cc:1064] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:09.142447  6514 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:09.149403  6502 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:09.152789  6514 ts_tablet_manager.cc:1434] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.016s	user 0.000s	sys 0.007s
I20260709 12:25:09.154851  4766 tablet_copy_service.cc:342] P 49277a7653524787a715232ab908bbc6: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-4f682ea56c3b4f2bb6e9413f3d98c6fd received from {username='slave'} at 127.0.0.1:42632
I20260709 12:25:09.155313  4766 tablet_copy_service.cc:434] P 49277a7653524787a715232ab908bbc6: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-4f682ea56c3b4f2bb6e9413f3d98c6fd on tablet 4f682ea56c3b4f2bb6e9413f3d98c6fd with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:09.175803  5344 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:09.178108  6499 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:09.179488  4753 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:09.180792  4597 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:09.181044  6499 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:09.183924  6002 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:09.203816  6499 raft_consensus.cc:2955] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.213743  5344 raft_consensus.cc:2955] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.242645  4496 catalog_manager.cc:5697] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.248277  4753 raft_consensus.cc:2955] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.249816  4597 raft_consensus.cc:2955] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.283963  4676 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:09.305529  5344 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:09.306973  4605 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:09.308717  6499 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:09.314136  6499 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:09.315073  6525 raft_consensus.cc:1064] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:09.318964  6525 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:09.332298  6346 raft_consensus.cc:2955] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.347289  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ea533a99927c40d48b894cf00a75301d with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:09.348759  4605 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:09.349749  6530 raft_consensus.cc:1064] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:09.352497  6530 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:09.334206  5344 raft_consensus.cc:2955] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.358089  6450 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:09.363682  4602 raft_consensus.cc:2955] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.384913  5345 raft_consensus.cc:1217] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:09.392665  4676 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:09.395157  5345 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:09.398694  4605 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:09.401129  6467 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:09.404388  6467 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:09.410215  4676 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:09.411448  4496 catalog_manager.cc:5697] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 4 to 5, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New cstate: current_term: 2 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.414456  6467 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:09.421268  6415 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:09.431542  4737 tablet_service.cc:1558] Processing DeleteTablet for tablet ea533a99927c40d48b894cf00a75301d with delete_type TABLET_DATA_TOMBSTONED (TS 49277a7653524787a715232ab908bbc6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43852
I20260709 12:25:09.442461  4753 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.443187  4602 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:09.445070  6524 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:09.451251  4676 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.461295  5857 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:09.463788  6530 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:09.452940  5345 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.472163  5857 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:09.475728  4679 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:09.498293  4597 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:09.508498  6545 tablet_replica.cc:333] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:25:09.509615  6545 raft_consensus.cc:2243] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:09.510368  6545 raft_consensus.cc:2272] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:09.519371  6414 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:09.520787  6002 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:09.524418  5857 raft_consensus.cc:2955] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.534101  4753 raft_consensus.cc:2955] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.546211  5345 raft_consensus.cc:2955] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.549686  6414 raft_consensus.cc:2955] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.558813  4496 catalog_manager.cc:5697] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.564044  4672 raft_consensus.cc:2955] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.578696  5344 raft_consensus.cc:2955] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.580152  4601 raft_consensus.cc:2955] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.599087  4753 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:09.604598  6545 ts_tablet_manager.cc:1916] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:09.605530  4499 catalog_manager.cc:5697] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.616212  6002 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:09.626122  4672 raft_consensus.cc:2955] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.646303  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:09.664651  6545 ts_tablet_manager.cc:1929] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:25:09.665735  6545 log.cc:1199] T ea533a99927c40d48b894cf00a75301d P 49277a7653524787a715232ab908bbc6: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root/wals/ea533a99927c40d48b894cf00a75301d
I20260709 12:25:09.670218  4488 catalog_manager.cc:5028] TS 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): tablet ea533a99927c40d48b894cf00a75301d (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:25:09.669660  4601 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:09.682852  5344 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:09.684214  4748 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:09.686084  5857 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:09.688783  4756 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:09.698057  5857 raft_consensus.cc:2955] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.704149  6525 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:09.714387  5346 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:09.701275  5344 raft_consensus.cc:2955] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.727425  4672 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:09.727154  4756 raft_consensus.cc:2955] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.734733  6415 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:09.737839  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 70fb561fb8b446969622ee2193d90917 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:09.741411  6467 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:09.741796  4499 catalog_manager.cc:5697] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 reported cstate change: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New cstate: current_term: 1 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.745970  6467 raft_consensus.cc:2955] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.765309  4673 raft_consensus.cc:2955] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.770615  4488 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 559316f720c74aa1b50405e2ed791f6a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:09.772289  4665 tablet_service.cc:1558] Processing DeleteTablet for tablet 70fb561fb8b446969622ee2193d90917 with delete_type TABLET_DATA_TOMBSTONED (TS 3a5aeedd11fe49938079fe86cb01232c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51854
I20260709 12:25:09.782856  4756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:09.792201  5344 raft_consensus.cc:2955] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.797219  6557 tablet_replica.cc:333] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:25:09.798640  6557 raft_consensus.cc:2243] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:09.799512  6557 raft_consensus.cc:2272] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:09.801128  4756 raft_consensus.cc:606] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:09.801649  4498 catalog_manager.cc:5697] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 4 to 5, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.807651  4601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70fb561fb8b446969622ee2193d90917"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:25:09.808362  4601 raft_consensus.cc:606] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:09.810154  6557 ts_tablet_manager.cc:1916] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:09.813220  4672 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:09.813925  4672 raft_consensus.cc:606] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Received request to transfer leadership to TS dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:09.816682  4672 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:09.817301  4672 raft_consensus.cc:606] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:25:09.820222  4756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:09.820832  4756 raft_consensus.cc:606] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:09.822118  4589 tablet_service.cc:1558] Processing DeleteTablet for tablet 559316f720c74aa1b50405e2ed791f6a with delete_type TABLET_DATA_TOMBSTONED (TS 5c3201048f8346009aefca07428828f3 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60018
I20260709 12:25:09.822500  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:09.824994  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:09.825855  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:09.852331  6558 tablet_replica.cc:333] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:25:09.854058  6558 raft_consensus.cc:2243] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:09.855084  6558 raft_consensus.cc:2272] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:09.867974  6558 ts_tablet_manager.cc:1916] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:09.870409  6557 ts_tablet_manager.cc:1929] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:09.871061  6557 log.cc:1199] T 70fb561fb8b446969622ee2193d90917 P 3a5aeedd11fe49938079fe86cb01232c: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root/wals/70fb561fb8b446969622ee2193d90917
I20260709 12:25:09.873059  4486 catalog_manager.cc:5028] TS 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): tablet 70fb561fb8b446969622ee2193d90917 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:25:09.886238  6558 ts_tablet_manager.cc:1929] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:25:09.886859  6558 log.cc:1199] T 559316f720c74aa1b50405e2ed791f6a P 5c3201048f8346009aefca07428828f3: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-0-root/wals/559316f720c74aa1b50405e2ed791f6a
I20260709 12:25:09.888780  4486 catalog_manager.cc:5028] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): tablet 559316f720c74aa1b50405e2ed791f6a (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:25:09.889161  6530 raft_consensus.cc:1064] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:09.892060  6530 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:09.918727  5344 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.923537  6530 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:09.925148  4601 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.925307  4673 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.926769  6530 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:09.927760  6414 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:09.944128  6530 raft_consensus.cc:2955] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.948215  4672 raft_consensus.cc:2955] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.950295  4605 raft_consensus.cc:2955] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.956405  5344 raft_consensus.cc:2955] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:09.972071  4498 catalog_manager.cc:5697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 2 to 3, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.974196  6346 raft_consensus.cc:993] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower dbe960c35c84451faf4e5b5c9233fa39 to start an election
I20260709 12:25:09.981931  6499 raft_consensus.cc:1081] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Signalling peer dbe960c35c84451faf4e5b5c9233fa39 to start an election
I20260709 12:25:09.983755  5346 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
 from {username='slave'} at 127.0.0.1:50638
I20260709 12:25:09.984298  5346 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:09.984648  5346 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:09.992260  5346 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:09.993753  4756 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:10.038386  5346 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:10.041112  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:10.042025  4748 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:10.070329  4748 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:10.072069  5314 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:10.077646  4601 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:10.083854  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:10.084518  4673 raft_consensus.cc:3055] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:10.084823  4673 raft_consensus.cc:740] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:10.085590  6414 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:10.085676  4673 consensus_queue.cc:260] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:10.087004  4673 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:10.089906  6459 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:10.090727  6459 raft_consensus.cc:697] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:10.091924  6459 consensus_queue.cc:237] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:10.095993  4673 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:10.099457  6450 raft_consensus.cc:993] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:10.099978  6450 raft_consensus.cc:1081] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:10.101190  5346 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:10.100453  6563 raft_consensus.cc:2955] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:10.113651  6414 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:10.126837  5344 raft_consensus.cc:2955] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:10.128259  4499 catalog_manager.cc:5697] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 4 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:10.136242  4601 raft_consensus.cc:2955] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:10.148972  4673 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:25:10.149739  4673 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:10.149772  4488 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4f682ea56c3b4f2bb6e9413f3d98c6fd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:10.152545  6551 raft_consensus.cc:993] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:10.153043  6551 raft_consensus.cc:1081] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:10.155122  4673 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.160511  4605 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60032
I20260709 12:25:10.161187  4605 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:10.161593  4605 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:10.167070  4499 catalog_manager.cc:5697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 3 to 4, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:10.170836  4605 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:10.176527  4673 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:10.182307  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:10.182981  4672 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:10.183274  4672 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:10.183979  4672 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:10.185012  4672 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:10.186525  4605 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:10.187641  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:10.188256  4665 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f682ea56c3b4f2bb6e9413f3d98c6fd with delete_type TABLET_DATA_TOMBSTONED (TS 3a5aeedd11fe49938079fe86cb01232c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51854
I20260709 12:25:10.189471  6557 tablet_replica.cc:333] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:25:10.190412  4673 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:10.190974  6557 raft_consensus.cc:2243] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:10.192802  6557 raft_consensus.cc:2272] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:10.195906  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:10.196533  4756 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:10.197048  4756 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:10.198297  5857 raft_consensus.cc:993] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:10.197906  4756 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:10.198781  5857 raft_consensus.cc:1081] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:10.199225  4756 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.205090  6557 ts_tablet_manager.cc:1916] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:10.205873  4753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70fb561fb8b446969622ee2193d90917"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:25:10.209702  4753 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:10.210106  4753 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.188303  4748 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:10.223486  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:10.224119  4605 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.225834  4753 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:10.226594  6557 ts_tablet_manager.cc:1929] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:10.227088  6557 log.cc:1199] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 3a5aeedd11fe49938079fe86cb01232c: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root/wals/4f682ea56c3b4f2bb6e9413f3d98c6fd
I20260709 12:25:10.206861  4756 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:25:10.229296  4486 catalog_manager.cc:5028] TS 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): tablet 4f682ea56c3b4f2bb6e9413f3d98c6fd (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:25:10.229826  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:10.230374  4748 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:10.230463  4601 raft_consensus.cc:3055] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:10.229807  4753 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:10.230804  4647 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:10.230993  4601 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:10.231735  6499 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:10.232241  6499 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:10.231880  4601 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:10.229328  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:10.233243  5344 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.233604  4601 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.233314  6499 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:10.236966  4571 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:10.241894  4601 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:10.243396  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:10.245323  6415 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:10.245877  6415 raft_consensus.cc:697] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:10.245987  4672 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:10.246148  5857 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:10.251121  5857 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:10.251987  5857 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:10.249928  4605 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:25:10.247052  6415 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:10.279528  4500 catalog_manager.cc:5697] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:10.289891  4499 catalog_manager.cc:5697] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:10.297751  5344 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:10.300697  6415 raft_consensus.cc:993] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:10.301164  6563 raft_consensus.cc:1081] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:10.301330  4500 catalog_manager.cc:5697] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 1 to 2, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 2 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:10.302968  4672 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:25:10.303501  4672 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:10.303829  4672 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:10.311709  4672 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:10.314163  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:10.314180  4672 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:10.314700  4748 raft_consensus.cc:3055] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:10.315013  4748 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:10.315143  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:10.315838  5344 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:10.315774  4748 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:10.317072  4748 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:10.323761  5344 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:10.325239  4644 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:10.325356  4748 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:10.326231  6499 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:10.326893  6499 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:10.327747  6499 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:10.338558  4500 catalog_manager.cc:5697] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 2 to 3, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:10.477919  4672 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:25:10.479313  6564 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:10.500403  4753 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:25:10.504444  6459 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:10.670367  5344 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:10.671630  6563 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:10.675136  4605 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:10.677886  6502 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:10.730015  4604 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:10.732432  6563 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:10.741729  4753 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:10.757995  6499 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:10.763485  5346 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:10.765826  6502 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:10.786005  4756 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:10.788231  4756 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:10.789484  6525 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:10.790438  6499 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:10.811942  4675 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:10.813833  6524 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:10.914497  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:10.956975  4604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:25:10.957662  4604 raft_consensus.cc:606] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:10.960537  4675 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:10.961045  4675 raft_consensus.cc:606] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:25:10.964780  4675 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:10.965332  4675 raft_consensus.cc:606] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:25:10.967873  4748 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:10.968393  4748 raft_consensus.cc:606] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:10.969810  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:10.971050  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:10.971695  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:11.001585  6563 raft_consensus.cc:993] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:11.002204  6530 raft_consensus.cc:1081] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:11.003665  4675 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:25:11.004245  4675 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:11.004608  4675 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:11.014335  4675 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:11.016850  4675 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:11.017408  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:11.018118  4604 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:11.018118  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:11.018989  4748 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:11.019263  4748 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:11.019897  4748 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:11.020853  4748 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:11.025156  4604 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:11.026690  4645 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:11.026863  4748 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:11.027424  6575 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:11.027938  6575 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:11.028846  6575 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:11.041929  4500 catalog_manager.cc:5697] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 3 to 4, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 4 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:11.055204  6524 raft_consensus.cc:993] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:11.055817  6524 raft_consensus.cc:1081] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:11.057264  4756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:25:11.057732  4756 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:11.058007  4756 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:11.062777  4756 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:11.064477  4756 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:11.065608  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:11.066182  4604 raft_consensus.cc:3055] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:11.066478  4604 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:11.066380  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:11.067031  4675 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:11.067215  4604 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:11.068554  4604 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:11.075659  4675 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:11.076879  4604 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:11.077273  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:11.083400  6530 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:11.083968  6530 raft_consensus.cc:697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:11.085071  6530 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:11.090265  6575 raft_consensus.cc:993] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:11.090827  6499 raft_consensus.cc:1081] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:11.092535  4604 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60032
I20260709 12:25:11.093173  4604 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:11.093652  4604 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:11.107990  6499 raft_consensus.cc:993] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:11.108580  6499 raft_consensus.cc:1081] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:11.124811  4605 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60032
I20260709 12:25:11.125463  4605 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:11.121655  4604 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:11.561931  4604 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:11.126417  4605 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:11.589157  4605 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:11.593410  4605 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:11.768131  4599 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:11.788430  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:11.789361  5346 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:11.790831  6563 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:11.816020  5346 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:11.828118  4568 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:11.832662  6525 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:11.845325  6525 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:11.847019  6525 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:11.857439  4675 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:11.874641  6415 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:12.026971  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:12.166600  6524 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.320693  6524 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.337815  6618 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:12.338454  6618 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.340375  6629 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.375996  4672 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
W20260709 12:25:12.398957  4600 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.009s	sys 0.006s
W20260709 12:25:12.399389  4600 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.009s	sys 0.006s
I20260709 12:25:12.394958  6589 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.371567  6629 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.402630  6624 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.436841  6618 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:12.500108  6589 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.533070  6630 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.533869  6630 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.527851  6582 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.535503  6582 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.536623  6630 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:12.537592  6603 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.538305  6603 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.540377  6590 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.540654  6603 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:12.542574  6642 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.543284  6599 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.543345  6642 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.544180  6599 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.546684  6602 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.547312  6602 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.548774  6589 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.552316  6613 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.553197  6589 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:12.557181  6609 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:25:12.578622  4500 catalog_manager.cc:5697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:25:12.518438  6615 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.552994  6613 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.613863  6414 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.614570  6414 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.615242  6613 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.616739  6621 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.617095  6414 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:12.617669  6414 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.617336  6621 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.618202  6414 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:12.608227  4702 debug-util.cc:398] Leaking SignalData structure 0x7b080016dde0 after lost signal to thread 3434
I20260709 12:25:12.541186  6590 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.519326  6624 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.529698  6588 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.610982  6629 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:12.611282  6622 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.612182  6630 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.612830  6642 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.557602  6598 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.628728  5344 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:12.630441  6631 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:12.762698  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 127382us
I20260709 12:25:12.626947  6624 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:12.623191  6590 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.625229  6582 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:12.621521  4702 debug-util.cc:398] Leaking SignalData structure 0x7b080016b4c0 after lost signal to thread 4536
I20260709 12:25:12.557612  6589 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.620468  6414 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:12.626039  6640 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:12.619829  6621 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:12.769241  6642 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.793710  6650 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:12.794036  6614 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.794334  6650 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.774057  6603 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:12.803242  4720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.009s	sys 0.011s
W20260709 12:25:12.803633  4720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.009s	sys 0.011s
I20260709 12:25:12.803339  6603 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:12.803944  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 932152us
I20260709 12:25:12.777837  6582 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:12.805945  6645 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.805848  6582 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.806497  6645 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.807672  6607 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.808411  6582 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.798432  6643 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.798790  6625 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.799240  6635 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.799292  6652 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.800422  6650 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:12.796427  6624 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.626667  6588 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.801708  6647 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.821552  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:12.638651  6622 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.802186  6599 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.775027  6598 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.828358  6598 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.808594  6645 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:12.777650  6590 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:12.829649  6590 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.808231  6607 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.630097  6615 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.814324  6603 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:12.794608  6614 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.822265  5345 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
W20260709 12:25:12.817950  4702 debug-util.cc:398] Leaking SignalData structure 0x7b0800335040 after lost signal to thread 4630
I20260709 12:25:12.762123  6630 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.849649  6645 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.862884  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:12.863123  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:12.773623  6631 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.863706  4597 raft_consensus.cc:2393] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:12.866475  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:12.811954  6642 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.860287  6616 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.820502  6624 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.793138  6640 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.874827  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:12.875998  4748 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:25:12.876718  6634 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.877339  6648 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:12.877300  6634 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.878005  6648 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.880488  6590 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.870478  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:12.864060  6653 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.880422  6648 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:12.900631  6634 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.845153  6598 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:25:12.900960  6655 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:12.901955  6599 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:12.906267  6602 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.902839  6642 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.815552  6625 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.917171  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:12.917455  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:12.917759  4672 raft_consensus.cc:3055] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:12.906136  6607 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.907097  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:12.907769  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:12.907694  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:12.817010  6635 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.926509  6635 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.902681  6661 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.926898  6655 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.928532  4604 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:12.815832  6643 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.911891  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:12.871989  6645 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.940164  6588 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.904659  6622 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:12.953153  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:12.915895  6599 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:12.875293  6616 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.904644  6624 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:12.918179  4672 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:13.008525  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.008690  4672 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:13.009510  6616 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.009896  4672 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:13.010627  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.012238  6639 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:13.013164  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.010636  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.013451  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:12.918681  4756 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:12.920267  5347 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:12.920974  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:12.922151  6589 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.013830  4679 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:12.831995  6647 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.927516  6661 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.823262  6652 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:12.945973  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:12.915251  4702 debug-util.cc:398] Leaking SignalData structure 0x7b0800303560 after lost signal to thread 4706
I20260709 12:25:12.959702  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.006398  6660 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:13.007110  6599 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:13.007220  6659 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:12.922909  6631 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.146140  6643 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:13.145792  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:12.942220  6524 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.930723  6603 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.942564  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:12.942962  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.141114  4748 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:25:12.946489  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:12.941990  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.019148  6621 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:25:13.148120  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 130464us
I20260709 12:25:13.012735  6639 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:13.017719  6588 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.148495  4672 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:12.944267  4597 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:13.150933  4681 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:13.151234  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.017081  6525 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:13.152042  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:12.951978  6640 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:12.929987  6634 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:12.944960  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:12.926549  6614 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.154758  6415 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:13.154081  6662 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:12.938609  6655 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:13.159212  4599 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:13.159307  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.150287  6650 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:13.166476  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:12.917469  6653 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.175050  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:13.174227  6624 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:13.194808  6415 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:13.189842  4702 debug-util.cc:398] Leaking SignalData structure 0x7b08002adfe0 after lost signal to thread 4780
W20260709 12:25:13.231361  4702 debug-util.cc:398] Leaking SignalData structure 0x7b08003766a0 after lost signal to thread 5373
W20260709 12:25:13.231791  4702 debug-util.cc:398] Leaking SignalData structure 0x7b080009ae00 after lost signal to thread 6459
I20260709 12:25:13.177469  6593 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:13.183109  4752 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:25:13.159969  6589 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.187486  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.188180  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.191246  4681 raft_consensus.cc:606] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:13.189666  5314 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:13.239297  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 49740.6us
I20260709 12:25:13.193781  6621 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:13.199048  6588 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.201161  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.201823  6662 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.210170  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.212167  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.181433  6622 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:13.242942  6593 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.246901  6622 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.247500  6623 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:25:13.249214  6622 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.250072  6645 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.251348  6621 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:13.212057  6616 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:13.252723  6616 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.215525  6525 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.253743  4755 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:13.214962  4676 raft_consensus.cc:3055] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:13.256177  4676 raft_consensus.cc:740] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:13.256914  4676 consensus_queue.cc:260] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.172593  6660 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:13.177420  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.173484  6659 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:13.218688  6624 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.219425  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:13.220122  6650 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.244410  6589 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.273147  6650 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.273674  6589 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.274675  6588 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.274979  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.276213  4605 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:13.222803  6415 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.185609  6603 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.207408  6634 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.226788  6615 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.283516  6589 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:13.283874  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.284108  6589 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.284981  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.228091  6598 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:13.229391  6625 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.207010  6643 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:13.234022  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.238339  4747 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:13.243165  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.288051  6643 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.243428  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.289527  4748 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:13.245968  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.290437  4752 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:25:13.257731  4676 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:13.265349  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.268850  6642 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:13.270226  4749 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:13.257776  6602 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:13.254846  6616 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.324350  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.325282  4755 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:13.275115  6628 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:13.347292  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.317801  6602 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:13.372006  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.372493  6664 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:13.372670  4751 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:13.348799  6635 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:13.328646  6642 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.372139  6603 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.376210  4749 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:13.286685  6598 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.353114  6588 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:13.363332  6602 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:13.373011  6664 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.380466  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:13.369923  4571 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.763s (SENT) [suppressed 5 similar messages]
W20260709 12:25:13.382079  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 136866us
I20260709 12:25:13.384007  4755 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:25:13.379534  4676 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:13.369493  6645 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:13.385399  6645 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.385883  6652 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.386266  6624 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.383164  6588 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:13.387531  6645 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.391558  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.376933  6623 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:13.369369  6634 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.407053  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.407972  4756 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:25:13.407132  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:13.383380  6589 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.389488  6525 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.357086  6628 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:13.383755  6635 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.410959  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.461925  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:13.463567  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.464016  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.464730  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.437897  6593 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.468204  6635 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.434028  6662 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.469728  6642 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.430529  6643 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:13.470867  6666 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.059s	user 0.000s	sys 0.002s
I20260709 12:25:13.431700  6661 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.430873  6653 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.472707  6598 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.447337  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:13.474104  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.474747  6659 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:13.428179  6623 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:13.475360  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.470881  6628 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.476282  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.476632  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.477180  6623 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:13.494249  6602 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:13.477516  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.451629  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.430750  6588 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:13.446857  6664 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.411588  6624 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:13.440399  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:13.493115  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.494508  6653 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.495515  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:13.437264  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.496934  4676 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:13.554188  4500 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52808 (request call id 258) took 1163 ms. Trace:
I20260709 12:25:13.578697  4498 catalog_manager.cc:5697] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 2 to 3, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 3 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:13.579697  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:13.497740  6643 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:13.625057  6643 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:13.625945  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.507026  4719 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:13.626796  6563 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:13.627034  6602 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.555698  6639 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:13.627404  6563 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:13.627838  6643 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:13.626861  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:13.641673  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.521339  4597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
W20260709 12:25:13.628150  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 121200us
I20260709 12:25:13.642802  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:13.531077  6660 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:13.627921  6563 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:13.533612  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.629285  4750 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:13.582109  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:13.505317  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.650154  4604 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:13.413194  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:13.621622  6653 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.494540  6630 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.516409  4677 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:13.551407  6647 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:13.531935  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:13.511408  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:13.582861  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:13.657467  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 125623us
I20260709 12:25:13.658264  5314 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:13.551048  6599 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:13.661906  5314 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:13.663067  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:13.664076  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:13.621333  6624 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:13.670334  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.671372  4756 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:13.607465  4499 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52818 (request call id 281) took 1098 ms. Trace:
I20260709 12:25:13.582515  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:13.637140  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.623057  4748 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:13.542912  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.642952  4597 raft_consensus.cc:606] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:13.674479  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.645670  5344 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:13.675037  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.653944  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.655573  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.649328  6642 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:13.656258  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:13.656850  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 243724us
I20260709 12:25:13.660023  6625 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:13.679164  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.660532  6628 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:13.666272  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.687213  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.687353  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.666942  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.668004  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.689415  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.623692  4680 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:13.676420  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:13.678685  4751 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:13.691155  4569 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:13.692302  6628 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:13.692538  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.692756  6628 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:13.693750  6635 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:13.678910  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.694228  6635 raft_consensus.cc:697] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:13.695111  6635 consensus_queue.cc:237] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.623206  4500 rpcz_store.cc:276] 0709 12:25:12.391114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.391269 (+   155us) service_pool.cc:224] Handling call
0709 12:25:12.550516 (+159247us) spinlock_profiling.cc:243] Waited 37.2 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee420090 00007effee4bef3a 00007efff4274e9c 00007efff437304e 00007efff28342a4 00007efff2834232 00007efff2834161 00007efff28340dc 00007efff2833392 00007effec7f148c 00007effec7f08c6
0709 12:25:13.554084 (+1003568us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:12.783349 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:12.783765 (+   416us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:12.783787 (+    22us) write_op.cc:435] Acquired schema lock
0709 12:25:12.783798 (+    11us) tablet.cc:662] Decoding operations
0709 12:25:12.784159 (+   361us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:12.784235 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:12.784265 (+    30us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:12.784461 (+   196us) tablet.cc:701] Row locks acquired
0709 12:25:12.784477 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:25:12.784656 (+   179us) write_op.cc:270] Start()
0709 12:25:12.784775 (+   119us) write_op.cc:276] Timestamp: P: 1783599912784566 usec, L: 0
0709 12:25:12.784792 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 12:25:12.785364 (+   572us) log.cc:844] Serialized 538 byte log entry
0709 12:25:13.269250 (+483886us) op_driver.cc:464] REPLICATION: finished
0709 12:25:13.446627 (+177377us) write_op.cc:301] APPLY: starting
0709 12:25:13.446710 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:13.446812 (+   102us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:13.446936 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:13.447154 (+   218us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:13.447326 (+   172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:13.447374 (+    48us) write_op.cc:312] APPLY: finished
0709 12:25:13.447570 (+   196us) log.cc:844] Serialized 23 byte log entry
0709 12:25:13.447725 (+   155us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:13.447805 (+    80us) write_op.cc:454] Released schema lock
0709 12:25:13.447841 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:13.448157 (+   316us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:13.448185 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":85541632,"child_traces":[["op",{"apply.queue_time_us":177104,"apply.run_cpu_time_us":1533,"apply.run_wall_time_us":1534,"num_ops":1,"prepare.queue_time_us":152232,"prepare.run_cpu_time_us":13105,"prepare.run_wall_time_us":230894,"raft.queue_time_us":114873,"raft.run_cpu_time_us":9421,"raft.run_wall_time_us":138882,"replication_time_us":484308,"thread_start_us":507273,"threads_started":4,"wal-append.queue_time_us":361448}]]}
I20260709 12:25:13.673822  4499 rpcz_store.cc:276] 0709 12:25:12.509311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.509749 (+   438us) service_pool.cc:224] Handling call
0709 12:25:13.607372 (+1097623us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52663,"reported_tablets":5}
I20260709 12:25:13.697809  6657 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:13.676548  6582 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:13.698715  6582 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:13.699131  6582 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:13.699223  6656 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:13.699712  6656 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:13.700088  6656 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:13.700975  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.678280  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.685256  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.703949  5314 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:13.687848  4644 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:13.705462  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:13.705933  4644 consensus_peers.cc:597] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b7b0c6c811e455abe898cb4d21acad4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:13.670010  6599 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.712277  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.716470  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.677115  6637 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:25:13.719005  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 349078us
I20260709 12:25:13.717922  4644 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:13.677366  6624 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:13.679490  6625 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:13.725204  5312 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:13.726861  4569 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:13.727483  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:13.729012  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.730098  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.676985  6647 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:25:13.711053  6653 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:13.733668  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.739711  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.733211  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.716645  6657 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:13.731856  6625 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:13.763863  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.740231  4754 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:13.736142  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.734169  6602 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:13.688210  6642 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:13.754217  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.734613  6623 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:13.755841  5314 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:13.756579  4673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:13.773798  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.774394  4673 raft_consensus.cc:606] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:25:13.759941  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.774827  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.714010  6639 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:13.762342  4753 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:13.792714  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.794013  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.795179  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.796303  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.796593  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.797884  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.798123  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.800340  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.801877  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.802533  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:13.803246  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.806106  6653 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:13.811148  6637 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:13.842376  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.837934  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.765976  6588 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:13.854926  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.877213  4644 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:13.927210  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.934458  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.964238  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.795120  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.931303  6647 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.009927  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:13.854107  6588 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:13.931017  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.985862  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:13.986346  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:13.986515  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 230764us
I20260709 12:25:13.987517  4647 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.018487  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:13.990018  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:13.989847  6628 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:13.990696  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:13.992654  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 190216us
I20260709 12:25:13.994153  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:14.005159  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:14.022195  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.013165  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:14.012566  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 135460us
I20260709 12:25:14.023348  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.011487  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:14.028507  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:14.030480  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.030866  4644 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:14.032557  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:14.035188  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:14.035856  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 309060us
I20260709 12:25:14.036633  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:14.046044  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:14.046707  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:14.047907  6664 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.049160  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:14.023161  6588 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:14.050014  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:14.022364  6658 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.024585  6612 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:14.037550  6599 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.034569  6668 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.058591  4644 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:14.059772  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:14.051014  6647 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.043182  6646 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.034417  6630 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:14.032998  6617 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:25:14.069233  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:14.074290  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:14.068917  6660 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:25:14.080772  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:14.095333  6582 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:14.069231  6630 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:14.101058  6582 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.072172  6658 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.103618  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:14.095798  6639 raft_consensus.cc:3055] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:14.063819  6599 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:25:14.106055  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 81165.2us
W20260709 12:25:14.114732  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 93489.1us
I20260709 12:25:14.117825  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:14.120046  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.101785  6630 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:14.126092  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 81460.2us
I20260709 12:25:14.130981  4644 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:14.131572  6646 raft_consensus.cc:2955] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:14.106946  6642 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:14.134083  6675 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:25:14.134949  6679 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.135890  6651 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.137306  6628 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.122689  6643 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.139676  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.140467  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:14.123020  6599 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.143567  6643 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.143677  6588 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.145175  6599 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.145962  6588 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.110163  6639 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.144961  6625 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.148308  6639 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:14.149055  4751 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:14.149746  6639 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:14.152771  6625 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:14.154546  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:14.156628  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.157258  6625 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.160970  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:14.164520  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.167076  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.176630  6624 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.177910  4750 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:14.181427  4647 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:25:14.182720  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 55917.1us
I20260709 12:25:14.187510  6612 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.188453  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:14.197209  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.198165  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.198482  6643 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.199658  6624 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:14.199862  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:14.206105  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 97608.8us
I20260709 12:25:14.206687  6664 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.201745  6669 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:14.207666  6669 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:14.212673  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:14.219125  6643 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:14.218412  4596 raft_consensus.cc:2955] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
W20260709 12:25:14.222088  6656 consensus_meta.cc:287] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.522s	user 0.017s	sys 0.004s
I20260709 12:25:14.222489  6656 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:14.224438  6656 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.230798  6669 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.206861  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.232170  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:14.234337  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.234969  5343 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:14.235466  6612 raft_consensus.cc:993] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:14.235709  6625 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:14.236464  6664 raft_consensus.cc:1081] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:25:14.237227  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.238535  6643 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.238452  6647 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:14.239584  6686 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:14.239939  6686 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.236498  4647 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.240617  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.241168  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.240847  6686 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:14.241849  4647 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.242053  6686 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:14.242801  6664 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:14.242801  6653 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.243448  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.244052  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:14.244400  6653 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:14.244647  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.245345  6653 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:14.245464  6599 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:14.246950  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.248560  4600 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60032
I20260709 12:25:14.248807  4720 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:14.249104  4600 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:14.249495  4600 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:14.250885  6624 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:14.251544  6624 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:14.254691  5343 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:14.237957  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.256116  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.257033  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:14.252723  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.257171  4600 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.257850  6624 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.258203  6669 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.258587  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.259209  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.259529  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.261649  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.264340  4647 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.266681  4600 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.265674  5312 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.265879  6640 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:14.267412  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:14.273520  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.271929  4571 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:14.284783  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:14.286428  4571 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:14.287846  4571 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:25:14.269817  6563 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.604s	user 0.024s	sys 0.023s
I20260709 12:25:14.266057  6624 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:14.269140  6653 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.283069  6612 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.298780  6612 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:14.292407  6640 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.290704  6525 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:14.304822  6525 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:14.305179  6525 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:14.290016  6678 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:14.305903  6678 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:14.306270  6678 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:14.268837  6664 consensus_queue.cc:1059] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:25:14.309672  6598 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:14.310525  6664 raft_consensus.cc:3055] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:14.310878  6664 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.311527  4569 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:14.311655  6664 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:14.293052  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.312919  6664 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:14.312712  6525 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.313045  5343 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:14.314126  6647 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:14.314704  6658 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:14.283152  6630 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.315426  6525 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.315837  6658 raft_consensus.cc:3055] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:14.316248  6658 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.316606  6630 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:14.289973  6524 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:14.317103  6524 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:14.317554  6524 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:14.317021  6658 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:14.289914  6625 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:14.319293  6658 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:14.293083  6563 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:14.320380  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:14.321508  4752 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:14.322209  6563 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:14.297741  5312 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.324460  6588 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:14.324995  6588 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:14.325042  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:14.325664  6588 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:14.326048  6685 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:14.326432  6685 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:14.326573  4571 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:14.326823  6685 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:14.313879  4647 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.328191  6525 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:14.328593  6525 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:14.328948  6525 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:14.338752  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.339401  4571 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:14.342903  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.344168  4673 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:14.344525  4673 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.345542  4673 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.347244  4673 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:14.347915  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.351819  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:14.352445  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:14.353588  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 61730.3us
I20260709 12:25:14.355224  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.353089  6647 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:14.357264  6652 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:14.358094  6652 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:14.358477  6652 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:14.361698  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:14.355662  6642 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:14.365975  6642 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:14.366356  6642 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:14.367586  6678 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.355362  6599 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.372288  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:14.372895  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.351451  6640 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.374465  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.375241  6668 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.376121  6679 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:14.383069  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:14.386516  4571 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:14.374506  6647 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.388486  6677 raft_consensus.cc:2764] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:14.395247  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.395491  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:14.396807  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.397738  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.399309  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.400383  6525 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.403919  6689 ts_tablet_manager.cc:933] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:14.406667  6685 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.411403  4677 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:14.425634  6623 consensus_queue.cc:1059] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:14.426668  6623 raft_consensus.cc:3055] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:14.426774  6685 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.427064  6623 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.427917  6623 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.429347  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.414222  4500 catalog_manager.cc:5697] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.419418  4752 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:14.432664  4752 raft_consensus.cc:606] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:14.429180  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:14.434165  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47593.1us
I20260709 12:25:14.435441  4571 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:14.434432  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.436303  6684 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:14.436544  6690 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.436843  6684 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:14.437173  4571 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.437469  6684 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:14.438060  6677 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:14.374846  6646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:48182 (request call id 8) took 2542 ms. Trace:
I20260709 12:25:14.441753  4571 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.443579  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:14.429263  6623 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:14.424489  6678 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.452394  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:14.415935  6647 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.458097  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.461270  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.461457  4571 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.462531  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.370204  6588 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:14.467257  4673 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:14.468619  4569 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:14.463495  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.466202  6592 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.466598  4647 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.473193  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:14.467736  6677 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.411897  6652 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.485541  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.456593  6525 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.469763  6678 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.467024  6563 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.424402  6612 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:25:14.497592  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:14.499388  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.475726  6685 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:14.515537  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.472350  6651 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.522202  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.485351  6694 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.505599  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.517491  4601 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:14.441421  6660 consensus_queue.cc:1059] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:25:14.476132  6624 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.521332  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.487105  6681 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:14.542372  4751 raft_consensus.cc:2376] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4. Re-sending same reply.
I20260709 12:25:14.542997  4748 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:14.545686  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.547099  4756 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:14.552837  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.563126  4748 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:14.564371  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.566129  4571 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.515700  6635 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.572567  4571 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.544747  6685 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:14.582355  6685 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.587266  6678 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:14.588670  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.590222  6677 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:14.590358  4571 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.593389  6693 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.591977  6524 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.606150  4748 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:14.607086  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.570397  6681 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:25:14.614962  6692 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:25:14.615228  6589 raft_consensus.cc:993] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:14.615710  6681 raft_consensus.cc:1081] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:14.574079  6694 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.611994  6678 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:25:14.623306  6660 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:14.626128  4756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:25:14.626840  4756 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:14.626008  4571 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.627259  4756 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:14.629171  6684 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.632330  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.635291  4601 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:14.571480  6642 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.636615  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.643795  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.643795  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.645745  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.650701  6678 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:14.651343  6652 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.652283  6689 tablet_copy_client.cc:323] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.194:42599
I20260709 12:25:14.653868  6588 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.659387  4598 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:14.667330  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 171590us
I20260709 12:25:14.668200  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:14.642599  6635 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.672283  4756 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.645011  6524 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.674728  4756 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:14.675222  6653 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:14.677803  6653 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:14.678205  6682 raft_consensus.cc:3055] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:14.678609  6682 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.678771  6653 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:14.679569  6682 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.680848  4691 tablet_copy_service.cc:140] P 3a5aeedd11fe49938079fe86cb01232c: Received BeginTabletCopySession request for tablet 1b7b0c6c811e455abe898cb4d21acad4 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:48198)
I20260709 12:25:14.681466  4691 tablet_copy_service.cc:161] P 3a5aeedd11fe49938079fe86cb01232c: Beginning new tablet copy session on tablet 1b7b0c6c811e455abe898cb4d21acad4 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:48198: session id = dbe960c35c84451faf4e5b5c9233fa39-1b7b0c6c811e455abe898cb4d21acad4
I20260709 12:25:14.683017  6701 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:14.683529  6701 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.665946  6651 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:14.685006  6651 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.450807  6646 rpcz_store.cc:276] 0709 12:25:11.828226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.297771 (+469545us) service_pool.cc:224] Handling call
0709 12:25:12.416782 (+119011us) log.cc:844] Serialized 499 byte log entry
0709 12:25:14.370801 (+1954019us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":192805,"raft.queue_time_us":309501,"raft.run_cpu_time_us":2549,"raft.run_wall_time_us":96317,"thread_start_us":96328,"threads_started":3,"wal-append.queue_time_us":152316,"wal-append.run_cpu_time_us":1730,"wal-append.run_wall_time_us":1001756}
I20260709 12:25:14.687291  6651 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.688308  6642 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.691227  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.691836  6684 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:14.694406  4678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:25:14.694489  6677 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:14.695093  4678 raft_consensus.cc:606] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.695317  6677 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:14.696273  6677 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.697016  6701 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:14.702666  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:14.703163  6682 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:14.704681  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:14.706010  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:14.707285  6707 log.cc:927] Time spent T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:25:14.708840  6647 raft_consensus.cc:3055] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:14.709283  6647 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.710189  6647 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.711256  6647 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:14.712378  6695 log.cc:927] Time spent T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.093s	user 0.003s	sys 0.001s
I20260709 12:25:14.717497  4601 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:14.719791  6703 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:25:14.724254  4691 tablet_copy_source_session.cc:215] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:14.674465  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:14.726897  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 120776us
I20260709 12:25:14.730029  6701 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.730903  6689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b7b0c6c811e455abe898cb4d21acad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:14.736830  6698 consensus_queue.cc:1059] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:14.738672  6698 raft_consensus.cc:3055] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:14.738961  4747 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:14.739029  6698 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.740547  4597 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:14.741067  4596 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:14.741217  6698 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:14.742527  6698 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:14.744827  4602 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:14.746196  4598 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:14.747953  6677 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:14.747771  6651 consensus_queue.cc:1059] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:14.748625  6639 consensus_queue.cc:1059] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:14.749367  6646 consensus_queue.cc:1059] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:14.749857  6623 raft_consensus.cc:3055] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:14.750265  6623 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.750504  6658 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:14.750851  6658 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.751159  6623 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.751614  6658 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.752349  6623 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:14.753319  6639 raft_consensus.cc:3055] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:14.747722  6599 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:14.753768  6639 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.754580  6599 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:14.755131  6599 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.756116  6599 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.759188  6658 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:14.721681  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:14.754920  6639 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.767138  6639 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:14.730930  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 56594.3us
I20260709 12:25:14.777474  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.777601  6682 consensus_queue.cc:1059] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:25:14.779498  6653 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:14.778414  6682 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:14.784057  6682 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:14.785127  6682 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.786379  6682 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:14.797713  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.797865  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.798837  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.797199  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:14.799537  4752 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:14.799808  4747 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:14.764361  4499 catalog_manager.cc:5697] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.805397  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:14.806138  4598 raft_consensus.cc:3055] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:14.806447  4598 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:14.807140  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.807734  4679 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:14.808140  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.807159  4598 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.810559  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:14.810832  4598 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:14.811187  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:14.811758  4678 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:14.812672  4571 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.805503  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:14.814329  6681 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.814779  4750 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:14.816090  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.817185  6681 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:14.818087  4756 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:14.819576  4596 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:14.819737  4678 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:14.757375  6599 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:14.828413  4571 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:14.829155  6642 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.831187  4498 catalog_manager.cc:5697] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 5 to 6, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 6 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.834972  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:14.836679  4747 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:14.836995  6701 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:14.844677  6701 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:14.845170  6701 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:14.849882  4754 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 12:25:14.835695  4719 consensus_peers.cc:597] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3be428c36f9242c6a2b8aa853f2c8b2c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:14.859043  4647 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:14.860824  6642 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:14.866109  4571 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:14.868276  6642 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:14.868764  6642 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:14.869666  6642 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.872959  6599 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.838394  4679 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:14.838572  6563 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:14.839303  6690 raft_consensus.cc:993] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:14.835968  6689 tablet_copy_client.cc:806] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:14.895778  6690 raft_consensus.cc:1081] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:14.895928  6689 tablet_copy_client.cc:670] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:14.843852  4598 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:14.900621  6689 tablet_copy_client.cc:538] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:14.914628  4598 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:14.917858  6689 tablet_bootstrap.cc:492] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:25:14.920030  4499 catalog_manager.cc:5697] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 4 to 5, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 5 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.921481  6690 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:14.922104  4499 catalog_manager.cc:5697] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 5 to 6, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 6 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.889250  6701 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.930048  4754 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:14.933104  6677 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:14.934231  6677 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:14.934607  6677 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:14.935499  6677 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.936808  6701 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:14.936861  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:14.937322  6677 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:14.941675  6701 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:14.946202  6690 consensus_queue.cc:1059] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:14.946239  6701 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:14.947889  6701 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:14.952486  4680 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:25:14.953044  4680 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:14.953401  4680 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:14.955217  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:14.956209  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:14.950649  6690 raft_consensus.cc:3055] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:14.962426  6690 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:14.999020  4754 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:25:14.998162  4679 raft_consensus.cc:2955] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:15.007513  6690 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:15.008924  6690 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:14.996256  6625 raft_consensus.cc:2955] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:15.010814  6598 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:25:15.023528  4680 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:15.030308  4498 catalog_manager.cc:5697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:15.067276  6689 tablet_bootstrap.cc:492] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:15.069383  4680 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:15.072748  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:15.073336  4754 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:15.073745  4754 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:15.074436  4754 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:15.075383  4597 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:15.075451  4754 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:15.078068  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:15.078644  4598 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:15.083015  4756 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:15.090649  4601 raft_consensus.cc:2955] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:15.093462  4754 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:15.093907  4598 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:15.097604  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:15.099071  6689 tablet_bootstrap.cc:492] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:25:15.108254  4647 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:15.109126  6647 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:15.112305  6647 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:15.113293  6647 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:15.116640  6701 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:15.118359  6689 ts_tablet_manager.cc:1403] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.209s	user 0.029s	sys 0.032s
I20260709 12:25:15.120652  6701 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:15.123117  6701 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:15.125964  4498 catalog_manager.cc:5697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 4 to 5, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 5 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:15.132683  4597 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:15.134513  6679 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:25:15.135672  4719 consensus_peers.cc:597] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e8a74137ba104fe0b465cab6399c4940. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:15.135986  4680 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:15.127117  6689 raft_consensus.cc:359] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:15.138028  6689 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:25:15.138756  6689 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:15.139739  6690 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:15.141247  6689 ts_tablet_manager.cc:1434] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.000s
I20260709 12:25:15.143920  4680 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:25:15.145289  4691 tablet_copy_service.cc:342] P 3a5aeedd11fe49938079fe86cb01232c: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-1b7b0c6c811e455abe898cb4d21acad4 received from {username='slave'} at 127.0.0.1:48198
I20260709 12:25:15.145769  4691 tablet_copy_service.cc:434] P 3a5aeedd11fe49938079fe86cb01232c: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-1b7b0c6c811e455abe898cb4d21acad4 on tablet 1b7b0c6c811e455abe898cb4d21acad4 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:15.149209  6693 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:15.178534  5349 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:15.191654  4679 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:15.198978  6625 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:15.202632  6625 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:15.202225  6690 raft_consensus.cc:2955] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:15.203218  6625 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:15.205499  6689 ts_tablet_manager.cc:933] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:15.208554  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:15.209355  4679 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:15.210608  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:15.210922  4500 catalog_manager.cc:5697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 5, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 5 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:15.213901  6643 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:15.211998  4601 raft_consensus.cc:2955] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:15.214322  6643 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:15.214785  6643 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:15.216923  6625 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:15.220851  4496 catalog_manager.cc:5697] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 5 to 6, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 6 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:15.229207  4679 raft_consensus.cc:2955] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:15.230768  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:15.231024  6643 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:15.231758  5343 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
W20260709 12:25:15.242707  4550 auto_rebalancer.cc:629] Failed to schedule move for tablet 18c2a1115a9a4b4a975dadb63bef992a: Illegal state: Replica 3a5aeedd11fe49938079fe86cb01232c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:25:15.246526  4747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43852
I20260709 12:25:15.247170  4747 raft_consensus.cc:606] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Received request to transfer leadership
I20260709 12:25:15.243582  6643 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:15.250075  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:15.255129  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:15.259670  4747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43852
I20260709 12:25:15.260354  4747 raft_consensus.cc:606] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Received request to transfer leadership
I20260709 12:25:15.268946  5343 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:15.281025  4677 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:15.283303  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:15.284801  4680 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:15.286528  6679 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:15.287282  6679 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:15.304446  6689 tablet_copy_client.cc:323] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.195:41049
I20260709 12:25:15.299896  6625 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:25:15.312139  4679 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:15.316991  6693 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:25:15.318192  4747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43852
I20260709 12:25:15.318899  4747 raft_consensus.cc:606] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Received request to transfer leadership
I20260709 12:25:15.292342  6679 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:15.332248  4766 tablet_copy_service.cc:140] P 49277a7653524787a715232ab908bbc6: Received BeginTabletCopySession request for tablet 3be428c36f9242c6a2b8aa853f2c8b2c from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:42632)
I20260709 12:25:15.332870  4766 tablet_copy_service.cc:161] P 49277a7653524787a715232ab908bbc6: Beginning new tablet copy session on tablet 3be428c36f9242c6a2b8aa853f2c8b2c from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:42632: session id = dbe960c35c84451faf4e5b5c9233fa39-3be428c36f9242c6a2b8aa853f2c8b2c
I20260709 12:25:15.340714  4498 catalog_manager.cc:5697] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index -1 to 2, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 1 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:15.350422  6656 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.351022  6656 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:15.371325  4550 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:25:15.372257  4550 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 3be428c36f9242c6a2b8aa853f2c8b2c already in progress
W20260709 12:25:15.372677  4550 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 3be428c36f9242c6a2b8aa853f2c8b2c already in progress
I20260709 12:25:15.384411  4766 tablet_copy_source_session.cc:215] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:15.388319  6689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3be428c36f9242c6a2b8aa853f2c8b2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:15.392012  6656 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:15.392673  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:15.403018  4756 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:15.406483  6677 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:15.412163  4500 catalog_manager.cc:5697] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 4 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:15.420223  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:15.420898  4674 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:15.427595  5312 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:15.429388  6657 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:15.429867  6657 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:15.430220  6657 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:15.437031  4681 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:15.440373  6625 raft_consensus.cc:993] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:15.440951  6563 raft_consensus.cc:1081] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:15.448118  6690 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:15.452411  4679 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:25:15.455578  4679 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:15.456116  4679 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:15.462767  6657 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:15.474910  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:15.480535  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:15.481141  4673 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:15.489722  4679 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:15.490854  4596 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:15.493590  4673 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:15.494769  5312 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:15.495580  6656 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:15.496066  6656 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:15.496978  6656 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:15.497608  6690 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:15.512753  6657 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:15.518011  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:15.518712  4601 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:15.523772  6689 tablet_copy_client.cc:806] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:15.524410  6689 tablet_copy_client.cc:670] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:15.526829  4601 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:15.528256  4645 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:15.529146  6599 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:15.529670  6599 raft_consensus.cc:697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:15.530496  4597 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:15.530647  6599 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:15.532091  6690 consensus_queue.cc:1059] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:15.532965  6679 raft_consensus.cc:3055] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:15.533316  6679 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:15.534204  6679 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:15.535781  6679 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:15.542963  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:15.549542  6689 tablet_copy_client.cc:538] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:15.550149  4679 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:15.560146  4754 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:15.563304  4756 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:15.564786  6599 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:15.574900  6689 tablet_bootstrap.cc:492] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:25:15.583737  4499 catalog_manager.cc:5697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 2 to 3, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:15.593297  4601 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:15.603519  6599 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:15.587610  4500 catalog_manager.cc:5697] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 4 to 5, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 5 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:15.618726  4679 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:15.627334  4679 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:15.629146  4601 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:15.633526  6563 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:15.634871  6563 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:15.695662  6689 tablet_bootstrap.cc:492] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:15.696624  6689 tablet_bootstrap.cc:492] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:25:15.697324  6689 ts_tablet_manager.cc:1403] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.123s	user 0.014s	sys 0.016s
I20260709 12:25:15.722674  4679 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:15.731601  6679 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:25:15.739583  6563 raft_consensus.cc:3055] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:15.739977  6563 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:15.740792  6563 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:15.741739  6599 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.741889  6563 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:15.742287  6599 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:15.744416  6599 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:15.745244  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:15.745879  4601 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:15.746869  4645 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:15.747938  6647 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:15.749961  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:15.761946  4754 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:15.763684  6647 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:15.764178  6647 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:15.787997  6689 raft_consensus.cc:359] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:15.797063  6689 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:25:15.784328  6647 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:15.800927  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:15.801512  4596 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:15.805014  6743 ts_tablet_manager.cc:933] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:15.809170  4596 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:15.810422  4645 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:15.811523  6740 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:15.797855  6689 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:15.834965  6599 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:15.835546  6599 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:15.837036  6647 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:15.839083  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:15.839692  4756 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:15.842455  6599 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:15.845044  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:15.845993  4601 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:15.846530  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:15.847254  4754 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:15.847705  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:15.863914  6647 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:15.864372  6647 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:15.868041  6647 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:15.869747  4756 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:15.871964  6743 tablet_copy_client.cc:323] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.195:41049
I20260709 12:25:15.873670  4766 tablet_copy_service.cc:140] P 49277a7653524787a715232ab908bbc6: Received BeginTabletCopySession request for tablet e8a74137ba104fe0b465cab6399c4940 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:42632)
I20260709 12:25:15.874199  4766 tablet_copy_service.cc:161] P 49277a7653524787a715232ab908bbc6: Beginning new tablet copy session on tablet e8a74137ba104fe0b465cab6399c4940 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:42632: session id = dbe960c35c84451faf4e5b5c9233fa39-e8a74137ba104fe0b465cab6399c4940
I20260709 12:25:15.875260  6689 ts_tablet_manager.cc:1434] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.154s	user 0.006s	sys 0.001s
I20260709 12:25:15.867372  6599 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.877041  4765 tablet_copy_service.cc:342] P 49277a7653524787a715232ab908bbc6: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-3be428c36f9242c6a2b8aa853f2c8b2c received from {username='slave'} at 127.0.0.1:42632
I20260709 12:25:15.877269  6599 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:15.880148  6599 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:15.882072  4766 tablet_copy_source_session.cc:215] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:15.883164  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:15.877496  4765 tablet_copy_service.cc:434] P 49277a7653524787a715232ab908bbc6: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-3be428c36f9242c6a2b8aa853f2c8b2c on tablet 3be428c36f9242c6a2b8aa853f2c8b2c with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:15.883854  5343 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:15.886227  4644 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:15.886668  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:15.895277  6599 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:15.901479  6599 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:15.903825  6740 raft_consensus.cc:697] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:15.900940  6647 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:15.918486  6740 consensus_queue.cc:237] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:15.940328  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:15.941319  4673 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:15.941296  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:15.942750  4754 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:15.945070  6657 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:15.939677  6647 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:15.946695  6563 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:15.947525  6563 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:15.950563  6743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8a74137ba104fe0b465cab6399c4940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:15.954862  4596 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:15.956195  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:15.958734  6563 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:15.912871  6599 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:15.959782  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:15.960703  5343 raft_consensus.cc:2393] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:15.961944  5349 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEARNER]: Advancing to term 3
I20260709 12:25:15.966732  6745 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:15.967140  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:15.967332  6745 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:15.968412  6745 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:15.969832  6643 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:15.970337  6643 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:15.972533  6643 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:15.975445  5349 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:15.979894  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:15.987655  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:15.988569  4680 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:15.992328  6563 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:15.992978  6563 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:15.994819  6679 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:15.995090  6563 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:15.995566  6679 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:15.999140  4601 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:16.000938  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:16.002758  6625 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:16.003191  6625 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:16.003569  6625 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:16.006767  6679 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:16.007447  6647 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.007669  6599 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:16.008003  6647 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:16.008879  4681 raft_consensus.cc:2368] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5.
I20260709 12:25:16.015965  6647 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:16.017051  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:16.018131  4754 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:25:16.020700  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:16.017199  6647 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.021893  6647 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:16.027945  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:16.028774  4754 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:25:16.032307  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:16.033303  4596 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:16.034835  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:16.038645  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:16.039485  4756 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:16.034796  6599 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:16.042382  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:16.045768  6647 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:16.047260  6647 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:16.047660  6647 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.047987  6647 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:16.046908  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:16.048796  4596 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 6: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:16.040980  6599 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:16.050257  6599 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.050622  6599 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:16.051620  4756 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:16.054581  6743 tablet_copy_client.cc:806] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:16.055472  6743 tablet_copy_client.cc:670] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:16.059628  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:16.060411  4720 leader_election.cc:400] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5.
I20260709 12:25:16.060549  4673 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:16.027015  6625 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:16.060760  4720 leader_election.cc:403] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:16.067518  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:16.068216  4601 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:16.069298  6743 tablet_copy_client.cc:538] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:16.061923  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:16.074107  4596 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:16.064139  6625 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:16.075220  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:16.061894  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:16.076159  4679 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:16.061179  6657 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.078114  6657 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:16.080044  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:16.080320  6657 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:16.080761  5343 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:16.082490  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:16.082540  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:16.083161  4756 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:25:16.083244  5343 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:16.084654  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:16.085988  6657 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:16.086440  6657 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.086833  6657 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:16.091535  5343 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:16.115240  4644 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:16.103020  4756 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 12:25:16.115860  4756 pending_rounds.cc:85] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 6
I20260709 12:25:16.116096  4756 pending_rounds.cc:107] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:25:16.116359  4756 raft_consensus.cc:2889] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:16.103288  6599 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:16.064985  6745 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:25:16.118026  6740 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:16.118134  6647 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:16.118853  6599 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:16.120510  6599 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:16.121012  6599 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:16.120901  6656 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:16.121201  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:16.121793  4596 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:16.121999  6599 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:16.125823  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:16.130720  6642 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.130913  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:16.131505  4747 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:16.131356  6642 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:16.134384  6642 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:16.142982  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:16.143297  6740 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.143743  4756 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:16.144147  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:16.144851  4681 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:16.145815  6647 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:16.147203  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:16.147998  6642 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:16.148408  6642 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.148749  6642 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:16.148875  4601 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:16.151065  6745 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:16.151907  4596 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:16.151943  6598 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.162202  6598 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:16.164441  6598 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:16.166492  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:16.143831  6740 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:16.168172  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 51444.3us
I20260709 12:25:16.168401  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:16.169185  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:16.171154  6740 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:16.172102  6657 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:16.173069  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:16.173527  5343 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 4: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:16.172384  4496 catalog_manager.cc:5697] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:16.173807  4596 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:16.175908  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:16.161190  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:16.176856  6647 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:16.177592  6647 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:16.179014  6750 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:16.179428  6750 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.179768  6750 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:16.180145  6647 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:16.177068  4679 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 4: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:16.184342  6679 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:16.184805  6679 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:16.185209  6679 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:16.154291  4747 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:16.197664  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:16.174114  4754 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:16.209100  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:16.210096  4747 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:16.210175  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:16.212112  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:16.255115  6740 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:16.201284  6563 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5.
I20260709 12:25:16.201462  6745 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:25:16.261823  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 49820us
I20260709 12:25:16.263823  6751 raft_consensus.cc:1064] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:16.274787  4673 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
W20260709 12:25:16.269646  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 64312.2us
I20260709 12:25:16.197294  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:16.287868  4747 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:16.289808  6690 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:16.238055  6642 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:16.298123  6690 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:16.299065  6751 raft_consensus.cc:1068] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Unable to promote non-voter dbe960c35c84451faf4e5b5c9233fa39: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:25:16.315748  6625 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:16.284587  6745 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:16.339175  6690 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:16.456166  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 177311us
W20260709 12:25:16.478679  6756 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:25:16.404027  6679 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:16.482052  6679 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:16.419803  6625 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:16.499768  6740 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:16.528654  6740 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:16.635478  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 348702us
I20260709 12:25:16.766165  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:16.706697  6642 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:16.752753  6657 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:16.759502  6679 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.757792  6745 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:16.935408  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:16.936277  4681 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:16.959352  6677 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.993676  6743 tablet_bootstrap.cc:492] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:25:16.942123  6679 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:16.993248  6767 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.041659  6778 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.042059  6771 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.042270  6778 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:17.042560  6771 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:17.044473  6778 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:17.044607  6771 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:17.046418  6776 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:17.047675  6767 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:17.018919  6677 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:17.031383  6625 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:17.090255  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:17.051765  4571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.007s	sys 0.010s
W20260709 12:25:17.091895  4571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.007s	sys 0.010s
I20260709 12:25:17.093102  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:17.093796  4754 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:17.091002  4756 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:17.097071  4750 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:17.098270  4752 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:17.104444  6679 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:17.065412  6767 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:17.109550  6784 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:25:17.081213  6690 consensus_meta.cc:287] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.643s	user 0.011s	sys 0.000s
W20260709 12:25:17.079514  6750 consensus_meta.cc:287] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.899s	user 0.013s	sys 0.000s
W20260709 12:25:17.105224  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 907939us
I20260709 12:25:17.119905  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:17.121004  4680 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:17.121559  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:17.122236  4674 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:17.055516  6778 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.119973  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:17.136407  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:17.175226  4752 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 4: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:17.185942  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:17.186991  4596 raft_consensus.cc:2368] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7.
I20260709 12:25:17.210218  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:17.211099  4597 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 3: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:17.174552  6677 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:17.228055  4647 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:17.233896  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:17.234571  4756 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:17.239723  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:17.233992  4678 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:17.153990  6771 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:17.160962  6750 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:17.137777  6690 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:17.312203  5313 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.036s	sys 0.020s
W20260709 12:25:17.320788  5313 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.036s	sys 0.020s
I20260709 12:25:17.305271  4756 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:17.260957  6625 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:17.200659  6778 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:17.310164  6771 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:17.301689  6773 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:17.358876  4679 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:17.359977  6773 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:17.362689  6773 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:17.353477  6777 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:17.365242  6777 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:17.368753  6777 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:17.369406  6760 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:17.370249  6760 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:17.326233  6788 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:17.310721  6790 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:17.328737  6599 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:17.319555  6647 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:17.115687  6784 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:17.346724  4681 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
W20260709 12:25:17.407713  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 270822us
I20260709 12:25:17.433452  4569 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:17.443812  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 204185us
W20260709 12:25:17.377538  6755 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 12:25:17.357805  6762 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:17.347654  6625 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:17.668928  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:17.673453  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:17.673909  4748 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:17.674330  4753 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:17.662992  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:17.678748  4750 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:25:17.376463  6788 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.679958  6788 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:17.680825  6798 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:17.681641  6801 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:17.681982  6679 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:17.696585  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:17.376857  6780 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:17.377380  6772 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:17.374190  6764 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:17.417104  6599 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:17.373558  6760 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:17.376853  6790 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:17.704859  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 271503us
I20260709 12:25:17.969378  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:17.970917  4569 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:17.971474  4598 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:17.765525  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:17.983621  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 218048us
I20260709 12:25:17.983448  4747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:17.986951  4597 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:18.002875  4600 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:17.792845  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 139330us
W20260709 12:25:18.011193  4679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2173) took 1817 ms (client timeout 1805 ms). Trace:
W20260709 12:25:18.011849  4679 rpcz_store.cc:269] 0709 12:25:16.194176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.194338 (+   162us) service_pool.cc:224] Handling call
0709 12:25:18.010590 (+1816252us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540038f4d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:18.010922 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2667626368}
I20260709 12:25:18.012789  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:18.014106  4747 raft_consensus.cc:606] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:18.014655  4747 raft_consensus.cc:614] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:18.015398  5344 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:17.990988  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:18.036939  6802 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:18.038105  6802 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.055125  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:18.055763  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 83032.6us
I20260709 12:25:17.452194  6657 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:18.075294  6657 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.076445  6773 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:18.077889  6599 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:18.078665  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 66224.1us
I20260709 12:25:17.891369  6750 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.080163  6790 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.081012  6790 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:18.081143  6750 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:18.083104  6771 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:18.083374  6750 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:17.698433  6762 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.082209  4598 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:18.085592  6757 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:18.085815  6800 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.822449  6781 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:18.086835  6800 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.086639  6757 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:17.472998  6751 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.626s
I20260709 12:25:17.650915  6690 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:17.473481  6745 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.627s
I20260709 12:25:17.682592  6798 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.105612  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:17.764281  6775 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:18.085829  4604 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:18.106213  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:17.818434  6783 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:18.180100  6757 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.171149  6625 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:18.094393  6750 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:17.811255  6784 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.181386  6750 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:17.728415  6780 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.088307  6781 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.091480  6685 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:18.183869  6685 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.091845  6657 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:18.110129  6802 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.186770  6802 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:18.088922  6772 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.176311  4672 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:17.712854  6679 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.182291  6750 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.189843  6677 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.190598  6677 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.099472  6777 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:18.192052  6777 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.192572  6777 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:18.187656  6802 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.194103  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:17.470904  6647 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:18.195178  4674 raft_consensus.cc:2368] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7.
I20260709 12:25:18.196943  6775 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.095595  6774 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:18.198689  6774 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:18.199610  6679 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:17.450757  6793 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:18.100466  6786 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.101046  6795 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:18.202610  6786 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.203531  6795 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.205251  6781 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:18.071740  6656 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:17.820506  6778 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.100008  6782 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:18.179666  6783 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.202347  6793 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:17.866317  6767 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:18.216814  6793 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:17.777417  6797 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:18.094094  6763 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:17.747614  6765 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:17.870939  6791 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:18.087813  6642 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:18.217953  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:18.101022  6787 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:18.255636  6787 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.139465  6773 raft_consensus.cc:697] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:18.258160  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:17.785039  6770 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:18.258718  6773 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:18.260123  6770 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:18.216715  6782 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.262804  6770 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:17.823805  6669 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:18.239578  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:18.240762  6647 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.241206  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:18.242311  6767 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.243144  6810 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:18.245065  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:18.247331  6745 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:18.248116  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 47307.7us
I20260709 12:25:17.772411  6740 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:18.276780  6764 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.283568  4604 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:18.292398  6763 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.272190  6750 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:18.296442  6791 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.297955  6647 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:18.298831  6647 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.295905  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:18.300298  6802 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:18.206233  6656 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.322852  6745 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.296855  6740 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.263952  6669 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.293226  6797 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:17.708025  6801 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.614853  6803 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:18.685631  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:18.661854  4674 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:18.383957  4750 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:18.327817  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:18.687218  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:18.364641  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 50380.1us
I20260709 12:25:18.380052  4748 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:25:18.371698  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 52844.1us
I20260709 12:25:18.691505  6795 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.624346  6748 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:18.331960  6750 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:18.704281  6750 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.707031  6750 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.707039  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:18.617889  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:18.363808  6643 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:18.708227  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:18.696900  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:18.712167  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 384498us
I20260709 12:25:18.615694  6783 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:18.389369  6785 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.340415  6765 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.616170  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:18.618290  6685 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.683861  6800 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:18.617076  6763 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.684338  6782 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:18.333999  6802 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.386s
I20260709 12:25:18.721294  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:18.559275  6764 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.339946  6775 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.707916  6804 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.696580  6748 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.291316  6642 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.363739  6563 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:18.631333  6745 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.726446  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:18.728729  4749 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:25:18.697438  6772 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:18.727085  4677 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 1.
I20260709 12:25:18.346585  6599 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.732699  6795 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:18.728696  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:18.732874  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:18.620093  6822 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:18.331954  6823 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:18.712208  6780 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.686172  6803 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.741756  4755 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:25:18.732398  6765 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:18.743899  5346 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:18.771557  6801 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.772419  6599 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:18.774528  6788 consensus_meta.cc:287] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.092s	user 0.009s	sys 0.009s
I20260709 12:25:18.744622  6643 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.775055  6788 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.776465  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:18.776746  6791 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.776911  6643 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:18.751940  4720 leader_election.cc:400] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7.
I20260709 12:25:18.752882  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:18.777740  4720 leader_election.cc:403] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:25:18.758910  6797 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:18.777097  5347 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:18.744735  6563 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.328603  6810 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.767199  4569 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:18.780236  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:18.745438  6795 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.777192  6788 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.792227  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:18.785800  6777 consensus_meta.cc:287] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.592s	user 0.004s	sys 0.008s
I20260709 12:25:18.741464  6804 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.783771  6795 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:18.798202  4749 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
W20260709 12:25:18.798190  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 59530.4us
W20260709 12:25:18.799460  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 60288.7us
I20260709 12:25:18.802129  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:18.801728  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:18.808471  6818 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:18.799772  6823 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.814654  4644 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:18.816418  4644 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:18.816850  6669 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:18.817731  6815 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7.
I20260709 12:25:18.818586  6599 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:18.819026  6599 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:18.785511  6679 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:18.806732  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:18.781704  6785 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.798758  6771 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:25:18.871896  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 70620.5us
I20260709 12:25:19.018170  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:19.120507  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 43991.4us
I20260709 12:25:18.876235  4598 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
W20260709 12:25:19.073560  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 43929us
I20260709 12:25:19.125818  6801 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.039810  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.127257  6801 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:19.128742  4596 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 1.
I20260709 12:25:18.797123  6745 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:18.819020  6797 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:18.796072  6822 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:18.877485  6599 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:19.137347  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.019454  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:19.078685  6823 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.139470  6797 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.152245  6797 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:19.144184  6745 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:18.804119  6777 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.157361  6777 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.144290  6643 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:19.158630  4679 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:19.147919  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.018540  6679 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.159888  4605 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 1.
I20260709 12:25:19.161837  6679 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:19.155145  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:19.151681  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 165969us
I20260709 12:25:19.151288  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.164649  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.166673  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:19.167438  4598 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:19.158353  6643 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.132630  6656 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.040695  6798 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.130033  6563 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:19.130398  6818 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:19.202517  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 381447us
I20260709 12:25:19.163834  6765 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.332942  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:19.333889  5344 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:19.276338  4598 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
W20260709 12:25:19.308760  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 437989us
W20260709 12:25:19.268289  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 114086us
W20260709 12:25:19.340754  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 90953.6us
I20260709 12:25:19.344676  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:19.203442  6643 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:19.293728  6801 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.338164  6599 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.204628  6762 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.314096  6822 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.275367  6804 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.209380  6803 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:19.252532  6642 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.212977  6760 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.351284  6768 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.206343  6818 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.354102  6763 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.377940  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1367) took 1102 ms. Trace:
I20260709 12:25:19.354347  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.355355  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.394222  4752 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:19.378826  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1690) took 1001 ms. Trace:
I20260709 12:25:19.356106  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.393843  4674 rpcz_store.cc:276] 0709 12:25:18.275518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.280244 (+  4726us) service_pool.cc:224] Handling call
0709 12:25:19.377718 (+1097474us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b540017d754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:19.377865 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68394368}
I20260709 12:25:19.359203  6823 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:19.364748  6798 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:19.357463  6830 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.203712  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:19.358319  6822 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:19.276010  6786 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.210958  6785 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.252524  6748 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.381032  6801 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:25:19.372978  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 381116us
I20260709 12:25:19.349855  6802 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.375118  5312 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:19.384516  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.179140  6771 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.418928  6833 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.338757  6832 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:19.356197  6599 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.395036  4747 rpcz_store.cc:276] 0709 12:25:18.377110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.377584 (+   474us) service_pool.cc:224] Handling call
0709 12:25:18.712605 (+335021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.813976 (+101371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.814027 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:18.814139 (+   112us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:18.814225 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.378532 (+564307us) spinlock_profiling.cc:243] Waited 81.8 ms on lock 0x7b54000a7154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:19.378593 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.378757 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188064000}
I20260709 12:25:19.406946  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.409004  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:19.365491  6810 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.412966  4749 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:19.371259  6797 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.383430  6745 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.413715  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.414593  6823 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.413779  4719 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:19.420433  5347 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:19.413735  6795 consensus_meta.cc:287] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.605s	user 0.009s	sys 0.004s
I20260709 12:25:19.434527  6810 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:19.434454  6795 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:19.436511  6798 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.420713  6760 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.425338  6765 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.426522  4752 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:25:19.426954  6834 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:19.427222  6797 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.427022  6599 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:19.445690  6834 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.447188  5347 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:19.447952  6834 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:19.449162  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.449405  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.450042  4747 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:25:19.451224  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.451514  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.451936  4747 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:19.429013  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:19.436955  6795 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.425776  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:19.442440  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 67390.7us
I20260709 12:25:19.453078  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.453486  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:19.419332  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:19.454638  6797 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:19.455022  6797 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:19.455082  6822 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.421496  6788 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.456110  6788 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.456604  6771 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.455464  6797 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:19.456127  6643 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:19.467257  6762 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:19.467859  6762 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.468146  6823 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.468271  6762 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:19.422158  6802 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.471313  6802 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.471589  6760 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.466377  6788 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.467001  5312 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:19.473287  6780 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:19.473822  6780 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.474255  6780 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.466480  6798 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.423908  6748 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:19.476383  6643 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.477010  6643 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:19.414788  6768 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.457352  6822 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.481956  6768 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.482075  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.483866  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.476816  6748 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.478371  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:19.478291  6762 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.484416  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.457404  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.486136  4647 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:19.465689  6764 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:19.487214  4752 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:19.487394  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.488022  4672 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:25:19.490965  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:19.437788  6830 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.494757  6830 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.496294  6764 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:19.496783  6764 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:19.496892  6822 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:19.497180  6764 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:19.497488  6822 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.498904  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1689) took 1122 ms. Trace:
I20260709 12:25:19.499451  6643 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.499897  6822 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.500996  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:19.501345  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.504994  5312 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:19.492206  6762 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.499269  4754 rpcz_store.cc:276] 0709 12:25:18.376833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.381707 (+  4874us) service_pool.cc:224] Handling call
0709 12:25:19.498197 (+1116490us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b58000d2698. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391eee 000000000036eb39 00007efff34051e0 00007efff34ce0c4 00007efff34ce131 00007efff348ffc0 00007efff347a929 00007efff347a3d2 00007effec9f1bd4
0709 12:25:19.498336 (+   139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.498563 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.498588 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:19.498643 (+    55us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:19.498673 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.498727 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.498828 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10157824}
I20260709 12:25:19.513720  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.509944  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.506902  5312 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:19.510787  6677 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.515652  4750 raft_consensus.cc:2393] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:19.512244  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1364) took 1407 ms. Trace:
I20260709 12:25:19.512364  4752 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:19.508373  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.419946  6763 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:19.512090  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.516067  4680 rpcz_store.cc:276] 0709 12:25:18.105019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.105248 (+   229us) service_pool.cc:224] Handling call
0709 12:25:18.105751 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.732838 (+627087us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.732885 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:18.732945 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:18.733002 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.511901 (+778899us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b540014fad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:19.511946 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.512058 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1405418752}
I20260709 12:25:19.530750  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.531862  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:19.514226  6748 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.515132  6769 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:19.519187  6793 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:19.521469  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.540921  4747 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:19.522579  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.533485  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.541232  5312 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:19.532341  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:19.514819  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.534255  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.543277  4674 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:25:19.535957  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:19.536016  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 75034.4us
I20260709 12:25:19.544358  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:19.536394  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.441921  6801 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.548151  6811 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:19.536164  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:19.549392  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.533417  6762 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:19.545843  6791 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:19.563983  6834 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:19.564355  6791 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:19.564669  6834 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.545140  4754 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:25:19.550246  4596 raft_consensus.cc:2368] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8.
I20260709 12:25:19.459199  6833 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.554536  4597 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:25:19.557056  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:19.557710  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.558645  6841 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:19.559558  6822 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:19.561107  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:19.562129  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.563691  6643 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.546416  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.584043  4604 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:25:19.569499  6834 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:19.570080  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.588405  4599 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:19.588465  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:19.570953  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.572244  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:19.544138  6793 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.569104  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:19.589411  4596 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:19.555596  6762 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.592157  4647 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:19.583832  6769 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:19.602048  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:19.602867  6804 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:19.607995  5312 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:19.610049  5348 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:19.594310  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:19.586467  6768 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:19.610451  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.595280  6793 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:19.597927  6762 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:19.619153  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49163.6us
I20260709 12:25:19.621006  4645 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:19.621435  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.614921  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:19.609786  6769 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.565447  6791 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.625840  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:19.626184  6830 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:19.615163  6802 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:19.628283  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.610828  6788 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:19.630236  4604 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:19.611776  6804 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:19.637902  6804 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.591886  6780 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:19.641155  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 46975.5us
I20260709 12:25:19.614360  6745 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:19.634238  6838 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:19.644728  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:19.646325  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.625905  6769 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:19.616025  6677 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:19.652052  6814 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:19.625862  6819 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:19.628218  6801 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.640403  6768 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:19.650547  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 118674us
I20260709 12:25:19.664997  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.631973  6788 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:19.667547  5314 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:19.621222  6774 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.669107  6647 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.671818  6833 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.643703  6830 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:19.672645  6830 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.673480  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.669938  6811 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:19.669407  6788 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:19.643658  6745 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:19.674134  4603 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:19.681768  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:19.690294  6830 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.691603  6774 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:19.692233  6774 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.659711  6642 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.701762  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:19.664499  6819 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:19.666960  6768 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.691033  6830 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.666164  6783 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:25:19.749207  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:25:19.724035  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 80763.7us
I20260709 12:25:19.749202  5345 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:19.754802  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1693) took 1375 ms. Trace:
I20260709 12:25:19.763201  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.763883  6642 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.766847  6642 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.767189  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.767604  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.768056  6778 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:19.768409  4674 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:19.768466  6778 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.769093  6778 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:19.769616  6783 raft_consensus.cc:3055] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:19.770026  6783 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:19.770942  6783 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.772814  6783 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:19.676496  6794 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:19.754482  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1362) took 1650 ms. Trace:
W20260709 12:25:19.715732  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 58498.6us
I20260709 12:25:19.682827  6801 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:19.754856  6762 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.786799  4571 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:19.675061  6814 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.776697  4678 rpcz_store.cc:276] 0709 12:25:18.103865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.104368 (+   503us) service_pool.cc:224] Handling call
0709 12:25:18.104812 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.721536 (+616724us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.721589 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:18.729161 (+  7572us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:18.729212 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.754147 (+1024935us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b540017eb54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:19.754190 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.754309 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1410326912}
I20260709 12:25:19.791801  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:19.796816  5344 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:19.797807  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:19.775185  6768 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:19.762540  4751 rpcz_store.cc:276] 0709 12:25:18.379245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.380332 (+  1087us) service_pool.cc:224] Handling call
0709 12:25:18.380687 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.203891 (+823204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.203938 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:19.204043 (+   105us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:19.204100 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.421717 (+217617us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b1800340ea0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3972a6 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419
0709 12:25:19.754489 (+332772us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b54001f2554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:19.754534 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.754646 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22675,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":143,"spinlock_wait_cycles":1842825344}
I20260709 12:25:19.776381  6819 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:19.696219  6745 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:19.803401  6844 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:19.805374  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:19.805989  4680 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:19.808571  4720 leader_election.cc:400] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7.
I20260709 12:25:19.790732  6814 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.799850  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:19.809996  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.811312  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.794195  6763 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.813701  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:19.815052  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.815160  6690 consensus_queue.cc:1059] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:25:19.816274  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.817667  6774 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.819263  6764 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.815063  6762 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.848515  6764 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:19.778296  6830 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.809955  6642 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:19.823734  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:19.824069  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.824277  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.825203  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:19.826462  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.826646  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:19.827566  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1342) took 1583 ms. Trace:
I20260709 12:25:19.827253  6788 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.829035  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.795742  6801 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:19.871402  6801 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.873067  4754 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:19.682168  6677 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.830029  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.833762  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.863999  4602 rpcz_store.cc:276] 0709 12:25:18.244339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.244513 (+   174us) service_pool.cc:224] Handling call
0709 12:25:18.244846 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.336277 (+ 91431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.336308 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:18.336366 (+    58us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:18.336404 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.827234 (+1490830us) spinlock_profiling.cc:243] Waited 91.2 ms on lock 0x7b540032b9d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:19.827276 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.827402 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209681280}
I20260709 12:25:19.834674  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.834982  6784 raft_consensus.cc:1064] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:19.880056  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.880795  4673 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:25:19.835382  6679 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:19.835462  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.799505  6832 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:19.836445  5346 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:19.833612  4680 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:19.838053  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:19.838351  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.839191  6563 raft_consensus.cc:3055] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:19.839632  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.846196  6780 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.823072  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:19.851121  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:19.851310  6787 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.851411  6791 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:19.853444  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:19.899156  4678 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:19.857707  6740 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.862142  4752 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
W20260709 12:25:19.863332  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 61092.8us
I20260709 12:25:19.861775  4747 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:19.862807  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:19.864436  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 64674.3us
W20260709 12:25:19.859665  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 215045us
I20260709 12:25:19.868860  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.834483  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.901546  4677 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:19.879159  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.858650  5343 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:19.896500  4681 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:19.891707  5312 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:19.892830  6563 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:19.906715  4672 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:19.907245  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:19.907649  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.907473  6563 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.861196  6642 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:19.902560  4755 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:19.892917  6760 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.889282  6679 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:19.906708  4647 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:19.903306  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.911343  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.911419  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.911513  6563 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.908244  4599 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:19.834672  6778 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.926420  4602 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:19.917436  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:19.911968  6775 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:19.921341  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.929844  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.930519  4596 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:19.921161  6760 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.921929  6679 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.912662  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:19.928375  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.930258  4647 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:19.882056  6784 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:19.931566  4720 leader_election.cc:400] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8.
I20260709 12:25:19.898881  6791 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:19.928966  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.931476  5345 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEARNER]: Advancing to term 4
I20260709 12:25:19.936164  4720 leader_election.cc:403] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:19.941452  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.920392  6642 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:19.944566  6822 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:19.944836  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.940528  6775 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:19.944597  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.948144  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.948757  4679 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:19.952013  6778 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.954413  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:19.956070  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:19.933876  6832 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:19.957680  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:19.958314  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.934216  6740 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:19.941780  6791 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:19.963373  4605 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:19.914160  6793 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:19.967437  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.963055  6830 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:19.981720  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:19.981832  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:19.983690  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:19.966542  6740 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:19.990221  6740 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:19.969875  6822 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:19.958437  6677 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:19.958243  6643 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8.
I20260709 12:25:19.985275  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:20.002715  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:19.963039  6690 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:20.008400  4599 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:19.961040  6832 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:20.064236  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 55571.7us
I20260709 12:25:20.074093  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:20.085837  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:20.088407  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:20.088434  6690 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:20.064502  6822 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:20.088119  6832 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:20.090701  6834 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:20.113862  6834 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:20.114336  6834 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:19.973440  6767 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:20.145351  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:20.146409  4598 raft_consensus.cc:2368] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6.
I20260709 12:25:20.104522  4755 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:20.104743  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:20.096935  6690 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:20.094419  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:20.106215  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:20.107897  6793 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:20.109766  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:20.110941  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:20.089814  4753 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:20.011320  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:20.094149  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:20.161319  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:20.168329  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:20.009305  6785 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:20.084635  6788 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:20.149799  4755 raft_consensus.cc:606] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:20.170092  4605 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:20.170976  4598 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:20.169332  4751 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 1.
I20260709 12:25:20.169667  4602 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:20.178963  4756 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:20.200264  4568 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:20.209244  6643 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:20.209896  6643 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:25:20.210027  6851 log.cc:927] Time spent T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:25:20.210318  6643 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:20.210932  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 100070us
I20260709 12:25:20.008927  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:20.216547  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:20.086858  6769 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:20.220674  4596 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:20.177522  6785 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:20.229547  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:20.241874  6791 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:20.246748  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:20.251982  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:20.114357  6775 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:20.253888  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:20.216259  6760 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:20.256812  4753 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:25:20.256878  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:20.260387  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:20.230695  6785 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:20.257601  4749 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:20.228242  6830 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:20.262962  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:20.300619  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 71123.5us
I20260709 12:25:20.302791  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:20.303761  4605 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:25:20.252249  6791 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:20.261122  6793 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:20.294674  5342 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:20.313717  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:20.253322  6787 consensus_queue.cc:1059] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:25:20.318128  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:20.319520  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:20.321815  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:20.326303  4647 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:25:20.331604  6804 consensus_meta.cc:287] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.693s	user 0.009s	sys 0.018s
I20260709 12:25:20.331382  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:20.332569  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:20.327708  4647 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:20.338515  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:20.316423  6793 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:20.356217  5345 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:20.324901  6791 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:20.381889  6791 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:20.374744  6801 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:25:20.382408  6791 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:20.383596  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:20.384287  4679 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:20.377384  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:20.358700  4752 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:20.385996  4748 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:20.333108  6823 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:20.404114  6854 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:25:20.415828  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:20.417553  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:20.423913  6852 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 12:25:20.373018  6797 consensus_meta.cc:287] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.866s	user 0.010s	sys 0.004s
W20260709 12:25:20.385121  6783 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.608s	user 0.013s	sys 0.018s
I20260709 12:25:20.461830  4496 catalog_manager.cc:5697] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 5 to 7, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 7 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:20.473225  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:20.474123  6797 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:20.475787  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.399173  6788 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:20.378736  6769 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:20.522069  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:20.526834  4674 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:20.374573  6853 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.136s
I20260709 12:25:20.529973  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:20.385035  6804 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:20.416883  6823 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:20.404742  6808 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.122s
I20260709 12:25:20.510025  6788 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:20.560149  4747 consensus_meta.cc:287] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.646s	user 0.011s	sys 0.025s
W20260709 12:25:20.560259  6563 consensus_meta.cc:287] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.631s	user 0.015s	sys 0.023s
I20260709 12:25:20.560552  4747 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:20.568949  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:20.556995  6823 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:20.559614  6788 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:20.593084  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 264484us
W20260709 12:25:20.603514  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 245125us
I20260709 12:25:20.609244  4754 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:25:20.614444  6563 consensus_queue.cc:1059] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:20.584726  5343 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:20.621021  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:20.603993  6768 consensus_meta.cc:287] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.777s	user 0.014s	sys 0.022s
I20260709 12:25:20.687991  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:20.700101  6803 raft_consensus.cc:3055] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:20.700574  6803 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:20.704735  6849 raft_consensus.cc:3055] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:20.705096  5312 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:20.705302  6849 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:20.706022  6780 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:20.706310  6849 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:20.707641  6797 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:20.708865  6804 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:20.711133  6743 tablet_bootstrap.cc:492] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:20.712179  6743 tablet_bootstrap.cc:492] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
W20260709 12:25:20.712951  4569 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.508s (SENT) [suppressed 1 similar messages]
I20260709 12:25:20.715535  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 297) took 1082 ms. Trace:
W20260709 12:25:20.721223  4678 consensus_meta.cc:287] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.802s	user 0.015s	sys 0.021s
I20260709 12:25:20.721813  4678 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:20.718804  4604 rpcz_store.cc:276] 0709 12:25:19.633656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.633833 (+   177us) service_pool.cc:224] Handling call
0709 12:25:19.634153 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.875403 (+241250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.875462 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:19.875503 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:19.875540 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.715229 (+839689us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54002e54d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:20.715290 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:20.715459 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554385664}
I20260709 12:25:20.766153  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:20.768483  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:20.692068  6834 consensus_meta.cc:287] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.577s	user 0.007s	sys 0.012s
I20260709 12:25:20.720809  4571 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:20.712970  6743 ts_tablet_manager.cc:1403] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 3.779s	user 0.046s	sys 0.040s
I20260709 12:25:20.714035  6783 raft_consensus.cc:3055] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:20.707641  6849 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:20.788894  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:20.780464  6804 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:25:20.781994  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 61055.4us
I20260709 12:25:20.628659  6768 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:20.605926  6793 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:20.795351  5344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53808 (request call id 363) took 1158 ms. Trace:
I20260709 12:25:20.803260  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:20.814467  5344 rpcz_store.cc:276] 0709 12:25:19.634421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.645590 (+ 11169us) service_pool.cc:224] Handling call
0709 12:25:19.655905 (+ 10315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.797251 (+141346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.792376 (+995125us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54007dc154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:20.792538 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323614464}
I20260709 12:25:20.781898  6769 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:20.791419  6774 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:20.821956  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 208522us
I20260709 12:25:20.790781  6783 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:25:20.802295  6865 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:25:20.853021  4675 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:20.780591  6834 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:20.859639  4755 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:20.849910  6740 consensus_meta.cc:287] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.859s	user 0.009s	sys 0.013s
W20260709 12:25:20.857007  6643 consensus_meta.cc:287] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.646s	user 0.008s	sys 0.027s
W20260709 12:25:20.859484  6690 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.693s	user 0.014s	sys 0.032s
I20260709 12:25:20.851701  6774 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:20.815428  6791 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:20.853114  6811 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:20.857172  6778 raft_consensus.cc:3055] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:20.889213  4756 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:20.891096  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 87969.3us
I20260709 12:25:20.849448  6769 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:20.870958  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 158016us
I20260709 12:25:20.892863  6774 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:20.895046  6778 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:20.892238  6804 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:20.923830  6768 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:20.930492  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 79150.7us
I20260709 12:25:20.871336  6643 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:20.937778  4753 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:20.938086  6778 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:20.939771  6778 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:20.880306  6740 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:20.942173  6804 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:20.944026  6740 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:20.944765  4677 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:20.927322  6642 consensus_meta.cc:287] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.968s	user 0.008s	sys 0.002s
I20260709 12:25:20.945598  6642 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:20.958249  6643 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:20.871805  6690 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
W20260709 12:25:20.965549  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.949725  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:20.925465  6834 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:20.917763  6783 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:20.984777  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 272) took 1133 ms. Trace:
I20260709 12:25:20.985284  4754 rpcz_store.cc:276] 0709 12:25:19.851156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.890005 (+ 38849us) service_pool.cc:224] Handling call
0709 12:25:20.984569 (+1094564us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54000a7654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:20.984687 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449107840}
W20260709 12:25:20.986586  6819 consensus_meta.cc:287] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 1.127s	user 0.012s	sys 0.018s
I20260709 12:25:20.986863  6677 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:20.986995  6819 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:20.987499  6677 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:20.988466  6677 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:20.989513  6819 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:20.972986  6642 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:20.973505  4603 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:20.998137  4597 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:20.975616  6823 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:20.979710  6867 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 12:25:20.980203  6679 consensus_meta.cc:287] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.021s	user 0.010s	sys 0.029s
W20260709 12:25:20.980603  6860 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:25:20.932451  6840 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:20.995402  6814 consensus_meta.cc:287] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 1.137s	user 0.008s	sys 0.016s
I20260709 12:25:20.996228  6690 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:20.995405  6745 consensus_meta.cc:287] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.144s	user 0.000s	sys 0.046s
I20260709 12:25:21.014328  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.014643  6745 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:21.004410  6788 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:21.011098  6679 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:21.009467  6814 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:21.017879  6745 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:21.018272  6788 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:21.043747  6823 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:21.028462  6791 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:21.008183  6785 consensus_meta.cc:287] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.699s	user 0.006s	sys 0.006s
I20260709 12:25:21.058373  5347 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:21.058910  6785 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:21.059897  5347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53808 (request call id 364) took 1397 ms. Trace:
I20260709 12:25:21.034627  4672 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:21.023562  6767 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:21.062077  6767 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:21.046154  6832 consensus_meta.cc:287] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.867s	user 0.012s	sys 0.016s
I20260709 12:25:21.063093  6767 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:21.063556  6832 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:21.064347  4750 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.064458  6767 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:21.064666  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2198) took 1236 ms. Trace:
I20260709 12:25:21.063467  5342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53808 (request call id 359) took 1479 ms. Trace:
I20260709 12:25:21.066222  4675 rpcz_store.cc:276] 0709 12:25:19.828549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.835176 (+  6627us) service_pool.cc:224] Handling call
0709 12:25:21.064489 (+1229313us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b540017d754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:21.064612 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2219791104}
I20260709 12:25:21.070264  5314 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:21.060474  5347 rpcz_store.cc:276] 0709 12:25:19.662725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.662912 (+   187us) service_pool.cc:224] Handling call
0709 12:25:19.663246 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:21.059056 (+1395810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.059588 (+   532us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54006fbdd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:21.059814 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3207741824}
I20260709 12:25:21.068476  5342 rpcz_store.cc:276] 0709 12:25:19.583956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.591596 (+  7640us) service_pool.cc:224] Handling call
0709 12:25:19.629956 (+ 38360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:20.323654 (+693698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.063158 (+739504us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b54006fca54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:21.063384 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1432336640}
I20260709 12:25:21.078009  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:21.081055  6878 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:21.081676  6878 raft_consensus.cc:697] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:21.082614  6878 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:21.085898  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:21.087365  6876 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:25:21.087756  4750 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:21.089876  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:21.091459  6811 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:21.094281  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:21.029925  6769 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:21.009387  6840 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:21.098757  6840 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:21.091605  4500 catalog_manager.cc:5697] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 3 to -1, term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195), NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) evicted, now has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 4 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:21.008193  6743 raft_consensus.cc:359] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:21.100440  5314 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:21.061483  6679 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:21.061468  6785 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:21.060006  6814 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:21.103204  4500 catalog_manager.cc:5697] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:25:21.075459  6832 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:21.100898  6743 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:25:21.105858  6811 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:21.126533  6840 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:21.127355  6840 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:21.111855  6743 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:21.164839  4500 catalog_manager.cc:5697] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:21.171761  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2203) took 1246 ms. Trace:
I20260709 12:25:21.172194  4677 rpcz_store.cc:276] 0709 12:25:19.925023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.925165 (+   142us) service_pool.cc:224] Handling call
0709 12:25:19.937621 (+ 12456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:20.945135 (+1007514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.171526 (+226391us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400086954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:21.171703 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2315754240}
I20260709 12:25:21.107029  6774 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:21.196417  6774 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:21.203444  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:21.204031  4596 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:21.204897  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:21.206085  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.206282  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.207216  4677 raft_consensus.cc:2410] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:25:21.186877  6855 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.954s	user 0.006s	sys 0.010s
I20260709 12:25:21.215276  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:25:21.216140  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.215617  4597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1356) took 1543 ms (client timeout 1585 ms). Trace:
W20260709 12:25:21.219220  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.219270  5344 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:21.220057  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:21.220624  4598 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:21.220881  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:21.198901  6783 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:21.191946  6787 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:25:21.219102  4597 rpcz_store.cc:269] 0709 12:25:19.664343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.664659 (+   316us) service_pool.cc:224] Handling call
0709 12:25:21.207481 (+1542822us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54002337d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:21.207595 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3039859968}
I20260709 12:25:21.226418  6803 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:21.203078  6773 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:21.239403  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:21.242440  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:21.244843  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:21.245325  6563 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:21.246500  4605 raft_consensus.cc:2368] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4.
I20260709 12:25:21.246414  4756 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:21.247372  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.241037  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.221644  6599 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.250114  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:21.248718  4675 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:21.250699  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:21.254983  4751 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.253324  6599 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:21.256059  4599 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:25:21.256810  4750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2149) took 1443 ms (client timeout 1822 ms). Trace:
I20260709 12:25:21.261842  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:21.249985  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:21.257978  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:21.258595  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.231540  6840 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:21.227381  6745 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:21.261109  4674 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:21.263666  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:21.262162  4750 rpcz_store.cc:269] 0709 12:25:19.813025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.813460 (+   435us) service_pool.cc:224] Handling call
0709 12:25:21.256659 (+1443199us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54001dfbd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:21.256757 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2834466176}
I20260709 12:25:21.251430  6773 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:21.272521  6786 consensus_queue.cc:1059] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:21.269683  4756 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:25:21.274539  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.282313  6743 ts_tablet_manager.cc:1434] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.492s	user 0.013s	sys 0.000s
I20260709 12:25:21.288827  4680 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.265790  4597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:25:21.227608  6811 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:21.267464  4748 raft_consensus.cc:3055] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:21.293818  4748 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:21.294291  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:21.295481  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:21.295630  6599 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:21.296548  6599 consensus_queue.cc:1059] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:21.296679  6791 consensus_queue.cc:1059] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:21.296897  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:21.241334  6787 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:21.297472  4753 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:21.297709  6791 raft_consensus.cc:3055] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:21.298125  6791 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:21.299157  6787 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:21.298870  6791 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:21.300652  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.264263  6803 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:21.309808  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.291239  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:21.312546  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:21.314232  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.294546  6811 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:21.315342  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.315829  6773 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:21.316438  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.317189  4677 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 2: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:21.317266  4766 tablet_copy_service.cc:342] P 49277a7653524787a715232ab908bbc6: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-e8a74137ba104fe0b465cab6399c4940 received from {username='slave'} at 127.0.0.1:42632
I20260709 12:25:21.317929  4766 tablet_copy_service.cc:434] P 49277a7653524787a715232ab908bbc6: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-e8a74137ba104fe0b465cab6399c4940 on tablet e8a74137ba104fe0b465cab6399c4940 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:21.292255  6786 raft_consensus.cc:3055] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:21.319383  6786 raft_consensus.cc:740] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:21.301470  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.302136  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.321887  4678 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:21.302984  4679 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:21.268158  4752 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 4: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:21.304153  6768 raft_consensus.cc:3055] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:21.304327  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.305459  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.260250  6785 consensus_queue.cc:1059] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:25:21.300968  6778 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:25:21.285694  6745 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:21.327924  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.273186  6679 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:25:21.331796  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.334568  4496 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52808 (request call id 274) took 1712 ms. Trace:
I20260709 12:25:21.289887  4597 raft_consensus.cc:606] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:21.309052  4571 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:21.316526  6811 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:21.323027  4747 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:21.325001  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:21.325719  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:21.325909  4678 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:21.323881  6768 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:21.291581  4751 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:21.256561  6774 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:25:21.387543  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.370558  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:21.389063  4752 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:21.389180  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.367583  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.379549  4756 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:25:21.344920  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:21.336000  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.391641  4754 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 2: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:21.387516  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:21.338709  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.338037  4755 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:21.396767  4674 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:21.365072  4748 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:21.400663  4596 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
W20260709 12:25:21.403895  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 100322us
I20260709 12:25:21.405099  4680 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:21.407106  5348 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 2: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:21.408839  4598 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:21.300122  6791 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:21.415019  4720 leader_election.cc:400] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4.
I20260709 12:25:21.418213  4720 leader_election.cc:403] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:25:21.434121  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:21.435976  4748 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:21.320282  6786 consensus_queue.cc:260] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:21.440032  4677 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:21.437129  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 128195us
I20260709 12:25:21.442039  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:21.443409  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:21.369130  6679 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:21.369916  6785 raft_consensus.cc:3055] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:21.445469  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.445845  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.443567  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.329470  6745 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:21.455024  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.446404  4672 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:21.405925  6780 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.457819  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:21.470780  4752 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:21.477164  4598 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:21.448702  6875 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:21.334966  4496 rpcz_store.cc:276] 0709 12:25:19.622519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.629767 (+  7248us) service_pool.cc:224] Handling call
0709 12:25:21.334482 (+1704715us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:20.783554 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:20.784359 (+   805us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:20.784440 (+    81us) write_op.cc:435] Acquired schema lock
0709 12:25:20.784486 (+    46us) tablet.cc:662] Decoding operations
0709 12:25:20.785003 (+   517us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:20.785087 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:20.785128 (+    41us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:20.785497 (+   369us) tablet.cc:701] Row locks acquired
0709 12:25:20.785554 (+    57us) write_op.cc:260] PREPARE: finished
0709 12:25:20.785816 (+   262us) write_op.cc:270] Start()
0709 12:25:20.785996 (+   180us) write_op.cc:276] Timestamp: P: 1783599920785725 usec, L: 0
0709 12:25:20.786022 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:25:20.786957 (+   935us) log.cc:844] Serialized 538 byte log entry
0709 12:25:21.138392 (+351435us) op_driver.cc:464] REPLICATION: finished
0709 12:25:21.197644 (+ 59252us) write_op.cc:301] APPLY: starting
0709 12:25:21.197760 (+   116us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:21.197872 (+   112us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:21.198011 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:21.198317 (+   306us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:21.202229 (+  3912us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:21.202290 (+    61us) write_op.cc:312] APPLY: finished
0709 12:25:21.208670 (+  6380us) log.cc:844] Serialized 23 byte log entry
0709 12:25:21.208883 (+   213us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:21.208989 (+   106us) write_op.cc:454] Released schema lock
0709 12:25:21.209052 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:21.209435 (+   383us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:21.209477 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":34019,"apply.run_cpu_time_us":2409,"apply.run_wall_time_us":20437,"num_ops":1,"prepare.queue_time_us":87025,"prepare.run_cpu_time_us":32449,"prepare.run_wall_time_us":141108,"raft.queue_time_us":151897,"raft.run_cpu_time_us":6029,"raft.run_wall_time_us":33675,"replication_time_us":352228,"thread_start_us":322225,"threads_started":4,"wal-append.queue_time_us":179486}]]}
W20260709 12:25:21.654403  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 211081us
W20260709 12:25:21.609402  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.483259  6780 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:21.658421  6780 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:21.480366  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.480557  4674 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:21.515946  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.528076  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.652333  4500 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52818 (request call id 300) took 1913 ms. Trace:
W20260709 12:25:21.517695  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.498025  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.455368  6677 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:21.513268  6774 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:21.684178  4498 catalog_manager.cc:5697] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 3 to 4, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 4 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:21.560911  6794 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.454413  6679 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:21.445353  6870 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4.
I20260709 12:25:21.448329  6778 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:21.529950  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.530346  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:21.555370  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 71225.4us
W20260709 12:25:21.697734  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.582368  6767 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.000s
I20260709 12:25:21.492753  4602 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:25:21.525914  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:21.678817  6786 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:21.454571  6785 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:21.679545  4753 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:21.575253  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.714632  4751 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:21.689142  4597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51130
W20260709 12:25:21.625077  6885 log.cc:927] Time spent T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
W20260709 12:25:21.692715  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.646812  6787 raft_consensus.cc:3055] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:21.720840  4681 raft_consensus.cc:2410] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:25:21.694300  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:21.715564  4597 raft_consensus.cc:606] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:21.706032  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:21.686239  6794 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:21.667119  4500 rpcz_store.cc:276] 0709 12:25:19.739374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.739804 (+   430us) service_pool.cc:224] Handling call
0709 12:25:21.652253 (+1912449us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:21.221450 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:21.231546 (+ 10096us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:21.231584 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:25:21.231605 (+    21us) tablet.cc:662] Decoding operations
0709 12:25:21.283602 (+ 51997us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:21.283660 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:21.283692 (+    32us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:25:21.283999 (+   307us) tablet.cc:701] Row locks acquired
0709 12:25:21.284016 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:25:21.284162 (+   146us) write_op.cc:270] Start()
0709 12:25:21.284276 (+   114us) write_op.cc:276] Timestamp: P: 1783599921284117 usec, L: 0
0709 12:25:21.284292 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 12:25:21.291339 (+  7047us) log.cc:844] Serialized 1550 byte log entry
0709 12:25:21.310970 (+ 19631us) op_driver.cc:464] REPLICATION: finished
0709 12:25:21.311959 (+   989us) write_op.cc:301] APPLY: starting
0709 12:25:21.312040 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:21.312198 (+   158us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:21.312301 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:21.312835 (+   534us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:21.313004 (+   169us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:25:21.313047 (+    43us) write_op.cc:312] APPLY: finished
0709 12:25:21.313267 (+   220us) log.cc:844] Serialized 35 byte log entry
0709 12:25:21.313544 (+   277us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:21.313660 (+   116us) write_op.cc:454] Released schema lock
0709 12:25:21.313697 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:21.314045 (+   348us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:21.314071 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17746,"reported_tablets":6,"spinlock_wait_cycles":3328,"child_traces":[["op",{"apply.queue_time_us":773,"apply.run_cpu_time_us":2031,"apply.run_wall_time_us":2031,"num_ops":3,"prepare.queue_time_us":109,"prepare.run_cpu_time_us":3933,"prepare.run_wall_time_us":76510,"replication_time_us":26557}]]}
I20260709 12:25:21.718910  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.717288  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:21.683668  6677 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:25:21.737924  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.730916  4754 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:21.732156  4678 raft_consensus.cc:2393] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:21.732079  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.733742  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:21.756596  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.760052  6794 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:21.662918  6768 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:21.802407  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 85238.5us
I20260709 12:25:21.693912  6745 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 12:25:21.806392  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.811753  4672 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:21.807523  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.802145  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:21.809722  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.816157  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.817102  4673 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:21.688150  6801 consensus_queue.cc:1059] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:25:21.825199  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:21.821188  4679 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:21.717875  6778 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:21.720709  6787 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:21.823792  4674 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:21.820037  6877 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:21.830006  5343 raft_consensus.cc:1217] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:21.831177  6768 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:21.831538  6679 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:21.835530  4672 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:21.836822  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.859745  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.843291  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.753296  6677 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:21.868957  4680 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:21.847323  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:21.844219  6803 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:21.845973  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.844018  6870 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:21.810014  6794 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:21.867655  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.870934  4750 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:21.852737  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.948192  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.872799  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:21.853901  6877 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:21.941639  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:21.882578  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:21.949141  4674 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
W20260709 12:25:21.874459  4644 connection.cc:256] RPC call timeout handler was delayed by -1.1415238509998744s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
I20260709 12:25:22.006282  4644 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.007869  4644 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:21.454169  6878 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.411s
W20260709 12:25:22.015580  4748 consensus_meta.cc:287] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.579s	user 0.006s	sys 0.003s
I20260709 12:25:22.016237  4748 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:25:22.016872  6563 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:25:22.018095  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:22.018334  4751 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:22.018146  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:22.018415  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 193298us
I20260709 12:25:22.021034  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:22.019929  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:22.023880  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:22.021807  4673 raft_consensus.cc:2376] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4. Re-sending same reply.
W20260709 12:25:22.026865  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 160120us
I20260709 12:25:22.012853  5339 tablet_service.cc:1558] Processing DeleteTablet for tablet 1b7b0c6c811e455abe898cb4d21acad4 with delete_type TABLET_DATA_TOMBSTONED (TS dbe960c35c84451faf4e5b5c9233fa39 not found in new config with opid_index -1) from {username='slave'} at 127.0.0.1:37154
I20260709 12:25:22.013281  6745 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:22.028651  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:25:22.032866  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 221727us
I20260709 12:25:22.025271  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:22.079107  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 50540.6us
W20260709 12:25:22.087994  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 62821.9us
I20260709 12:25:22.032450  6870 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:25:22.100010  6875 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.259s	user 0.004s	sys 0.004s
I20260709 12:25:22.101508  6669 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.102015  6669 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:21.927636  6785 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:22.004807  6778 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.084988  6844 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:22.095750  6803 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:22.127252  4644 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:22.128051  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:22.113086  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:22.132450  4753 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:25:22.018723  6787 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:22.109206  6669 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:22.108925  6870 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:25:22.148191  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.144711  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 55683.2us
W20260709 12:25:22.145160  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.147737  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.156539  4645 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.156607  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:22.112133  6767 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:22.180555  4645 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:25:22.186511  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 59368.6us
I20260709 12:25:22.187831  4644 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:22.189702  6745 raft_consensus.cc:697] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:22.190959  6745 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:22.108979  6794 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:25:22.146637  6840 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:22.218526  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 61213.8us
I20260709 12:25:22.219727  5312 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:22.153115  5343 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:25:22.218876  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.151027  6898 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:22.152114  6877 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:22.138551  6844 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:22.236337  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 156251us
I20260709 12:25:22.227734  6772 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:22.242268  6772 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.242682  6772 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:22.243052  6769 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:22.243808  6769 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:22.247694  6677 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:22.226341  6767 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:22.227831  6904 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:22.269882  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 111800us
I20260709 12:25:22.262971  6904 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:22.264860  6898 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:22.309298  6891 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:22.285702  6898 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:25:22.322105  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 141656us
I20260709 12:25:22.285444  6904 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:22.324147  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.358480  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:22.379912  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:22.382606  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1393) took 1129 ms. Trace:
I20260709 12:25:22.368983  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:22.356735  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 96488.7us
I20260709 12:25:22.383363  4673 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 7: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:22.382972  4681 rpcz_store.cc:276] 0709 12:25:21.253418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.255354 (+  1936us) service_pool.cc:224] Handling call
0709 12:25:22.378037 (+1122683us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b54001d2ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:22.382545 (+  4508us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1050666368}
I20260709 12:25:22.324447  6898 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:22.322054  6900 raft_consensus.cc:3055] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:22.389020  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:22.271389  6870 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.391134  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:22.393949  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:22.382352  6902 tablet_replica.cc:333] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:25:22.323654  6769 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:22.321058  6877 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.090s
I20260709 12:25:22.319535  6907 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.097s
I20260709 12:25:22.411365  6900 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:22.455397  6599 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:22.455886  6599 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:22.455629  6900 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.456877  6900 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:22.456782  6599 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:22.410558  6911 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.121s	user 0.000s	sys 0.005s
W20260709 12:25:22.357197  6786 consensus_meta.cc:287] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.639s	user 0.009s	sys 0.003s
I20260709 12:25:22.412487  5347 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 2: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
W20260709 12:25:22.405001  6791 consensus_meta.cc:287] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.967s	user 0.008s	sys 0.003s
I20260709 12:25:22.418004  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:22.457721  6677 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:22.438791  6772 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:22.458742  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 134685us
I20260709 12:25:22.463689  4645 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:25:22.418880  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.409214  6768 consensus_meta.cc:287] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.547s	user 0.008s	sys 0.000s
I20260709 12:25:22.468967  6772 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:22.460482  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 46361.9us
I20260709 12:25:22.407790  6898 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.415313  6801 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:22.476683  6774 raft_consensus.cc:993] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:22.477209  6892 raft_consensus.cc:1081] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:22.480995  6788 raft_consensus.cc:3055] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:22.481408  6788 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:22.482229  6788 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:22.483500  6788 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:22.466391  6679 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:22.415597  6899 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:22.489920  6899 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:22.490866  6899 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.491518  6898 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:22.469724  4675 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:22.469826  6909 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:22.495667  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:22.496266  4753 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:22.470006  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:22.472299  4680 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:25:22.426010  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:22.497473  4680 pending_rounds.cc:85] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 3
I20260709 12:25:22.497782  4680 pending_rounds.cc:107] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:25:22.498063  4680 raft_consensus.cc:2889] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:22.502351  4645 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.504312  4645 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.506006  6909 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:22.506834  6791 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:22.507645  6791 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:22.511405  4645 leader_election.cc:400] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6.
I20260709 12:25:22.512118  6787 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:22.512717  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.515025  6804 raft_consensus.cc:3055] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:22.515425  6804 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:22.516237  6804 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:22.471740  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:22.473829  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:22.517740  6909 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:22.517941  4676 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:22.518148  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:22.518921  6768 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:22.519279  6768 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:25:22.519497  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.519690  6768 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:22.520233  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.519747  4499 catalog_manager.cc:5697] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 5, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 5 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:22.474126  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 54728.7us
I20260709 12:25:22.522013  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.518165  6909 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:22.474361  6599 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:22.522634  6909 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:22.475333  6910 raft_consensus.cc:993] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:22.475808  6913 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:22.523732  6913 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:22.524621  6913 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.525889  6913 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:22.526929  5347 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:22.528416  6783 raft_consensus.cc:3055] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:22.528790  6783 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:22.529387  6768 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:22.529543  6783 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.529929  4499 catalog_manager.cc:5697] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:22.530710  6783 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:22.531486  6768 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:22.471141  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:22.476493  6801 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:22.494215  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:22.417760  6844 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:22.522022  4498 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 160) took 1492 ms. Trace:
I20260709 12:25:22.535728  4645 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.534396  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:22.538740  5314 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:22.540871  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2228) took 1013 ms. Trace:
I20260709 12:25:22.541249  4675 rpcz_store.cc:276] 0709 12:25:21.527536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.527714 (+   178us) service_pool.cc:224] Handling call
0709 12:25:22.540718 (+1013004us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b540009a454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:22.540814 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1851844352}
I20260709 12:25:22.542366  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:22.545249  6775 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:22.545559  6625 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:22.546092  6896 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:22.546319  6625 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.547536  6811 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:22.547919  6785 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:22.548403  6625 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:22.548902  6625 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:22.549903  6897 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:22.550287  6897 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:22.550639  6897 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:22.552564  6898 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:22.552824  6854 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:22.553282  4645 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.553339  6854 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.545737  6775 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.554401  6775 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:22.554855  6898 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:22.555598  6854 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:22.554615  4645 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:22.559736  4645 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.557399  6563 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.557695  4647 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:22.566012  6563 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:22.517429  6804 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:22.568809  4754 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:22.569199  4676 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:22.570297  4748 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:22.568780  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.570303  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:22.575079  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.576877  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:22.577512  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:22.579502  4680 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:22.579331  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:22.581643  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 109979us
I20260709 12:25:22.582157  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:22.582306  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:22.583348  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:22.583719  4596 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:22.585376  4596 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:22.585546  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:22.580673  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:22.586683  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:22.587270  4679 raft_consensus.cc:2368] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7.
I20260709 12:25:22.587653  4596 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:22.588191  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:22.588764  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:22.549374  6625 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.599967  4672 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:22.570613  6912 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:22.600785  6745 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:22.601166  6910 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:22.601290  6906 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:22.599754  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:22.603137  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:22.606731  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:22.607668  4753 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:22.608124  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:22.608884  6563 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:22.539758  6801 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:22.611559  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 89679.9us
I20260709 12:25:22.619160  4597 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:22.614389  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:22.615304  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:22.571292  6898 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:22.589650  6844 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:22.615690  4748 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:22.620327  4598 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:25:22.616394  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:22.620162  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:22.616155  4599 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:22.615962  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:22.631644  4569 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:22.601334  6625 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:22.625597  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:22.626747  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.615144  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:22.610730  6814 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:22.639532  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:22.623941  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:22.640349  4673 raft_consensus.cc:2368] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7.
I20260709 12:25:22.610307  6840 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.558604  6768 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 12:25:22.639669  4756 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:22.641511  4681 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:22.646533  6811 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:22.630268  4748 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:25:22.671682  4748 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:22.671989  4753 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:25:22.672192  4748 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:22.652326  6904 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.629657  6844 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.610021  6906 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:22.613824  6915 raft_consensus.cc:1081] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:25:22.540076  4498 rpcz_store.cc:276] 0709 12:25:21.029273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.030616 (+  1343us) service_pool.cc:224] Handling call
0709 12:25:22.521955 (+1491339us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:21.721085 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:21.838711 (+117626us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:21.838762 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:25:21.838790 (+    28us) tablet.cc:662] Decoding operations
0709 12:25:21.839266 (+   476us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:21.839345 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:21.839389 (+    44us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:21.839640 (+   251us) tablet.cc:701] Row locks acquired
0709 12:25:21.839661 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:25:21.839920 (+   259us) write_op.cc:270] Start()
0709 12:25:21.840061 (+   141us) write_op.cc:276] Timestamp: P: 1783599921839800 usec, L: 0
0709 12:25:21.840091 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:25:21.840947 (+   856us) log.cc:844] Serialized 533 byte log entry
0709 12:25:22.222457 (+381510us) op_driver.cc:464] REPLICATION: finished
0709 12:25:22.312410 (+ 89953us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effeb3ac0ac 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007efff1eecef0
0709 12:25:22.417276 (+104866us) write_op.cc:301] APPLY: starting
0709 12:25:22.417387 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:22.417504 (+   117us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:22.417523 (+    19us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:22.417830 (+   307us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:22.417992 (+   162us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:22.418033 (+    41us) write_op.cc:312] APPLY: finished
0709 12:25:22.418218 (+   185us) log.cc:844] Serialized 23 byte log entry
0709 12:25:22.418392 (+   174us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:22.418469 (+    77us) write_op.cc:454] Released schema lock
0709 12:25:22.418503 (+    34us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:22.418839 (+   336us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:22.418863 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":318329,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":194556,"apply.run_cpu_time_us":1569,"apply.run_wall_time_us":1570,"num_ops":1,"prepare.queue_time_us":4004,"prepare.run_cpu_time_us":3646,"prepare.run_wall_time_us":130990,"replication_time_us":382220,"spinlock_wait_cycles":9166592,"thread_start_us":189058,"threads_started":1}]]}
I20260709 12:25:22.645012  4596 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:22.656479  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:22.656268  6778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.014s	sys 0.027s
I20260709 12:25:22.660535  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:22.620991  6892 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:22.614720  6803 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:22.674962  4569 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.675850  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:22.628932  6854 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.683815  4748 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.684846  4604 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:22.688325  4598 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:25:22.640196  6891 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:22.719676  4748 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:22.691857  4675 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:22.634986  6921 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.625787  6877 consensus_queue.cc:1059] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
W20260709 12:25:22.702960  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.690653  6778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.014s	sys 0.028s
I20260709 12:25:22.733901  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.730659  6563 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:22.619498  6905 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 12:25:22.732246  6854 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.685161  6896 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.681488  6844 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:22.748761  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.752290  6894 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:22.751493  5312 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.723055  6625 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:22.724134  6908 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.620779  6912 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.106s
I20260709 12:25:22.758471  6780 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:22.720194  6811 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:22.773298  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.773625  6563 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:22.758484  6904 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:22.750502  6773 raft_consensus.cc:3055] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:22.685178  6893 consensus_queue.cc:1059] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
W20260709 12:25:22.814869  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 93564.5us
I20260709 12:25:22.672991  6840 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.781831  6625 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:25:22.793744  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 165316us
W20260709 12:25:22.827839  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.829088  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:22.830966  4605 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:22.794428  6902 raft_consensus.cc:2243] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEARNER]: Raft consensus shutting down.
I20260709 12:25:22.817070  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:22.833967  6863 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:22.817392  6878 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:22.819862  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 46623.8us
I20260709 12:25:22.835636  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:22.820010  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 145146us
I20260709 12:25:22.837311  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.820278  4677 raft_consensus.cc:2410] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:25:22.755616  6892 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:22.778463  6778 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:25:22.840843  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.813345  6773 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:25:22.844635  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 106204us
I20260709 12:25:22.847877  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.731802  6921 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.832679  6625 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:22.849997  4569 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.852370  6563 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:22.852952  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.849434  4645 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.865119  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.865839  4645 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.844652  6768 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.792611  6908 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:22.831954  6904 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:25:22.843895  6786 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:22.771688  6896 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:22.844019  6863 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:22.883529  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:22.892254  6791 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:22.892372  6773 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
W20260709 12:25:22.895470  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.896754  6854 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:22.861720  6902 pending_rounds.cc:70] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Trying to abort 1 pending ops.
W20260709 12:25:22.884805  6919 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
W20260709 12:25:22.882262  6916 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:25:22.942790  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 59323.5us
I20260709 12:25:22.823637  6803 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.897846  6907 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:22.951766  6910 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:22.884093  6929 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:25:22.955489  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 126685us
I20260709 12:25:22.883764  6870 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:22.886567  6840 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:22.954808  4569 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.885713  6928 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:22.969135  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 217781us
W20260709 12:25:22.966043  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 100297us
I20260709 12:25:22.837095  6906 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:22.967262  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 110597us
W20260709 12:25:22.966197  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 90282.4us
I20260709 12:25:22.946768  6778 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:22.980720  6932 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:22.970863  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.941179  6775 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:22.905692  6768 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.879803  6774 consensus_queue.cc:1059] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 12:25:22.973133  4569 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.997251  4569 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.998769  4569 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.980208  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:23.002681  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.949967  6902 pending_rounds.cc:77] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305625268964253696 op_type: NO_OP noop_request { }
I20260709 12:25:22.969483  6791 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:22.953742  6801 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:22.970772  6929 raft_consensus.cc:697] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:22.988453  6773 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:22.967079  6767 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:25:22.991794  6775 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:22.992554  5347 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:22.973443  6893 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:22.972889  6928 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:22.995937  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:22.875252  6811 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:23.001052  4569 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:23.012204  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:23.012266  4720 leader_election.cc:400] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7.
I20260709 12:25:23.012842  4677 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:22.972640  6927 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:22.979477  6912 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:23.039377  6912 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.019044  6933 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:23.023412  4645 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:23.025664  4754 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.003871  6786 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.003717  6778 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:23.016695  6787 consensus_meta.cc:287] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.000s
I20260709 12:25:23.013943  6834 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:23.009938  6902 raft_consensus.cc:2889] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEARNER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:25:23.015196  6679 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:23.031953  4720 leader_election.cc:403] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 12:25:23.025399  6900 consensus_meta.cc:287] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.565s	user 0.007s	sys 0.004s
I20260709 12:25:22.982228  6905 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.042320  4569 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:22.984318  6907 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:25:23.047025  6814 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:23.053017  4754 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:23.054131  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:22.998325  6897 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.056311  6927 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:23.117192  6927 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.056710  6896 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:23.063481  6832 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:23.074023  6935 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.077548  6563 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7.
W20260709 12:25:23.077912  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 65757.9us
I20260709 12:25:23.020515  6909 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.030318  6928 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:23.083091  6913 consensus_meta.cc:287] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.557s	user 0.007s	sys 0.003s
I20260709 12:25:23.084223  6677 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:23.091193  6921 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:23.094446  6870 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:23.095651  4600 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:23.098639  6902 raft_consensus.cc:2272] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEARNER]: Raft consensus is shut down!
I20260709 12:25:23.048409  6893 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.101485  6929 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:23.111986  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:23.112800  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.117172  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63116.2us
W20260709 12:25:23.119459  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 77208us
I20260709 12:25:23.137092  4569 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:23.122138  6625 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.138599  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:23.140466  4569 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:23.123798  4720 leader_election.cc:400] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7.
I20260709 12:25:23.123291  6832 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:23.131170  6909 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:23.131393  6897 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:23.126216  6787 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:23.151784  4599 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:23.153234  6801 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:23.148100  4569 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:23.143215  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:23.145092  6912 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:23.145959  6778 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:23.146154  6832 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.159577  6832 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.149470  6913 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.150002  6893 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:23.161206  4645 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:23.127686  6677 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:23.163890  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 145287us
I20260709 12:25:23.165629  4645 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:23.162420  4601 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:25:23.167765  6908 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:23.168236  6912 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:23.168550  6900 raft_consensus.cc:2764] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:23.168771  6912 raft_consensus.cc:697] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:23.169541  6788 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:23.171975  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:23.172617  4748 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:25:23.173184  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:23.173897  4675 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:23.174178  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:23.176688  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:23.177418  4675 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 2: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:23.181706  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:23.182127  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:23.182847  4675 raft_consensus.cc:2393] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:23.174289  5314 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:23.185700  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:23.146839  6804 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:23.185211  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:23.181181  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:23.190393  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:23.190891  4598 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:23.186421  4597 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:23.169977  6788 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:23.199043  6943 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:25:23.199939  6788 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.201776  6804 raft_consensus.cc:697] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:23.202545  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:23.113235  6905 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.206795  4678 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51856
I20260709 12:25:23.206698  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:23.207451  4678 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:23.203155  4604 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:23.207613  4605 raft_consensus.cc:3055] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:23.207723  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:23.169727  6912 consensus_queue.cc:237] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.214376  6563 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.206820  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:23.210615  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.208431  4605 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:23.207851  4678 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:23.216638  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.218146  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:23.218999  4676 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:25:23.219257  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:23.201442  6788 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.222599  4681 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:23.194841  6909 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:23.234266  4673 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:23.195055  6870 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:23.235800  6870 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:23.236186  6870 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.221303  6938 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.237984  6938 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:23.216725  4605 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.241650  4674 raft_consensus.cc:2410] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:25:23.241770  4605 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:23.224400  4569 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:23.218098  6625 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:23.226826  6900 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:23.204128  6814 consensus_queue.cc:1059] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:25:23.251849  6900 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:23.252405  6814 raft_consensus.cc:3055] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:23.252485  6900 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:23.253109  6814 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:23.254168  6814 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:23.255488  6814 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.186146  6893 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:25:23.228668  6768 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:23.259056  6909 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:23.260735  6909 raft_consensus.cc:3055] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:23.260967  6834 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:23.230253  6902 ts_tablet_manager.cc:1916] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:23.261488  6834 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:23.208568  6762 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:23.263538  6834 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:23.233085  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:23.265022  4604 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:23.264248  6762 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.265831  6762 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.269603  6834 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:23.270543  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:23.271163  4603 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:23.231967  6677 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.275358  6677 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:23.299767  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:23.302475  6933 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:23.306110  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:23.314741  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 90394.9us
I20260709 12:25:23.315778  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:23.312999  4678 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:23.328881  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:23.331636  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:23.332371  4601 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:25:23.334364  4748 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:23.335443  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:23.336114  4754 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:23.343930  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:23.344218  4678 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:23.337702  4645 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:23.340216  4748 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:23.346314  4645 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:23.342033  5347 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:23.342195  4750 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:23.347699  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:23.340817  6563 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:23.345288  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:23.202950  6804 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:23.351784  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:23.344466  4569 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:23.261245  6909 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:23.364737  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:23.365726  6563 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:23.365846  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:23.366393  6563 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:23.353703  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:23.367059  6563 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.353869  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:23.367308  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:23.354071  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:23.356860  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:23.354117  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:23.362110  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:23.369059  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:23.368598  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:23.362589  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:23.370078  5344 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.371196  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:23.372568  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:23.374807  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:23.375623  4747 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:23.366008  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:23.378890  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:23.392937  4749 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:23.389124  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:23.395861  4673 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:23.393508  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:23.344811  6767 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.396099  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:23.336853  6899 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:23.403294  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:23.411036  4678 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.405965  4680 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:23.407393  4571 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:23.408196  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:23.400406  4569 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:23.354827  6801 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:23.414345  4674 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:23.414904  4681 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 2: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:23.410354  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:23.355355  6768 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:23.356026  6876 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.415768  6926 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:25:23.421043  4604 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:23.421113  4677 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:23.383556  6679 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:23.386315  6599 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:23.436277  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:23.446583  4681 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:23.391906  6907 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:23.377431  6897 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:23.428529  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:23.441298  4673 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:23.445214  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:23.471194  4596 raft_consensus.cc:3055] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:23.472030  4596 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:23.432372  6876 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.426740  6854 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:23.449296  6917 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.473s	user 0.001s	sys 0.003s
I20260709 12:25:23.415781  6937 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:23.440404  6892 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.436139  6915 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:23.476507  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48072.4us
I20260709 12:25:23.518297  4645 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:23.451157  6907 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:23.441075  6906 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:23.440514  6768 raft_consensus.cc:697] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:23.417135  6774 consensus_queue.cc:1059] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:25:23.442219  6679 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.481290  6801 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:23.486953  6940 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:23.501917  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:23.483026  4596 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:23.511133  6930 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.148s
W20260709 12:25:23.511803  6941 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 12:25:23.512625  6934 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:23.480944  6844 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.699s	user 0.007s	sys 0.007s
I20260709 12:25:23.462255  6877 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.515993  5344 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:23.355654  6932 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 12:25:23.502457  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:25:23.587502  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 85150.2us
I20260709 12:25:23.531996  4678 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:23.477509  6876 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.459750  6599 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.450424  6625 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:25:23.475335  6938 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:23.552294  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:23.488583  6785 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.560307  6762 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:23.562249  6768 consensus_queue.cc:237] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.567739  6563 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.601518  4750 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:23.517481  6915 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.546789  6907 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.543031  6937 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.444648  6909 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:23.589143  4676 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:23.553706  6679 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.488852  6854 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:23.564891  6912 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:23.490908  6897 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.598428  6905 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:23.594882  6877 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:23.599754  6948 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:23.599016  6801 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.603514  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:23.606199  4747 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:25:23.606715  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 54555.4us
I20260709 12:25:23.610136  6563 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:23.595391  6935 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:23.618904  6915 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:23.730198  6679 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:23.593082  6844 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:23.765898  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:23.670109  6762 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:23.601215  6785 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.661039  6900 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.642942  6937 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:23.677237  6773 consensus_meta.cc:287] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.658s	user 0.002s	sys 0.008s
I20260709 12:25:23.667862  4605 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:23.691433  6936 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:23.692842  6599 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:23.694132  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:23.682001  6909 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.699839  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:23.785959  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:23.591852  6921 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:23.717895  6952 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:25:23.723537  6933 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.786684  4755 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.732087  4672 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:23.736101  4644 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:23.753674  4748 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:23.757891  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:23.787860  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 51860.1us
I20260709 12:25:23.761762  4674 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:23.767069  4499 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52812 (request call id 289) took 3288 ms. Trace:
I20260709 12:25:23.768903  4749 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:25:23.771297  4645 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:25:23.781206  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 353660us
I20260709 12:25:23.728106  6870 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.792939  4676 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:23.794672  6926 raft_consensus.cc:3055] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:23.795223  6926 raft_consensus.cc:740] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:23.796265  6926 consensus_queue.cc:260] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:23.651829  6927 consensus_queue.cc:1175] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49277a7653524787a715232ab908bbc6" responder_term: 6 status { last_received { term: 4 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:25:23.811654  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.819249  6928 consensus_meta.cc:287] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.688s	user 0.000s	sys 0.011s
I20260709 12:25:23.819768  6928 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.820933  4600 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:23.822985  6897 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:23.823994  6854 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:23.812201  4498 catalog_manager.cc:5697] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 5 to 7, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 7 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:23.828006  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:23.832218  4498 catalog_manager.cc:5697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 4 to 5, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 5 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:23.836989  6910 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:23.836943  4498 catalog_manager.cc:5697] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 2 to 3, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 3 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:23.842458  6677 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:23.844206  4498 catalog_manager.cc:5697] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added, no longer has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 4 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:23.845702  6677 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:23.828967  6926 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.854578  4756 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:23.864717  4604 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:23.875404  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:23.876065  4677 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.807789  6767 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:23.876775  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:23.878839  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:23.888258  4677 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:23.827754  6788 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.893455  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:23.901188  4604 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:23.901463  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.882618  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:23.899308  4680 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:23.788612  4499 rpcz_store.cc:276] 0709 12:25:20.470045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.470315 (+   270us) service_pool.cc:224] Handling call
0709 12:25:23.758622 (+3288307us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:22.645911 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:22.824457 (+178546us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:22.827366 (+  2909us) write_op.cc:435] Acquired schema lock
0709 12:25:22.832936 (+  5570us) tablet.cc:662] Decoding operations
0709 12:25:22.898159 (+ 65223us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:22.898244 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:22.898272 (+    28us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:25:22.898671 (+   399us) tablet.cc:701] Row locks acquired
0709 12:25:22.898689 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:25:22.898831 (+   142us) write_op.cc:270] Start()
0709 12:25:22.898948 (+   117us) write_op.cc:276] Timestamp: P: 1783599922898783 usec, L: 0
0709 12:25:22.898989 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:25:22.968821 (+ 69832us) log.cc:844] Serialized 918 byte log entry
0709 12:25:23.080218 (+111397us) op_driver.cc:464] REPLICATION: finished
0709 12:25:23.422000 (+341782us) write_op.cc:301] APPLY: starting
0709 12:25:23.433028 (+ 11028us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:23.447150 (+ 14122us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:23.543857 (+ 96707us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:23.559572 (+ 15715us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:23.559762 (+   190us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:25:23.559826 (+    64us) write_op.cc:312] APPLY: finished
0709 12:25:23.560091 (+   265us) log.cc:844] Serialized 29 byte log entry
0709 12:25:23.560303 (+   212us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:23.560420 (+   117us) write_op.cc:454] Released schema lock
0709 12:25:23.560472 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:23.560873 (+   401us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:23.560898 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":105717,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":301864,"apply.run_cpu_time_us":2422,"apply.run_wall_time_us":152908,"num_ops":2,"prepare.queue_time_us":81809,"prepare.run_cpu_time_us":4616,"prepare.run_wall_time_us":345626,"replication_time_us":181065,"thread_start_us":218608,"threads_started":2}]]}
I20260709 12:25:23.905225  6891 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.905823  6891 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.852322  6679 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:23.970840  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:23.971498  4676 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:23.966099  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:23.966131  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:23.970759  6563 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:25:23.966864  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 63317.8us
I20260709 12:25:23.973173  4647 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:23.973704  6563 raft_consensus.cc:697] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:23.975548  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:23.970191  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 87958us
I20260709 12:25:23.875991  6900 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:23.986073  4748 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:23.801748  6922 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 12:25:23.908000  6669 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.994647  4571 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:23.998536  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:23.999269  4748 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.002416  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:24.006505  6563 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:23.973820  6767 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:23.988124  6940 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:23.988829  6954 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:25:24.014895  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:24.051451  4755 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:25:24.051502  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:24.052238  4750 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:25:24.053203  5314 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:23.991195  6844 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:23.991060  6877 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:23.975462  6912 consensus_queue.cc:1059] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:25:23.991628  6891 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:24.071902  5344 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:24.073323  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:24.074110  5347 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
W20260709 12:25:24.044435  6834 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.774s	user 0.008s	sys 0.003s
I20260709 12:25:24.041980  6840 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:25:24.066529  6966 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.157s	user 0.002s	sys 0.000s
I20260709 12:25:24.048336  6940 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.009321  6669 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.051380  6962 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:24.072901  6762 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:24.033531  6964 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:25:24.078608  6877 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:24.037189  6935 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.091s
I20260709 12:25:23.988145  6973 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.145s
I20260709 12:25:24.045279  6876 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.101263  6785 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:24.148685  4748 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:25:24.150772  4751 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:24.048437  6915 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.112973  6773 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.115078  6940 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:24.010670  6863 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.116s
I20260709 12:25:24.160187  6915 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:24.160806  6961 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.270s	user 0.000s	sys 0.002s
I20260709 12:25:24.123767  6900 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:24.051039  6767 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.122558  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:24.163559  4596 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.172199  4604 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:24.146003  6962 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:24.202814  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 207987us
W20260709 12:25:24.203301  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 374006us
I20260709 12:25:24.147055  6762 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:24.218117  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:24.220464  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:24.200268  6936 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:24.208437  6934 consensus_meta.cc:287] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.000s
I20260709 12:25:24.223788  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:24.206457  6912 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.162133  6900 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.225768  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:24.210496  6962 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:24.226611  4719 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:24.150928  6877 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.208616  5343 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:24.193209  6773 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.229423  6934 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:24.224700  6785 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 12:25:24.286875  4645 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:24.289615  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:24.227376  6840 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:24.276481  6965 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:25:24.281772  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:24.284273  4601 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:24.215380  6964 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:25:24.228514  6679 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:24.214738  6894 consensus_queue.cc:1059] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 12:25:24.293501  6983 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:24.316768  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:24.317413  4602 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:24.306582  4598 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:24.307016  6762 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.310985  6921 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:24.298827  6906 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.319967  4604 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:25:24.320775  4599 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:24.348519  4604 pending_rounds.cc:85] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 4
I20260709 12:25:24.348896  4604 pending_rounds.cc:107] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:25:24.349223  4604 raft_consensus.cc:2889] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:24.247437  4596 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:24.251322  6834 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:24.322103  4645 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:24.357084  4676 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:24.240240  6936 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.360641  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:24.336148  6599 consensus_queue.cc:1059] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
W20260709 12:25:24.364269  6937 consensus_meta.cc:287] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.000s
I20260709 12:25:24.364727  6937 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.295634  6840 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.306057  6876 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:24.238379  6912 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.334813  6921 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:24.324438  6983 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:24.374583  6969 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 12:25:24.408310  4752 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:24.374673  6840 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.298760  6854 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:25:24.381542  6768 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:24.419423  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.410395  6669 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:24.443660  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 87167.4us
I20260709 12:25:24.409098  6983 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.410243  6834 raft_consensus.cc:697] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:24.335690  6957 consensus_queue.cc:1059] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 12:25:24.427953  6768 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:25:24.446525  6892 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.931s	user 0.009s	sys 0.004s
I20260709 12:25:24.443540  6978 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:24.465986  4674 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:24.470988  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:24.463922  6768 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.475165  6892 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:24.469715  6679 raft_consensus.cc:697] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:25:24.488305  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 64531.4us
I20260709 12:25:24.477768  6978 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:24.483790  6767 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:25:24.483476  6773 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:24.492565  4597 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
W20260709 12:25:24.489305  6909 consensus_meta.cc:287] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.000s
W20260709 12:25:24.490430  6907 consensus_meta.cc:287] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.844s	user 0.006s	sys 0.007s
I20260709 12:25:24.503062  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:24.503722  4753 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:24.494992  6900 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:24.510529  4647 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:24.502841  6877 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:24.506857  6958 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:24.526345  6978 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.528249  6978 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.503072  6834 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:24.544953  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2208) took 1206 ms. Trace:
I20260709 12:25:24.545475  4756 rpcz_store.cc:276] 0709 12:25:23.338541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.338663 (+   122us) service_pool.cc:224] Handling call
0709 12:25:23.342497 (+  3834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.855005 (+512508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.543405 (+688400us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b54001b1054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:24.544875 (+  1470us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1177194752}
I20260709 12:25:24.550925  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:24.557590  4602 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:24.547678  6993 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:25:24.561110  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:24.562203  5344 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:24.505505  6907 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.545293  6937 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:24.684352  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:24.686281  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:24.686933  4602 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:24.691403  4750 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:24.555670  6679 consensus_queue.cc:237] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.700342  6863 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.689314  6788 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:24.699327  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:24.683245  6892 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.709188  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:24.691380  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:24.713495  6999 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:24.697757  6778 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:24.698813  6937 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:24.705833  6788 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.714108  6991 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.222s
I20260709 12:25:24.700875  6863 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.713411  6995 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:25:24.727180  6863 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:24.722121  6907 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:24.721295  6980 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
W20260709 12:25:24.721232  6982 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:25:24.719209  6778 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.720330  6937 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.749372  6905 raft_consensus.cc:3055] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:24.749620  6937 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:24.749796  6996 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.712232  6892 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.750000  6905 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:24.724468  6940 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.518s	user 0.002s	sys 0.005s
I20260709 12:25:24.720935  6788 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:24.750454  6996 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.754274  6989 raft_consensus.cc:3055] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:25:24.761000  6926 consensus_meta.cc:287] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.901s	user 0.009s	sys 0.003s
I20260709 12:25:24.752445  6863 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:24.788581  6996 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:24.773691  6989 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:24.783416  6863 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.798338  6905 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.804884  6995 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.807466  6863 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:24.806073  6978 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:24.726147  6999 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.806813  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:24.763067  6962 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.814050  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:24.803279  6983 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:24.815002  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 338954us
I20260709 12:25:24.818035  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:24.807927  6912 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:24.818150  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:24.766059  6940 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.819075  4599 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.807927  6905 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.839807  4596 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:25:24.841117  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:24.842275  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:24.843226  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:24.843847  4600 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:24.844936  6999 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:24.846135  6996 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:24.854039  6996 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.855223  6962 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:24.856024  6996 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:24.807700  6995 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:24.864003  6778 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:24.846330  6907 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:24.865353  6907 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.868029  6907 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:24.821913  6892 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:24.816442  6926 consensus_queue.cc:1059] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:25:24.846043  6927 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:24.815050  6978 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.851152  6958 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:24.895742  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:24.896612  4602 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
W20260709 12:25:24.908051  4675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1422) took 1542 ms (client timeout 1509 ms). Trace:
W20260709 12:25:24.908591  4675 rpcz_store.cc:269] 0709 12:25:23.365343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.365605 (+   262us) service_pool.cc:224] Handling call
0709 12:25:24.907765 (+1542160us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b540017dc54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:24.907941 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1382538624}
I20260709 12:25:24.895599  6999 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:24.904276  7002 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.880967  6932 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:24.914657  6940 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:24.895406  6991 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.894042  7000 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:24.915122  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:24.929852  4602 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:24.895340  6907 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:24.913205  6999 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.885138  6927 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.914379  7002 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.851523  6989 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.939450  5342 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:24.948930  6940 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:24.966106  6999 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:24.966619  7002 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:24.938148  6907 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.953830  6978 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:24.894507  6958 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.940675  6991 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.968652  6803 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:24.970986  7004 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:25:24.943073  6932 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.969930  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:24.981762  6803 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:24.971767  6995 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:24.972596  6907 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:24.973802  6840 consensus_meta.cc:287] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.549s	user 0.000s	sys 0.012s
I20260709 12:25:24.974483  6983 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:24.970497  6940 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:24.978731  6768 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.980005  4599 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:24.936774  7000 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:24.979249  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:24.986361  4568 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:24.990701  6803 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:24.977783  6999 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:25:24.991689  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 292404us
I20260709 12:25:24.993480  4677 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:25.175536  6936 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.180639  6940 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:25.204185  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 217897us
I20260709 12:25:25.249639  5344 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:24.988425  7001 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:25:25.158351  6768 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.140293  6978 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:25.267424  5342 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:25.267568  6978 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.166556  7012 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.254227  6959 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:25.129371  6907 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:25.050701  7000 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.266155  6787 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.050026  6927 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:24.988256  6840 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.276250  5344 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:25.269793  6940 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:25.282055  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:25.285656  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:25.296162  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.296906  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.297312  4599 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:25.286033  4600 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:25.286564  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.270699  6959 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:25.290155  6932 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.268680  6902 ts_tablet_manager.cc:1929] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.4
I20260709 12:25:25.293627  6978 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.123696  6991 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.298177  4605 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:25.305851  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.306460  4605 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:25.308274  4605 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:25.298012  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:25.309557  4599 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 4
W20260709 12:25:25.276446  6921 consensus_meta.cc:287] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.862s	user 0.011s	sys 0.001s
I20260709 12:25:25.292374  6863 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.314123  4601 raft_consensus.cc:2393] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:25.299727  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:25.300763  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.301697  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.306098  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.297996  7016 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.269136  6999 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.298929  6959 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:25.323386  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.306788  6902 log.cc:1199] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-3-root/wals/1b7b0c6c811e455abe898cb4d21acad4
I20260709 12:25:25.301764  6940 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.288767  6787 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.274770  6907 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.271627  7012 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:25.334704  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2258) took 2110 ms. Trace:
I20260709 12:25:25.401716  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:25.442174  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.427229  4599 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:25.440793  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:25.455320  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 53705.4us
I20260709 12:25:25.456826  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:25.450554  4672 rpcz_store.cc:276] 0709 12:25:23.214765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.215184 (+   419us) service_pool.cc:224] Handling call
0709 12:25:23.226480 (+ 11296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.787217 (+560737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.324435 (+1537218us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b54001bbf54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:25.324677 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1161887616}
I20260709 12:25:25.315035  6958 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:25.477054  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:25.478171  4675 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:25.402252  6803 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:25.448212  6999 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.478986  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.453927  6895 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.479168  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:25.334131  6928 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.482753  6803 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.491421  6803 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:25.491981  6803 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:25.492515  6803 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.404438  6978 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.480079  4756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:25.496558  4756 raft_consensus.cc:606] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:25.496821  4756 raft_consensus.cc:614] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:25.471369  7014 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:25.471757  6990 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:25:25.315137  6921 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:25.500015  6894 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.118s	user 0.014s	sys 0.033s
I20260709 12:25:25.458268  6679 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:25.481688  4672 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:25.499048  6834 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:25.516130  6854 raft_consensus.cc:3055] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:25:25.516544  6788 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.744s	user 0.010s	sys 0.000s
I20260709 12:25:25.499802  6863 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:25.513316  6894 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.137s	user 0.014s	sys 0.034s
I20260709 12:25:25.483381  6895 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:25.522831  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 43761.9us
I20260709 12:25:25.519045  6854 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:25.524751  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.325994  7016 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.521929  6832 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.525894  6832 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.511675  6902 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37154 (request call id 1) took 3743 ms. Trace:
I20260709 12:25:25.527185  6902 rpcz_store.cc:276] 0709 12:25:21.853397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.875103 (+ 21706us) service_pool.cc:224] Handling call
0709 12:25:22.318635 (+443532us) ts_tablet_manager.cc:1151] Deleting tablet 1b7b0c6c811e455abe898cb4d21acad4
0709 12:25:22.326062 (+  7427us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:25:24.515860 (+2189798us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:25.257619 (+741759us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:25.499337 (+241718us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":158059,"tablet-delete.queue_time_us":200140,"thread_start_us":159239,"threads_started":1}
I20260709 12:25:25.527951  6832 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.528107  7012 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.528671  4672 raft_consensus.cc:2410] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:25:25.523561  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:25.509691  6978 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.532203  6854 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.538292  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.538945  4680 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:25.539556  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.539927  4680 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:25.534845  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:25.541224  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.541827  4680 raft_consensus.cc:2410] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:25:25.541842  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:25.533839  6854 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.538530  6990 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:25:25.540274  6894 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:25.520984  6788 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.547749  6894 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.548169  6625 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:25.548784  6625 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:25.548919  7016 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.533232  6921 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:25.549609  6788 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.548278  6894 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:25.544384  6769 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:25.544795  6907 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.544939  6895 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:25.545022  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2215) took 1532 ms. Trace:
I20260709 12:25:25.544143  6940 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.546525  6803 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.546587  6978 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:25.547021  4597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 1768) took 1789 ms (client timeout 2063 ms). Trace:
I20260709 12:25:25.554771  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:25.555091  4597 rpcz_store.cc:269] 0709 12:25:23.757345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.757656 (+   311us) service_pool.cc:224] Handling call
0709 12:25:25.546856 (+1789200us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b54003548d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:25.546964 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1620089216}
I20260709 12:25:25.556005  4672 raft_consensus.cc:2393] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 6: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:25.547077  7022 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:25.551297  4680 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:25.550393  6769 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.558842  4672 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:25.559751  6907 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:25.560058  6840 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.560582  6769 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:25.552909  4751 rpcz_store.cc:276] 0709 12:25:24.012826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.037346 (+ 24520us) service_pool.cc:224] Handling call
0709 12:25:24.150510 (+113164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.156447 (+  5937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.544840 (+1388393us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54000730d0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:25.544962 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259113344}
I20260709 12:25:25.548202  6959 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.560925  6937 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:25.561424  6937 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:25.557782  6870 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:25.554525  6958 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:25.563337  6958 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.563781  6958 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.562441  6937 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.565085  6625 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.566211  6937 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:25.566320  6625 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:25.557991  6787 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.562767  6959 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.573026  4498 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 161) took 1822 ms. Trace:
I20260709 12:25:25.594568  4604 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:25.596048  6940 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.596613  6940 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.597167  6909 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.597702  6788 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:25.597719  6909 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.598209  6788 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.598848  6940 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.596093  7012 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:25.599607  7012 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.600157  6788 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.600270  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:25.601470  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:25.601759  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:25.602281  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:25.602778  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:25.603523  6937 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.603919  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:25.604018  6937 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.605311  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:25.606106  5343 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:25.606585  6767 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.607061  6767 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.608228  6909 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.608880  6909 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:25.610265  7012 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.610800  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.611408  4680 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:25:25.612330  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.613325  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:25.613863  4680 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:25.614717  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.602317  6677 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:25:25.609444  6767 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.575269  6958 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:25.577732  4485 catalog_manager.cc:5028] TS dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): tablet 1b7b0c6c811e455abe898cb4d21acad4 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
W20260709 12:25:25.621833  4485 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 44250.3us
I20260709 12:25:25.622910  6958 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:25.623950  6937 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:25.624622  5348 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:25.625994  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:25.626276  6912 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.626824  6912 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.627946  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.628657  4673 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:25.628821  6912 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.629266  6912 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:25.629765  7012 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:25.629900  6912 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.630303  7012 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:25.630788  7012 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:25.632539  6912 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.634052  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.634639  4677 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:25.634894  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.635231  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.636221  4672 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:25:25.578038  6959 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:25.581449  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1438) took 1719 ms. Trace:
I20260709 12:25:25.583057  6894 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:25.583626  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:25.585209  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.585225  6963 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:25.586720  6989 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:25.587468  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1771) took 2208 ms. Trace:
I20260709 12:25:25.592161  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1415) took 1723 ms. Trace:
I20260709 12:25:25.593091  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1444) took 1236 ms. Trace:
I20260709 12:25:25.593024  6840 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.601858  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:25.574448  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:25.673058  5314 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:25.841531  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:25.919626  5346 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:25.919638  7018 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:25.920113  6894 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:25.920428  7018 raft_consensus.cc:697] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:25:25.921185  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 346780us
I20260709 12:25:25.556777  7022 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.921437  7018 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.769013  4644 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:25.922650  7022 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:25.923343  6897 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:25.638554  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:25.924661  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 183101us
I20260709 12:25:25.636575  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.667371  6785 raft_consensus.cc:3055] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:25.927386  6785 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:25.927596  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.928148  6891 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:25.928750  6891 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.695995  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:25.928534  6785 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.930310  6785 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:25.931022  6891 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.682543  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:25.609421  6909 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.745013  7012 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.934574  4603 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:25.936182  7012 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.766458  7023 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.665938  6999 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.937178  4681 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:25.664870  6940 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.708796  6788 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.938764  6840 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.938849  6999 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:25.937630  4679 rpcz_store.cc:276] 0709 12:25:24.356249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.356358 (+   109us) service_pool.cc:224] Handling call
0709 12:25:24.356733 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.379411 (+ 22678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.379453 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:24.379516 (+    63us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:24.379545 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.592896 (+1213351us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b54000b1dd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:25.592934 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.593037 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51706112}
I20260709 12:25:25.663218  4751 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:25.943969  6996 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.938418  6963 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.949220  6999 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.638762  6773 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:25.937988  7023 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.950498  6773 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.671203  4675 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:25.954618  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.874763  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:25.941121  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.923241  6854 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:25.956351  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:25.956657  6854 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:25.957070  4681 raft_consensus.cc:2393] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 7: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:25.957196  6854 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.958714  7022 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.959313  6854 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.854072  6991 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:25.959987  6991 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.938498  6940 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.657974  4756 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:25.961877  4677 raft_consensus.cc:2393] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 4: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:25.962803  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.961436  6909 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.665531  6904 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.964283  6904 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.963845  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:25.966305  6904 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.790825  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.960466  6963 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.967669  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.968324  4599 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:25.969348  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.969619  4645 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:25.642313  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.970520  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.970887  4601 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:25.971334  6904 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:25.971778  6904 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.725098  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:25.953001  6773 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.697901  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.652854  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1780) took 1431 ms. Trace:
I20260709 12:25:25.928246  4605 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:25.973538  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.973825  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.974187  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.973855  4645 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:25.974694  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.944501  6996 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:25.938910  6788 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.975618  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:25.977860  6788 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.978348  4605 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:25.952970  7023 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.978586  6788 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:25.940932  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.963519  6940 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.979072  6788 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.831728  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:25.664372  6801 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:25.979642  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:25.980129  6801 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.654561  4676 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:25.980995  4674 rpcz_store.cc:276] 0709 12:25:23.862395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.862575 (+   180us) service_pool.cc:224] Handling call
0709 12:25:23.862903 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.468303 (+605400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.581194 (+1112891us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b54000b27d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:25.581385 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1380461440}
I20260709 12:25:25.982455  4674 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:25.982999  6801 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.724689  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:25.983570  5346 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:25.926117  4644 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:25.593787  4498 rpcz_store.cc:276] 0709 12:25:23.750999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.751351 (+   352us) service_pool.cc:224] Handling call
0709 12:25:25.572944 (+1821593us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:24.489035 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:24.690823 (+201788us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:24.696034 (+  5211us) write_op.cc:435] Acquired schema lock
0709 12:25:24.698673 (+  2639us) tablet.cc:662] Decoding operations
0709 12:25:24.753212 (+ 54539us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:24.761486 (+  8274us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:24.761524 (+    38us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:25:24.761955 (+   431us) tablet.cc:701] Row locks acquired
0709 12:25:24.761972 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:25:24.762130 (+   158us) write_op.cc:270] Start()
0709 12:25:24.765802 (+  3672us) write_op.cc:276] Timestamp: P: 1783599924762086 usec, L: 0
0709 12:25:24.767109 (+  1307us) op_driver.cc:348] REPLICATION: starting
0709 12:25:24.822098 (+ 54989us) log.cc:844] Serialized 1736 byte log entry
0709 12:25:25.294269 (+472171us) op_driver.cc:464] REPLICATION: finished
0709 12:25:25.508123 (+213854us) write_op.cc:301] APPLY: starting
0709 12:25:25.508211 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:25.511192 (+  2981us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:25.522246 (+ 11054us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:25.526208 (+  3962us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:25.526389 (+   181us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:25:25.526434 (+    45us) write_op.cc:312] APPLY: finished
0709 12:25:25.526659 (+   225us) log.cc:844] Serialized 41 byte log entry
0709 12:25:25.526827 (+   168us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:25.526918 (+    91us) write_op.cc:454] Released schema lock
0709 12:25:25.526971 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:25.527297 (+   326us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:25.527325 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":213551,"apply.run_cpu_time_us":2208,"apply.run_wall_time_us":19174,"num_ops":4,"prepare.queue_time_us":588312,"prepare.run_cpu_time_us":63648,"prepare.run_wall_time_us":455819,"raft.queue_time_us":321000,"raft.run_cpu_time_us":13005,"raft.run_wall_time_us":179491,"replication_time_us":526153,"thread_start_us":773721,"threads_started":4,"wal-append.queue_time_us":109042}]]}
I20260709 12:25:25.891140  6912 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.990500  6801 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:25.991014  6801 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.990897  6912 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.991474  6801 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.992251  7022 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:25.654765  6563 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.993611  6912 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.993516  6563 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:25.995173  4600 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:25:25.660912  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:25.997093  4645 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:25.836472  6787 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:25.997447  4753 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:25.972219  6904 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:25.977908  6996 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.978432  7012 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:25.998775  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:25.998888  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:25.957825  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:26.000209  4681 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:25.997885  6787 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.004377  6787 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.979555  4679 raft_consensus.cc:2393] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 4: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:25.980080  4598 rpcz_store.cc:276] 0709 12:25:23.868741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.868912 (+   171us) service_pool.cc:224] Handling call
0709 12:25:23.869234 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.318002 (+448768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.591951 (+1273949us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b5400060754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:25.592103 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1005324672}
I20260709 12:25:26.020305  6940 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:25.752945  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:26.023756  4603 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:26.024070  5349 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:25.833716  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:26.024688  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:25.986522  4752 rpcz_store.cc:276] 0709 12:25:23.379628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.380173 (+   545us) service_pool.cc:224] Handling call
0709 12:25:23.409703 (+ 29530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.414311 (+1004608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.587214 (+1172903us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x7b540030a554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:25.587406 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2289252608}
I20260709 12:25:25.629436  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:25.690819  4747 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:26.025964  6940 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.026400  6940 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:26.028187  4677 raft_consensus.cc:2410] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:25.662915  6892 raft_consensus.cc:3055] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:25.979704  6788 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:25.636564  4672 pending_rounds.cc:85] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 6
I20260709 12:25:25.846184  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:25.659121  6967 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.232s	user 0.005s	sys 0.010s
I20260709 12:25:26.029320  4672 pending_rounds.cc:107] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 5.7
W20260709 12:25:26.029587  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.029726  4672 raft_consensus.cc:2889] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:25.912575  4604 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:25.637538  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:25.946471  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 273560us
I20260709 12:25:25.712937  6983 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.989677  6786 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:25:25.989908  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 63882.7us
I20260709 12:25:25.989863  4750 rpcz_store.cc:276] 0709 12:25:24.221029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.221173 (+   144us) service_pool.cc:224] Handling call
0709 12:25:24.221496 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.695528 (+474032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.652642 (+957114us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b5400309b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:25.652795 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080288000}
I20260709 12:25:25.995398  6563 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:25.996877  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.658424  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:25.998188  6999 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:25.998309  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:25.945724  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:25.972986  6991 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:25.922400  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:25.979457  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:26.044566  4605 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:26.006321  7012 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.975003  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:26.050261  4597 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:26.015861  4499 catalog_manager.cc:5697] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.018006  4675 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:26.019217  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:25.963866  6840 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:25.984576  7023 raft_consensus.cc:3055] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:26.024276  4681 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:26.031164  5314 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:26.036760  4748 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:25:26.031247  6892 raft_consensus.cc:740] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:25.868619  4680 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:26.039728  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.039739  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.042641  4598 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:26.043190  4644 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:26.043519  4603 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:26.044642  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 122304us
I20260709 12:25:26.044564  6563 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:25.971899  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:26.085156  4601 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 2.
I20260709 12:25:26.047825  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.054421  4599 raft_consensus.cc:2393] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:26.055461  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:26.056325  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:26.057475  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.084558  4679 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:26.011987  6904 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:26.041950  6999 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.057548  4499 catalog_manager.cc:5697] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 5 to 6, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 6 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.035586  6989 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:26.042161  6983 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.236553  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:26.233745  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 238945us
I20260709 12:25:26.297665  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:26.212538  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2263) took 2423 ms. Trace:
W20260709 12:25:26.234108  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 202556us
W20260709 12:25:26.167145  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.239638  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 155474us
I20260709 12:25:26.298816  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:26.298382  4678 rpcz_store.cc:276] 0709 12:25:23.789141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.789638 (+   497us) service_pool.cc:224] Handling call
0709 12:25:23.789974 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:23.790347 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.790373 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:23.793602 (+  3229us) log.cc:844] Serialized 26 byte log entry
0709 12:25:23.973858 (+180256us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:23.973922 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.974002 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.658438 (+1684436us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:26.212144 (+553706us) raft_consensus.cc:1685] finished
0709 12:25:26.212176 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.212454 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18488,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":40135,"spinlock_wait_cycles":4124032,"thread_start_us":178889,"threads_started":1,"wal-append.queue_time_us":534483}
I20260709 12:25:26.299830  4644 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:26.103272  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:26.107262  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 51017.6us
W20260709 12:25:26.291410  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 707796us
I20260709 12:25:26.239382  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:26.260612  4499 catalog_manager.cc:5697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 5 to 6, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 6 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.204877  7018 raft_consensus.cc:2764] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:25:26.096750  7007 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:25:26.270403  7028 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:26.256014  6891 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:26.056387  7012 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.278750  6996 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:26.217147  6900 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:26.109091  6999 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:26.203617  6863 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.215754  6935 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:26.167485  6563 raft_consensus.cc:697] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:26.303593  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:26.334174  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:26.277382  5343 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:25:26.334879  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:26.335281  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.335528  4596 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:26.336105  6863 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:26.338706  6863 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:26.303936  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.340121  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.340806  4674 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:26.340588  6788 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:26.340747  7032 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:25:26.326812  6989 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:26.341820  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:26.295205  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:26.342388  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:26.342995  4605 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:26.301200  4598 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:26.342597  6989 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.305147  4678 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 12:25:26.306413  4719 consensus_peers.cc:597] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:25:26.344698  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 46758.2us
I20260709 12:25:26.326988  7007 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:26.345811  6935 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.346318  6935 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:26.346591  7007 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:26.307058  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:26.348053  4603 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:26.334177  6563 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:26.352001  6801 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:26.353005  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:26.353659  6900 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.355892  6900 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:26.328984  6991 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:26.357050  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:26.357668  4600 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:26.358036  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:26.358721  4752 raft_consensus.cc:2368] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3.
I20260709 12:25:26.308344  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:26.359200  6991 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:26.360435  4644 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:26.360381  6892 consensus_queue.cc:260] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.362020  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:26.308636  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.363555  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:26.364253  6912 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:26.309876  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.365818  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:26.366362  4675 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:26.293668  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.293071  6669 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:26.367291  6669 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:26.368392  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:26.368235  6669 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:26.058832  7023 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:26.370728  6983 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:26.370545  7023 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:26.334826  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.371758  7023 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:26.372184  6983 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:26.372587  6983 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:26.372939  6983 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:26.373057  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:26.167464  6992 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.373754  4597 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:26.373966  6992 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.375761  7012 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:26.317158  6787 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:26.376205  6992 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:26.328480  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:26.376845  4673 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:26.376572  6787 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:26.377547  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49179.9us
I20260709 12:25:26.377923  4674 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:26.378561  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:26.379408  6904 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:26.379627  4569 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:26.380089  6677 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:26.330549  6599 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.300004  6995 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:26.343992  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.399742  4676 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:26.303467  4499 catalog_manager.cc:5697] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 3 to -1, term changed from 4 to 6, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) evicted, now has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 6 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:26.402128  4499 catalog_manager.cc:5697] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 2 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:25:26.369532  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:26.381302  4747 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43852
I20260709 12:25:26.405397  4747 raft_consensus.cc:606] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Received request to transfer leadership
I20260709 12:25:26.381358  6599 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.406544  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:26.051910  6877 consensus_queue.cc:1059] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.269s
I20260709 12:25:26.407666  6840 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:26.408179  6840 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.408517  6840 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.408968  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:26.409271  6801 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:26.403749  4499 catalog_manager.cc:5697] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 2 to 3, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.383813  4605 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:26.366536  4681 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:26.386601  4645 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:26.391057  4603 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
W20260709 12:25:26.432520  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 45997.9us
W20260709 12:25:26.410189  7030 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.387s	user 0.001s	sys 0.000s
I20260709 12:25:26.434314  6801 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:26.434728  6801 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.435120  6801 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:26.438786  6788 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:26.332930  6996 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.441112  6996 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:26.441330  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:26.441824  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:26.441973  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:26.442432  5343 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEARNER]: Advancing to term 5
I20260709 12:25:26.442538  4749 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.442814  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:26.443008  6904 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:26.443444  4751 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:26.445101  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:26.445132  6991 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:26.445843  4754 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
I20260709 12:25:26.445999  6935 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.447006  5342 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:26.448168  6935 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:26.215987  6894 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 12:25:26.450173  6894 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:26.450547  6894 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:26.451452  6894 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.452479  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:26.452661  6894 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:26.398667  4569 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:26.383131  4672 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 6: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:26.431535  5348 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:25:26.430264  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:26.410279  6786 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:26.458626  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:26.411800  4499 catalog_manager.cc:5697] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 4 to 6, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 6 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:25:26.459286  4755 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
W20260709 12:25:26.448294  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.460464  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.430009  6892 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:26.466574  6915 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:26.447049  4754 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:26.467181  6915 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.454707  4604 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:26.430745  6773 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:26.469643  6773 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:26.437240  4675 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:26.396869  4680 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:25:26.449497  6599 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:26.395332  4748 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:26.417922  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:26.457834  4605 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:26.417431  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:26.420095  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:26.466226  6999 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.327020  6891 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.459918  4499 catalog_manager.cc:5697] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:25:26.833415  4568 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:26.503525  4604 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:26.475744  5349 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:26.416235  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:26.821700  5312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.001s	sys 0.013s
W20260709 12:25:26.836503  5312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.001s	sys 0.013s
I20260709 12:25:26.501107  4749 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:26.838379  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:26.834604  4598 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:26.474078  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 75525us
I20260709 12:25:26.418593  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:26.835322  4603 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:26.831430  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.843094  4750 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
W20260709 12:25:26.848359  5314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.007s	sys 0.011s
W20260709 12:25:26.850189  5314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.007s	sys 0.012s
W20260709 12:25:26.850538  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 551803us
I20260709 12:25:26.894485  6563 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:26.854928  5349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 397) took 1094 ms. Trace:
I20260709 12:25:26.432133  6863 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:26.900475  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.908703  4598 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
W20260709 12:25:26.895794  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 475779us
W20260709 12:25:26.902170  4571 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.504s (SENT) [suppressed 32 similar messages]
I20260709 12:25:26.910007  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:26.910275  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:26.902067  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:26.912782  4677 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:26.911051  4749 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:26.902825  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:26.912736  4604 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:25:26.735322  7037 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:26.715250  6940 consensus_meta.cc:287] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.680s	user 0.003s	sys 0.009s
I20260709 12:25:26.905352  4596 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
W20260709 12:25:26.873396  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.925493  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.846681  6904 raft_consensus.cc:2764] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:25:26.916169  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:26.918908  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.920380  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:26.901197  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.930085  4599 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:26.930240  4753 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:26.923046  6563 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:26.930611  4748 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:26.596567  6787 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:26.894215  6877 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:26.926846  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:26.936386  4605 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:26.569588  6915 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:26.489475  6935 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:26.928640  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:26.941298  4675 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:26.894883  6992 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.808145  6677 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:26.921672  4645 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:26.924414  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:26.945549  4598 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:26.846542  6773 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:26.458832  6788 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:26.897424  5349 rpcz_store.cc:276] 0709 12:25:25.742219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.752555 (+ 10336us) service_pool.cc:224] Handling call
0709 12:25:26.836815 (+1084260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:26.469787  6786 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:26.846249  6991 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:26.469585  6801 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.906417  6863 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:26.939991  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:26.964411  4603 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:26.940443  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:26.924912  7037 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.565533  6983 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:26.950156  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.411223  6995 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.949681  6677 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:26.935938  6785 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.938867  6996 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.927510  6940 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:26.944247  6935 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.982045  6983 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:26.982626  6983 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.965740  6599 consensus_queue.cc:1059] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:26.983191  6940 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:26.983075  6983 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.981508  6995 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:26.983191  6935 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:26.943485  6877 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:26.966118  4604 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:26.951782  4598 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:26.970577  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:26.960671  6959 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:26.960337  7043 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:26.983651  6599 raft_consensus.cc:3055] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:26.973251  6563 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:26.960184  7048 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:26.984236  6995 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:25:26.967622  6863 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.989289  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:26.992435  6599 raft_consensus.cc:740] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:26.974218  6894 consensus_meta.cc:287] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.000s
I20260709 12:25:26.951483  6992 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.987461  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:27.040468  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:26.962669  6788 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:26.971199  7037 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:27.040396  7054 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:25:27.042675  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2287) took 1003 ms. Trace:
I20260709 12:25:27.044708  6599 consensus_queue.cc:260] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.045739  4604 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:27.040742  4603 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:26.970927  6989 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:25:27.045614  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 58232.3us
I20260709 12:25:26.984525  6785 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.048771  4756 raft_consensus.cc:2410] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:25:26.970746  6897 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:25:26.966776  6991 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.026648  6935 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:27.048920  6940 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:27.057545  4677 rpcz_store.cc:276] 0709 12:25:26.039453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.047601 (+  8148us) service_pool.cc:224] Handling call
0709 12:25:26.991032 (+943431us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b540007b554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:27.042617 (+ 51585us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1900879104}
I20260709 12:25:27.073050  6599 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:26.967283  6786 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.074590  4681 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:27.076057  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:26.987555  6996 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:26.993829  7043 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:27.079490  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:27.091441  4605 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:27.092868  6897 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:27.093326  6785 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:27.083405  7053 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:25:27.090365  6999 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:27.093538  6897 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:27.103528  6897 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.094060  6983 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:27.106243  7049 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.094218  7050 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:25:27.096742  6840 consensus_meta.cc:287] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.688s	user 0.006s	sys 0.003s
I20260709 12:25:27.100955  6996 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:27.098320  7043 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.108039  6840 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.108896  6801 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:27.109171  4600 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:27.093946  6891 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:27.105549  4605 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:27.107153  7050 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.044718  6959 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:27.042274  7048 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.083364  6935 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.082587  6989 raft_consensus.cc:697] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:27.099285  6892 consensus_meta.cc:287] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.633s	user 0.002s	sys 0.006s
W20260709 12:25:27.110224  5343 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.667s	user 0.006s	sys 0.006s
W20260709 12:25:27.207835  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 356674us
I20260709 12:25:27.210127  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1803) took 1389 ms. Trace:
I20260709 12:25:27.210886  4602 rpcz_store.cc:276] 0709 12:25:25.821212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.054888 (+233676us) service_pool.cc:224] Handling call
0709 12:25:26.153598 (+ 98710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.153876 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:26.761944 (+608068us) log.cc:844] Serialized 14 byte log entry
0709 12:25:27.209624 (+447680us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:27.209706 (+    82us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:27.209779 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.209876 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.210058 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111199,"prepare.queue_time_us":178340,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":80543,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":26330,"thread_start_us":917549,"threads_started":3,"wal-append.queue_time_us":192124}
I20260709 12:25:27.214010  7058 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:27.214941  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 374846us
I20260709 12:25:27.114812  6940 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:27.221067  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:27.219539  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:27.221393  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:27.217008  4568 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:27.219079  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 287192us
I20260709 12:25:27.124979  6935 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:27.223305  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:27.082825  6677 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.224087  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:27.125463  5343 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEARNER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:27.223273  7061 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:27.228358  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:27.231873  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:27.238682  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 159249us
W20260709 12:25:27.239305  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 337133us
I20260709 12:25:27.252029  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:27.258047  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:27.261507  4753 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:27.250893  4645 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:27.262806  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.263589  4750 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:27.262619  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:27.107149  7049 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:27.266602  4750 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:27.265632  4645 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:27.268023  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.269132  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:27.270147  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 46182.8us
I20260709 12:25:27.270892  6599 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:27.270946  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:27.271777  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:27.272202  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:27.272584  6599 raft_consensus.cc:697] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:27.224622  7065 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:27.274734  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.275951  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.276082  7072 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:27.276707  6940 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.229514  6788 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:27.230137  7052 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:27.259610  6996 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.259390  6915 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:27.217619  7058 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.298720  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1449) took 1054 ms. Trace:
I20260709 12:25:27.229449  6989 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:27.278640  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:27.330847  4604 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:27.252249  6891 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:27.299062  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.300822  4749 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:27.302367  4750 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:27.303020  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:27.304039  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.297847  6677 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:27.304296  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:27.306340  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.308080  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.308410  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:27.308933  4753 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:27.307158  6599 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.273552  6897 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:27.318804  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.294943  7060 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:27.322237  4675 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:27.322625  6563 raft_consensus.cc:3055] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:27.323376  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1461) took 1051 ms. Trace:
I20260709 12:25:27.325827  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:27.119689  6877 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:27.328958  5346 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:27.326520  4601 rpcz_store.cc:276] 0709 12:25:26.243976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.418668 (+174692us) service_pool.cc:224] Handling call
0709 12:25:27.086505 (+667837us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.086665 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.086681 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:27.086707 (+    26us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:27.086730 (+    23us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.086764 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.298561 (+211797us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b540032c8d0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:27.298663 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1416274176}
I20260709 12:25:27.298848  7057 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:27.260161  7061 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:27.306717  7072 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:27.315742  6894 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.309854  7018 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 12:25:27.310062  6773 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:27.536834  4756 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:27.516940  4747 raft_consensus.cc:2393] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:25:27.538555  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.390494  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:27.530857  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 268285us
I20260709 12:25:27.520772  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:27.539645  6563 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:27.537200  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.542465  4571 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:27.536947  4754 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 3: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:27.543569  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:27.544334  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.545221  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.546082  4754 raft_consensus.cc:2393] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:27.543569  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.547072  4753 raft_consensus.cc:2410] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:27.547865  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:27.548682  4681 raft_consensus.cc:2393] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:27.548588  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.547801  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:27.549331  4677 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:27.550292  4645 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:27.312690  6983 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:27.553932  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.540346  4600 rpcz_store.cc:276] 0709 12:25:26.272266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.552466 (+280200us) service_pool.cc:224] Handling call
0709 12:25:26.569249 (+ 16783us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.109641 (+540392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.323158 (+213517us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b540032b4d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:27.323317 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1241252224}
I20260709 12:25:27.554579  4748 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:27.555738  4748 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:27.556016  4754 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:25:27.539215  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 232974us
W20260709 12:25:27.386631  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 165252us
I20260709 12:25:27.553457  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.564482  4749 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
W20260709 12:25:27.566347  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.566716  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.568041  4754 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:27.567217  4750 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:27.570539  6563 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.324882  7052 raft_consensus.cc:697] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:27.325457  6996 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:27.607353  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47630.5us
I20260709 12:25:27.608096  4596 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:25:27.644587  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 332874us
I20260709 12:25:27.609217  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.646270  4752 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
W20260709 12:25:27.646728  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 99236.8us
I20260709 12:25:27.605095  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:27.649305  4672 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.651098  4755 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:27.575407  5314 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:27.654167  6563 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:27.384622  6991 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:27.654776  4753 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
W20260709 12:25:27.652366  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 77079.8us
I20260709 12:25:27.664763  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.666901  4747 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:27.540246  7060 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:27.562178  7064 raft_consensus.cc:2764] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:25:27.471560  6959 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:27.555125  7072 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:27.625244  7073 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 12:25:27.669241  4747 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:27.656203  4756 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:27.682250  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.679204  6991 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:25:27.330543  6915 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:27.385434  6992 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:27.570281  6904 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:27.625929  6892 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:27.727650  6892 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:27.728209  6892 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.557828  6897 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.675174  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:27.729833  6897 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:27.730207  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 55153us
I20260709 12:25:27.732106  4569 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:27.678722  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.603895  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:25:27.679960  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 99859.6us
I20260709 12:25:27.733860  4750 raft_consensus.cc:2410] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:25:27.733996  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.736776  4672 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:27.736954  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:27.670027  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.530411  7067 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.142s
I20260709 12:25:27.465337  7041 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.225s
I20260709 12:25:27.433825  6891 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:27.571027  7057 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:27.673295  7060 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:27.604125  7056 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 12:25:27.712505  7077 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:27.714457  7052 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:27.653990  7007 consensus_queue.cc:1059] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:25:27.604418  7066 consensus_queue.cc:1059] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:25:27.723723  6773 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:27.726392  7018 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:27.450210  6863 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.741724  4752 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:27.565027  6894 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:27.750305  6959 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:27.745415  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 99723us
W20260709 12:25:27.746057  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 104420us
I20260709 12:25:27.746501  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:27.795281  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 113618us
I20260709 12:25:27.709602  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.753656  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:27.796646  4747 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:27.628048  7058 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:27.713976  7061 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:27.707566  6904 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.752234  5342 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:27.623980  7048 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:27.770776  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:27.771418  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:27.663033  7049 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:27.774799  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:27.744020  7064 raft_consensus.cc:2764] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:25:27.781383  6677 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:27.789808  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 247421us
I20260709 12:25:27.834735  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:27.790746  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:27.835769  4749 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:27.737916  4674 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:25:27.796052  4754 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:27.838593  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:27.799849  7080 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:27.825611  4675 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:27.693929  7074 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.114s
I20260709 12:25:27.822157  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.798817  6959 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:27.798842  7043 consensus_meta.cc:287] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.690s	user 0.008s	sys 0.005s
I20260709 12:25:27.828483  4751 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:27.828574  4756 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:27.830801  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 80592.7us
I20260709 12:25:27.839645  4748 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:27.824717  6904 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:27.841260  4752 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:27.852079  4749 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:28.053344  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:28.053890  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.883540  7081 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.213s
I20260709 12:25:27.891175  7057 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.054548  4749 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 1.
I20260709 12:25:27.965867  5345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 363) took 1587 ms. Trace:
I20260709 12:25:27.888660  4604 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:28.055745  5314 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:27.995231  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.056864  4747 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
W20260709 12:25:28.058075  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.052735  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 214200us
I20260709 12:25:28.053658  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:28.058296  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:28.058910  4752 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:27.961117  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 112157us
I20260709 12:25:28.054056  4748 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:27.853839  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:28.062655  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:28.062880  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:28.064484  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:28.055388  5345 rpcz_store.cc:276] 0709 12:25:26.378482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.378641 (+   159us) service_pool.cc:224] Handling call
0709 12:25:26.378963 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:26.732026 (+353063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.732089 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:26.896857 (+164768us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.229803 (+332946us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:27.229852 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.229928 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:27.965658 (+735730us) raft_consensus.cc:1685] finished
0709 12:25:27.965676 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.965808 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":546860,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":10493,"thread_start_us":642571,"threads_started":2,"wal-append.queue_time_us":389929}
I20260709 12:25:27.825881  6915 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:27.829097  6992 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:28.080927  4752 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
W20260709 12:25:28.086992  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 233262us
I20260709 12:25:28.088675  4571 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:28.089468  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.838176  6677 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:27.886183  7078 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 12:25:27.883771  6863 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:27.884389  7075 log.cc:927] Time spent T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 12:25:28.092082  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.882851  7063 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:25:27.880138  6773 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:28.071192  7037 consensus_meta.cc:287] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.987s	user 0.011s	sys 0.002s
I20260709 12:25:27.883437  6995 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:28.097026  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.096832  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:27.894439  7067 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:28.061290  6959 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:28.090096  4752 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:27.965158  6785 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:27.874312  6996 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:28.085527  7049 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.103358  4748 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:27.884081  7084 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:25:28.121492  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 58704.3us
W20260709 12:25:28.141476  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 70494us
I20260709 12:25:28.144309  4720 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:28.092063  6891 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:28.153806  4747 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:27.971163  7072 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.097388  6677 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:28.160372  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:28.078156  7043 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.096774  7061 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:27.889667  7087 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.249s
I20260709 12:25:28.098212  6669 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:25:28.181509  5370 diagnostics_log.cc:230] Unable to collect stacks to diagnostics log: Incomplete: not collecting stack trace since debugger or strace is attached
I20260709 12:25:28.091516  6992 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.104877  6877 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:28.139186  6787 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.141969  7048 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.095597  7007 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:25:28.202574  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:28.138424  7089 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:28.166797  6996 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:28.208698  4756 raft_consensus.cc:2410] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:28.201957  6773 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:28.214084  5349 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:28.217792  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2234) took 1777 ms. Trace:
I20260709 12:25:28.155956  7067 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.163718  6995 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.165958  7080 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:25:28.158903  6785 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.154723  7037 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.202924  6669 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:28.442595  6669 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:28.232829  6773 consensus_queue.cc:1059] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.214s
I20260709 12:25:28.447979  6995 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:28.448873  6995 raft_consensus.cc:3055] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:28.222584  6787 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.451200  6891 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:28.226171  7048 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.452234  6891 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.452989  6787 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.453306  7085 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:25:28.431172  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 334793us
I20260709 12:25:28.455708  7048 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:28.456480  7048 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.433925  4605 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:25:28.435557  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 290189us
I20260709 12:25:28.221969  4751 rpcz_store.cc:276] 0709 12:25:26.440788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.444382 (+  3594us) service_pool.cc:224] Handling call
0709 12:25:26.444704 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.053506 (+1608802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.217429 (+163923us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400184054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:28.217715 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3181961600}
W20260709 12:25:28.438014  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.214411  6894 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:28.240139  6935 consensus_meta.cc:287] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.009s	user 0.010s	sys 0.003s
I20260709 12:25:28.439504  7107 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.212184  6877 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:28.490651  6877 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:28.490404  6935 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.167915  7049 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
W20260709 12:25:28.495548  7094 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.262s	user 0.008s	sys 0.000s
I20260709 12:25:28.460263  7045 raft_consensus.cc:3055] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:28.231671  7067 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:28.269870  6992 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:28.232506  6996 raft_consensus.cc:2764] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:28.498528  7049 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.267848  7043 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:28.452309  7037 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.205644  7061 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:28.516146  7061 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.231720  7072 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.461977  7088 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:28.499249  7045 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:28.518272  7088 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.215813  7089 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:28.172400  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1822) took 1247 ms. Trace:
I20260709 12:25:28.521991  4597 rpcz_store.cc:276] 0709 12:25:26.925202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.925322 (+   120us) service_pool.cc:224] Handling call
0709 12:25:26.925654 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:27.255293 (+329639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.255322 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:27.255811 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.580708 (+324897us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:27.581314 (+   606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.581410 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:28.172095 (+590685us) raft_consensus.cc:1685] finished
0709 12:25:28.172126 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:28.172317 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4041,"raft.queue_time_us":113011,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":7532,"thread_start_us":409087,"threads_started":2,"wal-append.queue_time_us":431389}
I20260709 12:25:28.521339  7089 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:28.462869  7083 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.464780  7085 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.527850  7083 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:28.529428  7085 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.529771  7048 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.530375  7083 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:28.468107  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 301938us
I20260709 12:25:28.530877  7088 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.532516  7048 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.532351  7066 consensus_queue.cc:1059] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:28.468528  6840 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.461334  7058 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:28.353013  7096 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:25:28.203329  7097 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:28.472031  5349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 402) took 1253 ms. Trace:
I20260709 12:25:28.547055  7097 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:28.546943  5349 rpcz_store.cc:276] 0709 12:25:27.219819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.219973 (+   154us) service_pool.cc:224] Handling call
0709 12:25:27.220324 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.221461 (+1001137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.471261 (+249800us) spinlock_profiling.cc:243] Waited 993 ms on lock 0x7b54004fb0d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:28.471698 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2284804352}
I20260709 12:25:28.547742  7097 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:28.548848  4647 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:28.549925  7058 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.473932  7077 raft_consensus.cc:3055] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:28.225708  6897 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.551397  7077 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:28.552573  7058 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.553858  6897 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.431723  4603 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:28.480140  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1800) took 1158 ms. Trace:
I20260709 12:25:28.477778  6785 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:28.482467  7050 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:28.507997  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.541123  7045 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.542379  7107 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:28.559798  5342 consensus_meta.cc:287] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.721s	user 0.004s	sys 0.010s
W20260709 12:25:28.560130  4605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 1831) took 1309 ms (client timeout 1578 ms). Trace:
I20260709 12:25:28.560274  5342 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:28.560933  5342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 366) took 1235 ms. Trace:
I20260709 12:25:28.552834  7077 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.449277  6995 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:28.561390  7107 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.552219  6877 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.562880  7077 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.480163  6891 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:28.559046  7045 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:28.555477  4753 rpcz_store.cc:276] 0709 12:25:27.321692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.548373 (+226681us) service_pool.cc:224] Handling call
0709 12:25:28.479677 (+931304us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400183654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:28.479875 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242620288}
I20260709 12:25:28.570050  7107 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.571972  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:28.572235  7107 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.573784  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:28.574990  7107 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:28.575930  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:25:28.560588  4605 rpcz_store.cc:269] 0709 12:25:27.250765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.251780 (+  1015us) service_pool.cc:224] Handling call
0709 12:25:28.559842 (+1308062us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400141ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:28.560006 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2694789504}
W20260709 12:25:28.565625  7093 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.332s	user 0.000s	sys 0.009s
I20260709 12:25:28.566365  4680 raft_consensus.cc:2368] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6.
I20260709 12:25:28.566219  6897 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:28.566454  6801 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:28.567390  6877 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:28.567629  7091 log.cc:927] Time spent T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.340s	user 0.002s	sys 0.005s
I20260709 12:25:28.561393  5342 rpcz_store.cc:276] 0709 12:25:27.325661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.328697 (+  3036us) service_pool.cc:224] Handling call
0709 12:25:28.560844 (+1232147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:28.569989  6992 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:28.588406  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:28.589716  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:28.592933  6999 raft_consensus.cc:3055] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:28.594137  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:28.596611  7056 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:28.616892  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:28.601001  6801 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.615265  6992 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:28.621054  7056 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:28.621259  6935 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.621413  6999 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:28.629371  6801 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:28.641237  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 92602.6us
I20260709 12:25:28.642311  4647 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:28.642731  7076 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:28.643160  7065 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:28.643671  7076 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.644542  6989 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.644802  7065 raft_consensus.cc:3055] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:28.645537  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:28.645236  6989 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.567883  6995 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:28.657835  6995 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:28.660918  5346 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:28.645275  7065 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:28.671603  6904 consensus_meta.cc:287] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.586s	user 0.008s	sys 0.010s
I20260709 12:25:28.671036  6677 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.672247  6904 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.675297  6904 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.675441  6677 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:28.690148  6940 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:25:28.690234  7065 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.691861  7065 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:28.692073  7060 consensus_meta.cc:287] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.850s	user 0.010s	sys 0.010s
I20260709 12:25:28.692521  7060 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.694900  7060 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:28.695186  7079 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.945s	user 0.002s	sys 0.016s
I20260709 12:25:28.697727  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:28.698087  7062 raft_consensus.cc:3055] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:28.698477  7062 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:28.698809  7007 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:28.699287  7007 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:28.699698  7007 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:28.699517  7062 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.700682  7062 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:28.638825  7089 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.704092  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1836) took 1160 ms. Trace:
I20260709 12:25:28.704669  7089 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:28.697211  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1830) took 1446 ms. Trace:
I20260709 12:25:28.707759  7056 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.708884  7056 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:28.704566  4603 rpcz_store.cc:276] 0709 12:25:27.543635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.544070 (+   435us) service_pool.cc:224] Handling call
0709 12:25:27.544369 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.568362 (+1023993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.703748 (+135386us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b54002e4fd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:28.704017 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2038911232}
I20260709 12:25:28.727995  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:28.728636  6999 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.730278  6999 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:28.731030  6892 consensus_meta.cc:287] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 1.002s	user 0.015s	sys 0.002s
I20260709 12:25:28.730619  7097 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.731493  6892 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.732733  4601 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:28.708256  4602 rpcz_store.cc:276] 0709 12:25:27.250297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.251690 (+  1393us) service_pool.cc:224] Handling call
0709 12:25:27.276834 (+ 25144us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:27.277157 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.277184 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:27.277764 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.517333 (+239569us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:27.517391 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.517453 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:28.694337 (+1176884us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:28.696580 (+  2243us) raft_consensus.cc:1685] finished
0709 12:25:28.696607 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:28.696962 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1513,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":10271,"spinlock_wait_cycles":5445504,"thread_start_us":199347,"threads_started":1,"wal-append.queue_time_us":367036}
I20260709 12:25:28.734035  6992 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.735457  6992 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:28.687541  7052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.016s	sys 0.027s
W20260709 12:25:28.737838  7052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.016s	sys 0.028s
I20260709 12:25:28.742420  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:28.747640  6991 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:28.749356  6669 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.750955  5345 raft_consensus.cc:1240] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:28.758512  6897 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:28.768545  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:28.769323  7097 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:28.771034  6877 consensus_queue.cc:1059] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:28.771783  4681 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:28.771970  6877 raft_consensus.cc:3055] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:28.772359  6877 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:28.773056  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:28.773386  6877 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.760823  6897 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:25:28.775636  6959 consensus_meta.cc:287] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.616s	user 0.006s	sys 0.006s
I20260709 12:25:28.795719  6877 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:28.796068  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 125723us
I20260709 12:25:28.796736  4569 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:28.802795  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:28.805008  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:28.795743  4596 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:28.809540  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:28.792384  5349 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:28.778532  6892 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:28.806546  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 446521us
I20260709 12:25:28.815431  7052 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:28.815801  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:28.816077  7052 raft_consensus.cc:697] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:28.816766  4673 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:28.815420  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:28.785542  6897 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:28.823515  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:28.877516  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.879175  4674 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:28.880460  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:28.881256  4673 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:28.890636  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:28.891817  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:28.878844  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:28.893808  7113 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:25:28.897940  7097 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:28.898599  7097 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:28.901309  4596 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:25:28.902465  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 84994.7us
I20260709 12:25:28.919128  4603 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:25:28.919641  4603 pending_rounds.cc:85] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 4
W20260709 12:25:28.924928  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.929003  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.919915  4603 pending_rounds.cc:107] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:25:28.812594  6959 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.944154  4603 raft_consensus.cc:2889] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:28.916023  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:28.931622  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:28.899606  7097 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:28.951784  4679 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:28.934772  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:28.952908  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:28.957078  4753 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:28.958736  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.961900  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:28.962659  4598 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:25:28.962510  6897 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:28.963946  6897 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:28.965731  7116 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
W20260709 12:25:28.975419  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.976176  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:28.977475  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:28.977910  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:28.978681  4673 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:28.978725  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:28.978960  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 161359us
I20260709 12:25:28.979377  4679 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:28.979758  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:28.980526  4673 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:28.985433  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:28.986584  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:28.986929  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:28.987893  4604 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:25:28.988457  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:28.989282  4756 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:28.936053  6989 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:28.957629  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:28.981460  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:28.974855  7007 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:28.935379  6995 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:28.953724  6669 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.001094  6959 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.001309  7007 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:29.001751  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:29.002156  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.002976  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:29.003302  4755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1796) took 1699 ms (client timeout 1718 ms). Trace:
I20260709 12:25:29.003888  4751 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:29.005091  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:28.991613  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.007093  4674 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 4: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:29.008652  4678 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:29.012705  4755 rpcz_store.cc:269] 0709 12:25:27.303968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.304078 (+   110us) service_pool.cc:224] Handling call
0709 12:25:29.002812 (+1698734us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b5400183154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:29.003067 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260635904}
W20260709 12:25:29.067598  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.066398  7070 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:25:29.030154  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:29.030443  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 106229us
W20260709 12:25:29.079159  4680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2299) took 1776 ms (client timeout 1515 ms). Trace:
I20260709 12:25:29.079568  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:29.080906  4604 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.064455  4673 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:29.083802  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.028039  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:29.079686  4680 rpcz_store.cc:269] 0709 12:25:27.302470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.302768 (+   298us) service_pool.cc:224] Handling call
0709 12:25:29.078827 (+1776059us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400201054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:29.079026 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2333621120}
I20260709 12:25:29.114878  4679 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:29.027627  6995 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:29.114491  4672 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 2.
I20260709 12:25:28.997864  6935 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 12:25:28.998402  7062 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 12:25:29.015942  6892 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:29.130376  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:29.130074  6892 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:29.135921  6892 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:29.136034  4674 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:29.130182  4756 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:28.990283  7052 consensus_queue.cc:237] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.129599  4679 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:25:29.143316  4753 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:29.160938  4569 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:29.161834  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:29.163192  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 152958us
I20260709 12:25:29.171583  4602 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:29.170055  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:29.158296  6959 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:29.175756  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:29.177682  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:29.181926  7129 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:25:29.182102  6787 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:29.182593  6787 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.182976  6787 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.187960  4756 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:29.198113  4681 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:29.204627  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:29.178186  6959 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:29.191102  6801 consensus_meta.cc:287] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.003s
I20260709 12:25:29.210299  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:29.216041  4750 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:29.217898  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:29.229400  4598 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:29.209399  6959 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:29.240685  4604 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:29.248106  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.246512  6877 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:29.211251  6801 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.277137  4748 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:25:29.277547  4748 pending_rounds.cc:85] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 4
I20260709 12:25:29.277786  4748 pending_rounds.cc:107] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:25:29.277995  4748 raft_consensus.cc:2889] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:29.272941  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 82727.9us
I20260709 12:25:29.275063  4644 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:29.274928  4754 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:29.228871  6935 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:25:29.225526  6995 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.287009  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:29.293210  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 118026us
W20260709 12:25:29.283550  7068 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.071s	user 0.043s	sys 0.057s
I20260709 12:25:29.299116  6940 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:29.299952  6940 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:29.304148  7068 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.195s	user 0.043s	sys 0.057s
W20260709 12:25:29.295290  7065 consensus_meta.cc:287] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.603s	user 0.007s	sys 0.008s
I20260709 12:25:29.319388  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:29.340164  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 79034.8us
I20260709 12:25:29.301252  6940 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:29.314116  7070 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:25:29.342628  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:29.345835  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.324255  7056 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:25:29.352849  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:29.363699  7023 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:29.364324  4720 connection.cc:256] RPC call timeout handler was delayed by -1.4280606170000283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
I20260709 12:25:29.364406  7023 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:29.364951  4720 connection.cc:256] RPC call timeout handler was delayed by -1.4271606170000268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
I20260709 12:25:29.360080  7041 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:25:29.365370  4720 connection.cc:256] RPC call timeout handler was delayed by -1.1895606170000974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
W20260709 12:25:29.365839  4720 connection.cc:256] RPC call timeout handler was delayed by -1.1724606170000698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
W20260709 12:25:29.366125  4720 connection.cc:256] RPC call timeout handler was delayed by -1.026660616999834s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19630000000000003s for a response
I20260709 12:25:29.367699  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.360080  6915 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.370122  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:29.378722  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.360579  7102 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:25:29.363744  6801 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.382684  4754 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:29.382421  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:29.389803  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:29.391539  6801 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:29.392310  6801 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:29.400260  4569 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:29.401368  4569 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:29.403254  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:29.405951  7127 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:25:29.412863  4754 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:29.410954  4569 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:29.414125  7108 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:29.414588  7108 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:29.415045  7108 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:29.416884  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:29.406369  7065 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:29.403888  7052 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:29.421347  4569 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:29.422209  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.411065  6897 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:29.424696  6773 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:29.428022  7131 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.068s	user 0.000s	sys 0.002s
I20260709 12:25:29.422207  7065 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:29.429584  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 59651us
I20260709 12:25:29.429931  7065 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:29.430759  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:29.432405  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:29.432713  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:29.402015  6915 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.434280  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.434990  4750 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:29.435133  6915 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.435926  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.436082  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.436329  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.436800  4755 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:29.437906  7023 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.440049  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.433213  4756 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:29.442581  4747 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:29.442451  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.443392  4751 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
W20260709 12:25:29.444216  7132 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.164s	user 0.002s	sys 0.000s
I20260709 12:25:29.444358  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:29.443181  6892 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:29.442498  7084 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:29.445658  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:29.446069  4753 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.420619  6788 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:29.394728  6801 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.424336  7052 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.456573  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.485061  4756 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:29.459116  6891 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:29.454244  6935 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:25:29.469527  4755 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:29.485312  4754 raft_consensus.cc:2410] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:25:29.461906  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:25:29.472146  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.472824  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:29.474144  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1839) took 1841 ms. Trace:
I20260709 12:25:29.426489  6897 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:25:29.504611  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.488229  4601 rpcz_store.cc:276] 0709 12:25:27.633169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.633336 (+   167us) service_pool.cc:224] Handling call
0709 12:25:27.667609 (+ 34273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.733258 (+1065649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.473797 (+740539us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003539d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:29.474053 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2448717184}
I20260709 12:25:29.489543  4753 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:29.487785  5347 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:29.520020  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:29.484293  7118 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.514s	user 0.003s	sys 0.000s
I20260709 12:25:29.451454  6915 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:29.521181  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:29.529459  4598 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:29.454200  7023 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:25:29.523483  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 90885.4us
I20260709 12:25:29.489544  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:29.539156  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 77448.5us
W20260709 12:25:29.540326  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 59908.8us
I20260709 12:25:29.476374  7089 raft_consensus.cc:3055] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:25:29.546614  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.546300  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:29.547106  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.467401  6877 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:29.513866  6897 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:29.543151  4645 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:29.489880  7052 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.522083  6773 raft_consensus.cc:2764] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:29.475654  6787 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.557206  4675 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:29.560293  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:29.562973  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:29.541036  6992 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:29.540850  6788 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.534958  7023 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:29.560389  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:29.566673  4673 raft_consensus.cc:2368] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6.
I20260709 12:25:29.545710  7089 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:29.569223  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:29.569777  4674 raft_consensus.cc:2410] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:29.570556  4599 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.572860  6891 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:29.573269  6891 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:29.573673  6891 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:29.577910  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:29.578586  4756 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.583752  6892 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:29.584442  6788 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.568902  7045 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:25:29.569465  7023 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:29.589721  6999 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:29.589958  6891 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.591229  7062 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:29.591653  7062 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.592017  7062 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:29.592130  6891 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.593799  6669 raft_consensus.cc:3055] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:29.594170  6669 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:29.594982  6669 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:29.596120  7080 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.595719  7089 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:29.596657  7080 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:29.598894  7080 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.600258  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 327) took 1007 ms. Trace:
W20260709 12:25:29.602551  4571 leader_election.cc:336] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.504s (SENT)
I20260709 12:25:29.603159  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:29.567373  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:29.607409  4605 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:29.612416  4756 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:29.563053  7092 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:29.549635  7056 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:25:29.621891  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.597456  7089 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:29.586567  6788 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.629477  4645 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:29.600792  4676 rpcz_store.cc:276] 0709 12:25:28.592464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.595584 (+  3120us) service_pool.cc:224] Handling call
0709 12:25:28.596062 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.596665 (+   603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.596696 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:28.597314 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:25:28.859198 (+261884us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:28.859251 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.859336 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:29.280615 (+421279us) spinlock_profiling.cc:243] Waited 9.76 ms on lock 0x7b180015fac0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:25:29.483425 (+202810us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:29.599855 (+116430us) raft_consensus.cc:1685] finished
0709 12:25:29.599890 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.600063 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1687,"raft.queue_time_us":4063,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":3909,"spinlock_wait_cycles":68800640,"thread_start_us":256256,"threads_started":1,"wal-append.queue_time_us":372499}
I20260709 12:25:29.583781  6787 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.641788  7141 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:29.630475  7077 raft_consensus.cc:2764] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:29.652141  6992 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:29.660609  4644 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:29.663335  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 104339us
W20260709 12:25:29.663875  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 117637us
I20260709 12:25:29.669646  7102 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:29.677603  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48209.8us
I20260709 12:25:29.614066  6959 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.685773  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.680229  4645 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:29.686465  4753 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:29.704927  4599 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:29.720068  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.721853  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:29.697912  6915 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:29.713299  6999 raft_consensus.cc:3055] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:29.670145  7056 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.051s
W20260709 12:25:29.745728  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 98565.9us
I20260709 12:25:29.713356  7092 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:29.744103  4754 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:29.720693  6935 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:29.753985  6935 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.754443  6935 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.756016  7062 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.716297  7076 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:29.757920  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.758237  7065 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.758941  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.752170  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:29.759467  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:29.753159  4753 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:29.760071  4755 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.670768  7108 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.719889  6788 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:29.764647  4756 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:29.742249  7141 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:29.768410  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.746990  6999 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:29.798978  4754 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:25:29.753259  7023 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:29.803673  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 51578.5us
I20260709 12:25:29.808696  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:29.809814  4756 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:29.764712  7092 raft_consensus.cc:697] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:29.812143  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 151643us
W20260709 12:25:29.814936  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 115484us
I20260709 12:25:29.763794  6891 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:25:29.722062  7102 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:29.815626  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.782109  6788 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:29.818657  4569 leader_election.cc:400] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6.
I20260709 12:25:29.822124  4569 leader_election.cc:403] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 12:25:29.819159  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.823544  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:29.825932  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 138427us
I20260709 12:25:29.827440  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:29.827858  5314 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:29.829856  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:29.813742  7136 consensus_queue.cc:1059] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:25:29.769238  7052 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:29.834726  4756 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:29.836483  6891 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6.
I20260709 12:25:29.839184  7143 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:29.839782  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:29.842805  7056 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:29.842904  4719 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:29.837038  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:29.843562  7023 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.863577  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:29.844803  7065 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.844445  6788 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.845064  6787 raft_consensus.cc:3055] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:29.873760  6787 raft_consensus.cc:740] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:29.812592  7141 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:29.874737  6787 consensus_queue.cc:260] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:29.845739  6959 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.846516  7062 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.819191  7077 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:29.857522  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:29.858311  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:29.859119  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:29.859910  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:29.863361  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:29.864770  4679 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.833618  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:29.866259  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:29.866366  7092 consensus_queue.cc:237] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.867558  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:29.867197  6999 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.932940  6787 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:29.933396  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 61584.9us
I20260709 12:25:29.933923  6999 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.935029  7084 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:29.869769  7102 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.870270  7052 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.938390  4599 raft_consensus.cc:2393] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 6: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:29.870777  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:29.939074  4673 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:29.832357  6915 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:29.932049  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:29.870545  6897 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:29.871750  6669 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.892669  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:29.924818  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:29.940212  4598 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:29.906462  4645 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:29.942525  4601 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 6: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:25:29.949529  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 124645us
W20260709 12:25:29.950716  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 44385.6us
I20260709 12:25:29.876592  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:29.952519  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:29.952819  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:29.948593  4678 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.953954  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:29.954492  4677 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:29.951781  4681 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:29.955775  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:29.957033  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:29.958177  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:29.958477  4675 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:29.964994  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:29.971102  4679 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:29.924391  7097 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:29.974790  4673 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:29.976087  4755 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:29.864022  7143 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:29.979053  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 44936.6us
I20260709 12:25:29.937944  6801 consensus_queue.cc:1059] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:25:29.985060  6897 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:29.988633  6935 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:29.958379  7077 raft_consensus.cc:697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:29.989820  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:29.993935  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:29.959017  7151 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:29.997413  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 327590us
I20260709 12:25:29.999976  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:30.009609  4568 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:30.012109  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 178856us
I20260709 12:25:30.005023  7151 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:30.019883  4571 leader_election.cc:336] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.638s (SENT)
I20260709 12:25:30.020485  4719 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:30.025180  4571 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:30.043448  4678 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:30.018404  7141 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:30.047326  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:30.048059  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 202458us
I20260709 12:25:30.048405  4678 raft_consensus.cc:2410] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:25:30.053124  4677 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:30.056013  6935 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:30.056775  7077 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:30.039873  6959 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:30.041054  6788 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.040047  7144 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:30.022984  7151 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:30.023000  7143 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:30.044736  7097 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:30.062487  4675 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:30.064222  6897 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.064937  4645 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:30.065701  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.068109  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 56658us
W20260709 12:25:30.069236  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 69481.5us
I20260709 12:25:30.077818  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.089489  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:30.090076  4596 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:30.065366  4681 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:30.078680  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:30.040884  7159 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:30.082489  4680 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:30.084072  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:30.091763  4756 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:30.100690  4601 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:30.102186  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:30.103207  4601 raft_consensus.cc:2368] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8.
I20260709 12:25:30.086531  6788 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:30.106266  4597 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:30.111264  6935 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:30.111915  6935 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:30.120528  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.154412  4571 leader_election.cc:336] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.515s (SENT)
I20260709 12:25:30.217895  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:30.219302  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 355) took 1416 ms. Trace:
I20260709 12:25:30.219861  4605 rpcz_store.cc:276] 0709 12:25:28.802609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.810780 (+  8171us) service_pool.cc:224] Handling call
0709 12:25:30.219074 (+1408294us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b540032bed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:30.219231 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093658496}
I20260709 12:25:30.093724  7143 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:30.120268  6801 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:30.121531  7097 raft_consensus.cc:697] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:30.092021  6897 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:30.323392  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 104482us
I20260709 12:25:30.326628  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:30.327394  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 240901us
I20260709 12:25:30.330528  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:30.331008  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:30.331815  4678 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:30.332881  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:30.349890  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:30.351935  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:30.353870  4719 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:30.354645  7159 consensus_queue.cc:1059] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:30.355569  7159 raft_consensus.cc:3055] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:30.355882  7159 raft_consensus.cc:740] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:30.356747  7159 consensus_queue.cc:260] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:30.359061  6897 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:30.364995  5343 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:30.365617  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.365674  7076 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:30.367201  7097 consensus_queue.cc:237] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:30.378108  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.369271  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:30.366029  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:30.379413  4679 raft_consensus.cc:2410] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:25:30.385020  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.394846  4597 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:25:30.395926  4597 pending_rounds.cc:85] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 5
I20260709 12:25:30.396445  4597 pending_rounds.cc:107] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:25:30.396742  4597 raft_consensus.cc:2889] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:30.408339  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 253952us
I20260709 12:25:30.414361  4596 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:25:30.415647  6995 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:30.426097  6995 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:30.430056  6992 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:30.430668  6992 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.432895  6992 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:30.434369  4681 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:25:30.433619  7080 raft_consensus.cc:3055] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:30.437920  7080 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:30.438829  7080 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.439854  7080 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:30.444808  4571 leader_election.cc:336] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.564s (SENT)
I20260709 12:25:30.445778  4681 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:30.411324  7151 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:30.451826  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 101155us
I20260709 12:25:30.453666  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:30.444231  7159 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:30.455945  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.457903  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:30.444065  6788 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:30.459622  6787 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:30.461434  6801 raft_consensus.cc:697] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:30.472360  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 120600us
I20260709 12:25:30.474375  6801 consensus_queue.cc:237] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:30.475031  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.475492  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:30.477236  4644 consensus_peers.cc:597] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:30.479430  4719 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:30.485487  7098 raft_consensus.cc:3055] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:30.495401  4681 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 12:25:30.496620  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.483873  6669 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.502497  4681 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:30.503168  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:30.506672  6669 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.507186  7070 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:30.512265  7070 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:30.512784  7070 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:30.513319  6995 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:30.504249  7173 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:30.513952  7165 log.cc:927] Time spent T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:30.515447  6669 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:30.524749  4676 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:30.502161  7157 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:25:30.529445  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:30.529837  4569 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:30.498176  7098 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:30.530815  4569 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:30.531391  5348 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:30.531239  7098 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.531723  4569 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:30.532564  7098 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:30.532924  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:30.527175  4676 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:30.504608  7151 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:30.535102  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:30.537695  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:30.517181  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:30.514305  7173 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:30.538650  7173 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:30.538920  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:30.539561  4603 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:30.535621  6787 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:30.542023  6787 raft_consensus.cc:697] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:30.542140  4596 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:30.536002  4605 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:30.536085  7143 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:30.543828  7143 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:30.544770  7143 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:30.536145  6959 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:25:30.558318  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 82887.8us
I20260709 12:25:30.558866  4645 leader_election.cc:400] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8.
I20260709 12:25:30.559279  4645 leader_election.cc:403] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:25:30.559947  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:30.562188  7045 consensus_queue.cc:1059] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:30.563006  6995 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:30.564103  6995 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.565125  7076 consensus_queue.cc:1059] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:25:30.567395  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:30.566987  7045 raft_consensus.cc:3055] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:30.584167  5348 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:25:30.585969  5348 pending_rounds.cc:85] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 5
I20260709 12:25:30.586267  5348 pending_rounds.cc:107] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:25:30.586546  5348 raft_consensus.cc:2889] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:30.585754  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 80252.7us
W20260709 12:25:30.575472  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.566076  7092 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8.
I20260709 12:25:30.605290  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:30.580536  7153 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.584368  6915 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.567642  7164 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:25:30.580996  7084 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.605512  7184 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:30.614231  4675 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:30.618690  4605 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:30.584349  6788 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:30.567852  7062 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:25:30.609752  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:30.580538  7045 raft_consensus.cc:740] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:30.599195  7146 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.646840  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.649420  4750 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:30.630254  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.599427  7152 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.637121  5349 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:30.641739  5348 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:30.651046  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:30.587796  7172 consensus_queue.cc:1059] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 12:25:30.644186  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.644398  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.644187  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:30.645502  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:30.646121  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.543119  6787 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:30.628379  7177 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.647421  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.588275  7167 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:25:30.633881  7089 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:25:30.647291  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 114421us
I20260709 12:25:30.665048  4747 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:30.650758  6788 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:30.680068  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.672688  7077 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.681107  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:30.682703  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.683586  4755 raft_consensus.cc:2393] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 7: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:30.684332  7089 raft_consensus.cc:697] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:30.666774  7045 consensus_queue.cc:260] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.679631  7092 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:25:30.649252  7184 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:30.669140  6940 raft_consensus.cc:3055] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:30.607452  4756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:30.641345  7097 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.675063  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.668224  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:30.690891  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:30.696600  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:30.693446  4603 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:30.639060  7141 raft_consensus.cc:3055] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:30.625548  7153 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.697258  4754 raft_consensus.cc:2376] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7. Re-sending same reply.
I20260709 12:25:30.637720  6915 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:30.700408  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.702159  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:30.706185  4755 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 7: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:30.686307  7045 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:30.709699  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.710687  4752 raft_consensus.cc:2410] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:30.711436  7176 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:30.711691  7173 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.714411  7173 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:30.691762  6959 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:30.721011  4749 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:30.657932  7146 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.723708  7146 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:30.726027  7185 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:30.726538  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:30.726809  7180 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:25:30.727217  4672 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:30.698704  7141 raft_consensus.cc:740] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:30.728451  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.728849  7141 consensus_queue.cc:260] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:30.730034  4756 raft_consensus.cc:606] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:30.730494  4756 raft_consensus.cc:614] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:30.731830  7183 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:30.732398  7183 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:30.738045  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.738107  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.739889  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:30.741019  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.744526  4752 raft_consensus.cc:2368] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6.
I20260709 12:25:30.744035  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:30.691521  7167 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.745225  5348 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:30.753239  4754 raft_consensus.cc:2393] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 4: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:25:30.754779  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.755592  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.694097  6940 raft_consensus.cc:740] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:30.698730  7076 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:30.758453  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:30.761430  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:30.763662  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.781738  4605 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:30.779403  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.780237  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:30.685354  7089 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.681530  7077 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.662374  7152 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:30.762903  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:30.792497  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:30.794703  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:30.671403  7084 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.797176  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:30.787707  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 115486us
I20260709 12:25:30.760967  6787 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:30.761201  7173 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:30.802873  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:30.803947  4752 raft_consensus.cc:2410] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:30.805830  5343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 372) took 1456 ms. Trace:
I20260709 12:25:30.806316  5343 rpcz_store.cc:276] 0709 12:25:29.349606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.349928 (+   322us) service_pool.cc:224] Handling call
0709 12:25:29.371457 (+ 21529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.402629 (+1031172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.805540 (+402911us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b5400893854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:30.805763 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2217150336}
I20260709 12:25:30.807283  7187 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:30.807672  7179 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:30.808128  7179 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.808488  7179 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:30.810648  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49298.2us
I20260709 12:25:30.811358  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:30.675951  7177 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:30.790321  7159 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:30.716393  6959 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:30.704170  7097 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:30.842440  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.801333  7172 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:30.843283  4752 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
W20260709 12:25:30.824565  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.845340  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:30.845513  4749 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:30.819082  4748 raft_consensus.cc:2376] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8. Re-sending same reply.
I20260709 12:25:30.855820  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:30.856623  4752 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:30.857082  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 122733us
I20260709 12:25:30.730561  7184 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:30.816164  6669 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.866354  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:30.819082  7176 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:30.870458  4750 raft_consensus.cc:2410] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:25:30.845072  6787 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:30.733435  7183 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:30.863150  7172 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:30.792863  7167 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:30.866204  7173 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:30.892923  4749 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
W20260709 12:25:30.899367  7178 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.133s	user 0.001s	sys 0.001s
I20260709 12:25:30.899837  7043 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.900287  7188 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:30.901302  7189 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:25:30.902114  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:30.905256  7141 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:30.905977  6787 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:30.874660  6669 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:30.915423  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.918435  4755 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:30.908823  7176 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:30.929229  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 324009us
I20260709 12:25:30.847674  6788 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:30.940142  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.917579  7196 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:30.941179  7196 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.943516  7196 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:30.944434  7097 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:30.945003  7199 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:25:30.946413  6915 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:30.947304  6935 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:30.950269  7181 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.268s	user 0.000s	sys 0.003s
W20260709 12:25:30.955013  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.932960  7176 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.973640  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:30.977782  4754 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:30.932029  7191 ts_tablet_manager.cc:933] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:30.973654  7176 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:30.919829  7043 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:30.886317  6940 consensus_queue.cc:260] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:31.048987  4752 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:31.018733  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:30.998335  4755 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:31.098613  5339 tablet_service.cc:1558] Processing DeleteTablet for tablet 1b7b0c6c811e455abe898cb4d21acad4 with delete_type TABLET_DATA_TOMBSTONED (TS dbe960c35c84451faf4e5b5c9233fa39 not found in new config with opid_index -1) from {username='slave'} at 127.0.0.1:37154
I20260709 12:25:31.010653  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:31.012127  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 341) took 1016 ms. Trace:
I20260709 12:25:31.079598  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:31.116189  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:31.037456  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:30.980208  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 49231.4us
I20260709 12:25:30.993789  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:31.001847  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:31.113096  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 345822us
I20260709 12:25:31.143586  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:31.151650  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:31.152861  4755 raft_consensus.cc:2410] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:30.997936  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:31.154268  4600 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 4: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:31.143231  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 367) took 1055 ms. Trace:
I20260709 12:25:31.161303  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:31.159251  4598 rpcz_store.cc:276] 0709 12:25:30.087561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.093517 (+  5956us) service_pool.cc:224] Handling call
0709 12:25:31.018520 (+925003us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b5800002798. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:25:31.143180 (+124660us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5966720}
I20260709 12:25:31.150382  4748 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:31.162372  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:31.163961  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:31.146688  4679 rpcz_store.cc:276] 0709 12:25:29.995324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.430512 (+435188us) service_pool.cc:224] Handling call
0709 12:25:30.430818 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.433932 (+  3114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.433960 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:30.433999 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:30.434038 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.011888 (+577850us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b54002dfd54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:31.011937 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.012066 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6768512}
I20260709 12:25:30.999126  4499 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52818 (request call id 301) took 5042 ms. Trace:
I20260709 12:25:31.161667  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:31.164559  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:31.257650  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.177691  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:31.177973  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:31.071542  7070 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:31.287554  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:31.288256  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:31.288622  7070 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:31.288995  4752 raft_consensus.cc:2393] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:31.171363  5348 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:31.289903  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:31.017647  7153 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:31.099306  7195 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 12:25:31.283679  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 56041.4us
I20260709 12:25:31.165208  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:30.994051  7177 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:31.016994  7159 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:31.169657  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:31.013435  7179 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:31.228338  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1490) took 1255 ms. Trace:
I20260709 12:25:31.071334  6773 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.017481  7084 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:31.316104  5312 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:25:31.228685  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 76054us
I20260709 12:25:31.238473  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:31.285809  4747 raft_consensus.cc:2410] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:31.173293  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:31.197364  7188 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:30.995869  6669 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:31.018501  7152 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:31.018090  7077 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:31.288499  4748 raft_consensus.cc:2376] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6. Re-sending same reply.
I20260709 12:25:31.284068  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:31.206099  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.206149  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 125074us
I20260709 12:25:31.316516  4752 raft_consensus.cc:2376] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3. Re-sending same reply.
W20260709 12:25:31.302132  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 153718us
I20260709 12:25:31.336606  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:31.295629  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 429310us
I20260709 12:25:31.302461  7194 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:31.319283  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:31.319576  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:31.173350  7043 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:31.315874  6940 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:31.313138  4678 rpcz_store.cc:276] 0709 12:25:29.972855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.436218 (+463363us) service_pool.cc:224] Handling call
0709 12:25:30.436508 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.436720 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:30.454012 (+ 17292us) log.cc:844] Serialized 14 byte log entry
0709 12:25:31.022213 (+568201us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b180007d0e0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:25:31.227613 (+205400us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:31.227693 (+    80us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:31.227769 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.228087 (+   318us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.228256 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17078,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":88311,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":3625,"spinlock_wait_cycles":5575936,"thread_start_us":127357,"threads_started":1}
I20260709 12:25:31.023600  7108 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:31.329015  4676 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:31.238955  4749 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:25:31.330313  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:31.330822  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.332125  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:31.332988  5347 raft_consensus.cc:2393] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:31.332953  4750 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:31.034917  7167 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:31.334823  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:31.335508  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:31.295674  7141 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.294178  7143 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:31.318354  4754 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:31.371649  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:31.340843  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.372418  4678 raft_consensus.cc:2393] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:31.341730  4680 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:31.373505  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:31.373699  5314 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:31.373764  4753 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:31.348850  4719 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:31.341876  4647 leader_election.cc:336] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.528s (SENT)
I20260709 12:25:31.165364  4499 rpcz_store.cc:276] 0709 12:25:25.956161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.956305 (+   144us) service_pool.cc:224] Handling call
0709 12:25:30.998883 (+5042578us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:27.210484 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:27.211247 (+   763us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:27.211297 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:25:27.211323 (+    26us) tablet.cc:662] Decoding operations
0709 12:25:27.212168 (+   845us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:27.212255 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:27.212289 (+    34us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:25:27.213073 (+   784us) tablet.cc:701] Row locks acquired
0709 12:25:27.213101 (+    28us) write_op.cc:260] PREPARE: finished
0709 12:25:27.836678 (+623577us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b2c00000448. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbea0a 00007effecdbe693 00007effecdbe631 00007effee4f1902 00007effee4f16dd 00007efff1eec243 00007efff1eebd58 00007efff1ef09f1 00007efff1ef09a9 00007efff1ef0939
0709 12:25:27.836740 (+    62us) write_op.cc:270] Start()
0709 12:25:27.836943 (+   203us) write_op.cc:276] Timestamp: P: 1783599927216416 usec, L: 0
0709 12:25:27.836969 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:25:28.082206 (+245237us) log.cc:844] Serialized 3481 byte log entry
0709 12:25:29.071557 (+989351us) spinlock_profiling.cc:243] Waited 6.55 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effeb3ac0ac 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b
0709 12:25:29.659769 (+588212us) spinlock_profiling.cc:243] Waited 7.25 ms on lock 0x7b18002f9080. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effee05e4bc 00007effeb3ac48b 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1
0709 12:25:29.664715 (+  4946us) op_driver.cc:464] REPLICATION: finished
0709 12:25:29.839360 (+174645us) write_op.cc:301] APPLY: starting
0709 12:25:29.839511 (+   151us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:29.839790 (+   279us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:29.839831 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:29.841148 (+  1317us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:29.855577 (+ 14429us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 12:25:29.855642 (+    65us) write_op.cc:312] APPLY: finished
0709 12:25:29.856086 (+   444us) log.cc:844] Serialized 65 byte log entry
0709 12:25:29.856291 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:29.856484 (+   193us) write_op.cc:454] Released schema lock
0709 12:25:29.856547 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:29.857069 (+   522us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:29.857095 (+    26us) write_op.cc:454] Released schema lock
0709 12:25:30.426694 (+569599us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b332a 00007effee438f14 00007effee438ec3 00007effee438e38 00007effee438df3 00007effee4380dc 00007effee461e16 00007effee449cbb 00007effee43439d 00007effee437d75
Metrics: {"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":148972,"apply.run_cpu_time_us":3793,"apply.run_wall_time_us":33150,"num_ops":8,"prepare.queue_time_us":276000,"prepare.run_cpu_time_us":118689,"prepare.run_wall_time_us":2449693,"raft.queue_time_us":184487,"raft.run_cpu_time_us":54576,"raft.run_wall_time_us":809722,"replication_time_us":1827487,"spinlock_wait_cycles":137401984,"thread_start_us":1671363,"threads_started":4,"wal-append.queue_time_us":1221431,"wal-append.run_cpu_time_us":18927,"wal-append.run_wall_time_us":1545044}]]}
I20260709 12:25:31.351621  4644 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:31.186791  4500 catalog_manager.cc:5697] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 4 to 6, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 6 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:31.361274  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:31.243425  6992 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 12:25:31.376413  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:31.363363  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:31.361272  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:31.368683  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:31.329133  7070 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:31.374094  4749 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:31.379297  5312 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:25:31.312280  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.386987  5348 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Advancing to term 3
I20260709 12:25:31.386700  4500 catalog_manager.cc:5697] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 1 to 3, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 3 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:31.388533  4500 catalog_manager.cc:5697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 1 to 3, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 3 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:31.390281  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:31.390793  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:31.378782  4647 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:31.392227  5349 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:25:31.394167  4571 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:31.395548  4600 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:31.395983  4752 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:25:31.398581  4754 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:25:31.399041  4754 pending_rounds.cc:85] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 3
I20260709 12:25:31.399302  4754 pending_rounds.cc:107] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:25:31.399570  4754 raft_consensus.cc:2889] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:31.399933  4754 pending_rounds.cc:107] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:25:31.400170  4754 raft_consensus.cc:2889] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:31.393098  4500 catalog_manager.cc:5697] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 2 to 4, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 4 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:31.400996  5348 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:31.403697  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:31.404564  4601 raft_consensus.cc:2410] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:25:31.405154  4571 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:31.315122  6773 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:31.417626  4756 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:31.418598  4749 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:31.417150  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:31.426275  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:31.423368  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 74591.7us
I20260709 12:25:31.423992  4748 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:25:31.427914  4748 pending_rounds.cc:85] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 2
I20260709 12:25:31.428175  4748 pending_rounds.cc:107] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:25:31.428413  4748 raft_consensus.cc:2889] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:31.424386  4747 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:31.369915  7185 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:31.428459  4749 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:31.395104  7167 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:31.390811  7080 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:31.431876  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:31.451439  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:31.393235  7070 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.396718  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:25:31.473434  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 131594us
W20260709 12:25:31.474156  4647 leader_election.cc:336] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.540s (SENT)
I20260709 12:25:31.474294  4755 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:31.474542  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:31.478068  4750 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:31.453730  7142 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:30.997936  7172 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:31.468632  7205 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:25:31.404132  7041 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:31.478229  4674 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:31.481366  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:31.455868  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:31.403911  7108 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:31.467490  7196 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:31.372274  7184 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:25:31.719238  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:31.482033  4748 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:31.482147  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:31.482700  4605 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:31.418866  7077 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:31.432958  6999 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.431753  6995 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:31.413883  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:31.390209  7141 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:31.690696  7201 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:31.586262  7193 consensus_queue.cc:1059] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.190s
I20260709 12:25:31.540136  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:31.454820  6801 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:31.692564  4604 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:31.687570  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.694424  7159 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.587236  7191 tablet_copy_client.cc:323] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.194:42599
I20260709 12:25:31.694976  7097 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.696626  6773 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:31.476086  7080 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.698840  7151 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:31.540460  7173 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:25:31.850122  7159 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:31.479702  7070 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:31.855762  7159 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:31.704105  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 298988us
I20260709 12:25:31.704202  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:31.857558  4750 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:31.857609  4571 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:31.431993  7143 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:31.863471  7143 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:31.511181  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:31.864969  4596 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 3: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:31.707868  6788 raft_consensus.cc:3055] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:31.710243  4753 raft_consensus.cc:3055] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:31.871093  4753 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:31.711354  4749 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:31.479395  7177 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.118s
I20260709 12:25:31.687685  4602 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:31.686720  4599 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:25:31.713796  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 373039us
I20260709 12:25:31.690223  7142 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:31.721184  4751 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:31.721312  4597 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:31.720114  4600 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
W20260709 12:25:31.727399  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 303003us
I20260709 12:25:31.744729  4755 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:25:31.883491  4755 pending_rounds.cc:85] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 2
I20260709 12:25:31.883832  4755 pending_rounds.cc:107] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:25:31.884107  4755 raft_consensus.cc:2889] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:31.746730  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:31.760074  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 285935us
W20260709 12:25:31.884965  4647 leader_election.cc:336] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.550s (SENT)
W20260709 12:25:31.784471  4721 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.3.89.254:36317, user_credentials={real_user=slave}} blocked reactor thread for 62317.8us
I20260709 12:25:31.752640  6992 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.718546  7196 raft_consensus.cc:697] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:31.843979  7043 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.844022  7080 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:31.850222  4485 catalog_manager.cc:5008] TS dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): delete failed for tablet 1b7b0c6c811e455abe898cb4d21acad4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1b7b0c6c811e455abe898cb4d21acad4 already in progress: copying tablet
W20260709 12:25:31.955528  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 58814us
I20260709 12:25:31.849840  7097 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:31.827811  7152 consensus_queue.cc:1059] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:25:31.851766  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:31.866194  4750 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:31.873334  4571 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:31.835394  7141 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:31.898675  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:31.897230  4601 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:31.747591  6999 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:31.949723  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.900170  4753 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:31.950711  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.945007  4569 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:31.870853  6788 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:31.881795  7159 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.743533  7077 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:31.882046  7179 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:32.016010  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 59741.6us
W20260709 12:25:31.957832  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.882535  7183 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:31.837216  7201 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:32.135779  4601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:25:32.136986  4755 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:31.956951  4600 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:31.942346  6992 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:31.957558  7043 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:32.077553  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 374) took 1068 ms. Trace:
I20260709 12:25:32.143873  4753 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:31.958062  7151 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.148237  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:32.136631  4601 raft_consensus.cc:606] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
W20260709 12:25:32.048243  4485 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 198196us
I20260709 12:25:32.158921  4750 rpcz_store.cc:276] 0709 12:25:31.009025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.861490 (+852465us) service_pool.cc:224] Handling call
0709 12:25:31.865967 (+  4477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.077244 (+211277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.077483 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:32.178077  7179 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.069541  7215 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:31.957396  6773 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:32.218727  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 345436us
I20260709 12:25:32.219856  4571 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:32.204849  4753 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:25:32.233307  4753 pending_rounds.cc:85] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 1
I20260709 12:25:32.068312  7159 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.255410  4753 pending_rounds.cc:107] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 2.2
W20260709 12:25:32.184067  7176 consensus_meta.cc:287] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.929s	user 0.010s	sys 0.002s
W20260709 12:25:32.218632  7194 consensus_meta.cc:287] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.855s	user 0.016s	sys 0.000s
I20260709 12:25:32.162827  7097 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:32.275033  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 330079us
I20260709 12:25:32.274317  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:32.281149  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 61416.3us
W20260709 12:25:32.281821  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 396880us
I20260709 12:25:32.281725  4569 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:32.282375  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.282668  4571 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:32.218598  7152 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:32.272020  4753 raft_consensus.cc:2889] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:32.230343  7218 raft_consensus.cc:2764] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:25:32.291325  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1876) took 1487 ms. Trace:
W20260709 12:25:32.295383  4647 leader_election.cc:336] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.577s (SENT)
I20260709 12:25:32.295727  4753 rpcz_store.cc:276] 0709 12:25:30.804037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.709703 (+905666us) service_pool.cc:224] Handling call
0709 12:25:31.710072 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.290994 (+580922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.291256 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:32.294152  6959 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:32.255227  7183 raft_consensus.cc:697] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:32.301914  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:32.301704  4602 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:32.298902  5349 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:32.203691  7151 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.355926  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1480) took 1740 ms. Trace:
I20260709 12:25:32.312734  5342 raft_consensus.cc:2410] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:25:32.334246  7041 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:32.344352  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:32.348294  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:32.231698  6788 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:32.350498  7199 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:25:32.351366  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1865) took 1564 ms. Trace:
I20260709 12:25:32.288447  7045 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.284502  7194 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:32.353672  7184 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:32.256392  7215 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.377540  4675 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:32.217666  7142 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:32.302568  6787 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:32.297066  7219 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:32.256609  7179 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:32.377027  6995 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.382108  4756 rpcz_store.cc:276] 0709 12:25:30.786467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.425290 (+638823us) service_pool.cc:224] Handling call
0709 12:25:31.451270 (+ 25980us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.481540 (+ 30270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.481594 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.510584 (+ 28990us) log.cc:844] Serialized 26 byte log entry
0709 12:25:31.943261 (+432677us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:31.943305 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.943378 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.351068 (+407690us) raft_consensus.cc:1685] finished
0709 12:25:32.351098 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.351284 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3206,"raft.queue_time_us":3375,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":33289,"thread_start_us":253145,"threads_started":1,"wal-append.queue_time_us":518264}
I20260709 12:25:32.389463  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:32.390027  7041 raft_consensus.cc:3055] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:32.327958  7108 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:32.356383  4604 rpcz_store.cc:276] 0709 12:25:30.615641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.615827 (+   186us) service_pool.cc:224] Handling call
0709 12:25:30.616340 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.837016 (+1220676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.355670 (+518654us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b5400033754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:32.355863 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2115233024}
I20260709 12:25:32.392735  7097 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:32.284279  7176 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.419502  4598 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:25:32.255872  6773 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.368772  7141 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.268370  7196 consensus_queue.cc:237] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:32.437592  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48201.8us
I20260709 12:25:32.447762  4599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51130
W20260709 12:25:32.453234  7212 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:25:32.454335  6773 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:32.430577  7152 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:32.465459  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 182380us
W20260709 12:25:32.468498  4647 leader_election.cc:336] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.605s (SENT)
I20260709 12:25:32.430214  7219 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:32.472167  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 189587us
I20260709 12:25:32.435726  7041 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:25:32.477056  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.418402  7045 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:32.481652  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:32.440174  7204 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:32.446342  7187 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.437458  7108 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:32.455011  7208 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:25:32.492064  4571 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:32.504158  4571 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:32.506261  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:32.477780  7221 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.008s	sys 0.008s
I20260709 12:25:32.437659  6995 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:32.500159  7224 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
W20260709 12:25:32.497211  7213 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.440s	user 0.000s	sys 0.002s
I20260709 12:25:32.475992  6999 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:32.473923  7219 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:32.518175  7221 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.009s	sys 0.008s
W20260709 12:25:32.517129  4647 leader_election.cc:336] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.649s (SENT)
I20260709 12:25:32.518512  7043 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.505777  6992 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.521808  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:32.550096  4756 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 1.
W20260709 12:25:32.523047  7070 consensus_meta.cc:287] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.667s	user 0.009s	sys 0.000s
I20260709 12:25:32.502120  7077 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:32.523720  7220 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 12:25:32.523607  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:32.524950  7215 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.559137  7177 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.561981  7177 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.559638  7070 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:32.567842  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1874) took 1764 ms. Trace:
I20260709 12:25:32.570389  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1863) took 1784 ms. Trace:
I20260709 12:25:32.525790  7108 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.455063  7141 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:32.539191  4597 consensus_meta.cc:287] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.646s	user 0.011s	sys 0.000s
I20260709 12:25:32.519126  7201 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.471305  7183 consensus_queue.cc:237] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:32.523762  7230 log.cc:927] Time spent T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:25:32.558719  4750 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:32.552772  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 1871) took 1072 ms. Trace:
I20260709 12:25:32.551757  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:32.580363  4598 rpcz_store.cc:276] 0709 12:25:31.480337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.480514 (+   177us) service_pool.cc:224] Handling call
0709 12:25:32.552509 (+1071995us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b54002760d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:32.552670 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1337636224}
I20260709 12:25:32.570138  4749 rpcz_store.cc:276] 0709 12:25:30.803469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.453258 (+649789us) service_pool.cc:224] Handling call
0709 12:25:31.710923 (+257665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.872036 (+161113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.567631 (+695595us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54000bf4d0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:32.567758 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":589937408}
I20260709 12:25:32.573940  7043 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.568127  5314 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:32.549544  7159 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:32.573520  7228 log.cc:927] Time spent T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:25:32.560061  7184 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.587154  7070 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.533046  7152 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:25:32.570880  4754 rpcz_store.cc:276] 0709 12:25:30.785991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.414386 (+628395us) service_pool.cc:224] Handling call
0709 12:25:31.414755 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.424082 (+  9327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.424122 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.462693 (+ 38571us) log.cc:844] Serialized 26 byte log entry
0709 12:25:31.837977 (+375284us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:31.838015 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.838077 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.570196 (+732119us) raft_consensus.cc:1685] finished
0709 12:25:32.570216 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.570336 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8881,"raft.queue_time_us":9123,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":45988,"thread_start_us":374879,"threads_started":1,"wal-append.queue_time_us":687096}
I20260709 12:25:32.508515  7187 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.531386  6897 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.529709  7235 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:32.578372  4597 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:32.568095  7097 raft_consensus.cc:1064] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:32.549098  7243 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.565542  7153 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:32.568393  7179 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.592928  5344 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:32.593920  5314 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:32.612577  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1880) took 1118 ms. Trace:
I20260709 12:25:32.577122  7215 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:32.594852  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:32.625298  4749 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:32.625815  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1881) took 1131 ms. Trace:
I20260709 12:25:32.598290  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 1874) took 1116 ms. Trace:
I20260709 12:25:32.599311  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:32.601444  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 172721us
I20260709 12:25:32.602510  7244 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.602392  7235 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:32.602890  7097 raft_consensus.cc:1068] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Unable to promote non-voter dbe960c35c84451faf4e5b5c9233fa39: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:25:32.599063  7218 raft_consensus.cc:2764] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:25:32.582195  7043 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:32.604013  7080 consensus_meta.cc:287] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.000s
I20260709 12:25:32.525821  7041 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:32.614151  7249 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:32.614880  6995 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:32.627935  7179 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.629472  7080 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:32.630121  7179 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:32.632119  7080 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:32.632207  7179 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:32.632716  6801 raft_consensus.cc:3055] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:32.633067  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 372) took 1684 ms. Trace:
I20260709 12:25:32.633224  6801 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:32.633479  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:32.634207  4754 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:32.574059  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 129133us
I20260709 12:25:32.635109  7244 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.637292  7244 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.596343  7151 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.617985  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1877) took 1161 ms. Trace:
I20260709 12:25:32.613050  4603 rpcz_store.cc:276] 0709 12:25:31.494423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.783809 (+289386us) service_pool.cc:224] Handling call
0709 12:25:31.857012 (+ 73203us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.871772 (+ 14760us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.871815 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.876773 (+  4958us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.230499 (+353726us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:32.230552 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.230629 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.612326 (+381697us) raft_consensus.cc:1685] finished
0709 12:25:32.612354 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.612498 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":149824,"raft.queue_time_us":15041,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":13051,"thread_start_us":269265,"threads_started":1,"wal-append.queue_time_us":428001}
I20260709 12:25:32.617200  7045 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:32.640769  4755 rpcz_store.cc:276] 0709 12:25:31.456909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.949516 (+492607us) service_pool.cc:224] Handling call
0709 12:25:31.949855 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.137543 (+187688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.617728 (+480185us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400264654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:32.617910 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429244672}
W20260709 12:25:32.617693  7167 consensus_meta.cc:287] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.144s	user 0.010s	sys 0.002s
I20260709 12:25:32.620478  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1862) took 1834 ms. Trace:
I20260709 12:25:32.618675  7242 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:32.622670  7187 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.601922  6897 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.592890  7177 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:32.626456  4597 rpcz_store.cc:276] 0709 12:25:31.481804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.481909 (+   105us) service_pool.cc:224] Handling call
0709 12:25:32.598209 (+1116300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:32.627622  4645 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:32.670886  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:32.673413  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 45894.9us
I20260709 12:25:32.633479  4748 rpcz_store.cc:276] 0709 12:25:30.949049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.751610 (+802561us) service_pool.cc:224] Handling call
0709 12:25:31.751895 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.152851 (+400956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.152897 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:32.152956 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:32.153018 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.632860 (+479842us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b5400183654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:32.632903 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.633014 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":921498496}
I20260709 12:25:32.596623  7108 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:32.619163  7243 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:32.617420  7241 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
W20260709 12:25:32.621399  6940 consensus_meta.cc:287] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.251s	user 0.013s	sys 0.000s
I20260709 12:25:32.597738  7202 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:32.626235  4596 rpcz_store.cc:276] 0709 12:25:31.494700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.883487 (+388787us) service_pool.cc:224] Handling call
0709 12:25:31.883929 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.078301 (+194372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.078347 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.152045 (+ 73698us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.612643 (+460598us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:32.612681 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.612789 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.625625 (+ 12836us) raft_consensus.cc:1685] finished
0709 12:25:32.625644 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.625764 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":234511,"raft.queue_time_us":240865,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":58951,"thread_start_us":332829,"threads_started":1,"wal-append.queue_time_us":421456}
I20260709 12:25:32.640128  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:32.646600  4754 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:32.670087  5346 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:25:32.675046  4645 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:32.641772  4752 rpcz_store.cc:276] 0709 12:25:30.785750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.397052 (+611302us) service_pool.cc:224] Handling call
0709 12:25:31.397412 (+   360us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:31.454098 (+ 56686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.454142 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:31.550284 (+ 96142us) log.cc:844] Serialized 52 byte log entry
0709 12:25:32.157759 (+607475us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:32.481108 (+323349us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.501704 (+ 20596us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.501897 (+   193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.501973 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.620211 (+118238us) raft_consensus.cc:1685] finished
0709 12:25:32.620237 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.620408 (+   171us) inbound_call.cc:177] Queueing success response
0709 12:25:32.623808 (+  3400us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b180030c160. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
Metrics: {"mutex_wait_us":28897,"prepare.queue_time_us":541513,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":58,"raft.queue_time_us":78372,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":11842,"spinlock_wait_cycles":9542912,"thread_start_us":576038,"threads_started":2,"wal-append.queue_time_us":626007}
I20260709 12:25:32.594408  7184 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:32.702600  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 185932us
I20260709 12:25:32.707468  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:32.720831  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 80753.1us
I20260709 12:25:32.696460  7249 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:32.725618  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:32.680199  7167 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:32.695876  7232 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.726540  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:32.643429  7242 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:32.727957  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:32.702167  7202 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.733714  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:32.705405  6787 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:32.708626  7045 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:32.739342  4674 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:32.740078  4681 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:32.718676  6992 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.671811  7097 consensus_queue.cc:1059] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
W20260709 12:25:32.743160  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 163237us
W20260709 12:25:32.743669  4719 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.196:39067 timed out after 1.532s (SENT) [suppressed 42 similar messages]
I20260709 12:25:32.744405  5348 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:32.744618  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:32.744009  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 239939us
I20260709 12:25:32.745163  4680 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:32.750517  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:32.751071  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:32.751868  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.752064  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:32.704188  7152 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:25:32.703913  7236 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:25:32.705243  7151 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:25:32.725636  7177 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:32.733222  7246 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:32.753347  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:32.762020  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:32.766767  4569 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:32.769110  4599 raft_consensus.cc:606] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:32.769507  4599 raft_consensus.cc:614] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:32.743510  7202 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:32.773020  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:32.743803  7242 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:32.703438  6801 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:32.744588  7045 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.778332  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:32.788393  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 82096.7us
I20260709 12:25:32.754988  7167 raft_consensus.cc:2764] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:32.810590  4645 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:32.801481  7237 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 12:25:32.765321  7142 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:32.765451  7196 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.764644  7183 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:32.822090  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:32.822546  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:32.807976  4569 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:32.812216  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:32.823242  4598 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
W20260709 12:25:32.823350  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 115971us
I20260709 12:25:32.823505  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:32.824208  4674 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:32.824438  4604 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:32.813205  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:32.744683  6995 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:32.756067  6992 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.822625  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:32.823684  4597 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 2.
I20260709 12:25:32.772444  7144 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:32.768167  7246 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.802168  7177 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:32.742624  7232 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.825089  4596 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:32.834538  4719 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:32.826275  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:32.826609  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:32.826825  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:32.835479  4674 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:32.808548  7250 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:32.806895  6787 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:32.830278  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:32.805967  7153 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:32.812345  7141 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:32.827783  7236 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:32.813932  7045 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:32.837791  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 59521.9us
I20260709 12:25:32.820196  6801 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:32.845458  4672 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:25:32.845469  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:32.846851  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:32.840276  7159 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:32.862804  7159 raft_consensus.cc:1064] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
W20260709 12:25:32.863195  7159 raft_consensus.cc:1068] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Unable to promote non-voter dbe960c35c84451faf4e5b5c9233fa39: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:25:32.847733  7247 log.cc:927] Time spent T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:25:32.847681  7235 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.848477  7043 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.835793  7246 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:32.850817  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:32.866936  4752 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:32.828519  4691 tablet_copy_service.cc:140] P 3a5aeedd11fe49938079fe86cb01232c: Received BeginTabletCopySession request for tablet 1b7b0c6c811e455abe898cb4d21acad4 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:48198)
I20260709 12:25:32.851267  7251 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:32.852341  7258 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
W20260709 12:25:32.852696  4751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1851) took 2116 ms (client timeout 2159 ms). Trace:
I20260709 12:25:32.853012  7254 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.853507  4747 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:25:32.853931  6959 consensus_meta.cc:287] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.001s
W20260709 12:25:32.818601  6999 consensus_queue.cc:1175] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "dbe960c35c84451faf4e5b5c9233fa39" responder_term: 6 status { last_received { term: 3 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: [5.6-5.6]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:32.885300  6959 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:32.855968  7184 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:32.846359  7236 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.858341  7243 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.858577  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:32.886155  7236 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:32.870265  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 62388.9us
W20260709 12:25:32.887809  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.873963  4672 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:32.877324  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:32.881053  4751 rpcz_store.cc:269] 0709 12:25:30.736642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.161031 (+424389us) service_pool.cc:224] Handling call
0709 12:25:32.852502 (+1691471us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b5400264154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:32.852636 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413611392}
I20260709 12:25:32.829026  7097 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:25:32.883980  5314 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:32.829546  7196 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.830727  7179 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:25:32.892637  6995 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:32.894362  7196 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.833475  7183 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:32.891528  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 81006.2us
I20260709 12:25:32.891835  4691 tablet_copy_service.cc:161] P 3a5aeedd11fe49938079fe86cb01232c: Beginning new tablet copy session on tablet 1b7b0c6c811e455abe898cb4d21acad4 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:48198: session id = dbe960c35c84451faf4e5b5c9233fa39-1b7b0c6c811e455abe898cb4d21acad4
I20260709 12:25:32.890792  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:32.846621  7153 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:32.903513  7183 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:32.904532  7153 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:32.877111  6787 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:32.906466  6787 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.910102  7097 raft_consensus.cc:3055] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:32.910606  7097 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:32.911542  7097 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.912833  7097 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:32.863404  5342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53808 (request call id 397) took 1536 ms (client timeout 1532 ms). Trace:
W20260709 12:25:32.913810  5342 rpcz_store.cc:269] 0709 12:25:31.327047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.328971 (+  1924us) service_pool.cc:224] Handling call
0709 12:25:32.863190 (+1534219us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b54007116d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:32.863333 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009799552}
I20260709 12:25:32.897040  7254 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:32.918800  7232 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:32.900548  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:32.921006  6669 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:32.921526  6669 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.921977  6669 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:32.925097  7204 raft_consensus.cc:3055] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:32.925544  7204 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:32.926564  7204 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.928099  7204 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:32.899691  7080 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:32.930751  7080 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:32.931697  7080 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:32.933920  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 87136.1us
I20260709 12:25:32.934746  4571 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:32.883682  7193 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:32.935259  6992 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:32.935595  7193 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.936022  7193 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:32.939670  5348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 383) took 1094 ms. Trace:
I20260709 12:25:32.949865  4674 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:32.940253  5348 rpcz_store.cc:276] 0709 12:25:31.845436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.845553 (+   117us) service_pool.cc:224] Handling call
0709 12:25:31.845923 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.749409 (+903486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.939361 (+189952us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b5400712354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:32.939590 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2065761664}
W20260709 12:25:32.952884  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 62192.6us
I20260709 12:25:32.953918  7173 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:32.957008  6788 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:32.958022  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 99538.3us
I20260709 12:25:32.958819  7176 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:32.959859  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:32.960987  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:32.961129  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:32.964026  7235 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:32.882681  7177 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 12:25:32.965545  7173 raft_consensus.cc:3055] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:32.965960  7173 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:32.963258  7176 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:32.898831  7167 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:32.935989  7194 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:32.974541  6940 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:32.967662  7176 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:32.975104  7177 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:32.979362  7215 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:32.918800  7254 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:32.985419  4571 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:33.026229  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:33.034122  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 135568us
W20260709 12:25:33.034765  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 146424us
I20260709 12:25:33.037207  4597 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:33.037371  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:33.051621  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:32.992184  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:33.052224  4672 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:33.053463  4647 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:33.055584  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:32.990823  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1873) took 2187 ms. Trace:
W20260709 12:25:33.038477  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 227866us
I20260709 12:25:33.057592  4605 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:33.058320  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:33.058897  5349 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:32.983516  7258 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:33.045411  4598 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:32.963569  7183 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:33.046166  4603 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:33.046559  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:33.064293  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.064724  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.050580  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.065284  4675 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:25:33.059628  4674 raft_consensus.cc:3055] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:33.057255  4747 rpcz_store.cc:276] 0709 12:25:30.803140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.434295 (+631155us) service_pool.cc:224] Handling call
0709 12:25:32.853091 (+1418796us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.884815 (+ 31724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.990576 (+105761us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400264150. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:32.990746 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3168790912}
I20260709 12:25:32.968113  7077 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:33.072476  4569 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:33.000062  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:32.966945  7173 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:33.061297  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:33.074340  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 88994.1us
I20260709 12:25:33.066071  4674 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:33.075513  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:33.075809  4674 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:33.077060  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:33.080318  7235 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:33.081023  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:33.060420  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.082937  4645 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:33.084067  4645 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:33.070528  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:33.070705  5312 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:33.073982  4596 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:33.056589  4691 tablet_copy_source_session.cc:215] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:33.088510  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:33.054313  7250 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.099766  4751 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:33.101219  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:33.093267  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:33.102227  4753 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:33.053689  7084 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:33.056721  7249 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:33.096640  4755 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:33.098047  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:33.103626  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:33.104036  4748 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:33.061618  7108 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:32.981477  7202 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:33.107329  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:33.112766  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:33.108435  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:33.113283  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:33.115993  4601 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:33.108819  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:33.131891  4753 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:33.110417  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:33.106490  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:33.118001  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.122386  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.122771  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.136689  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:33.091889  7215 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:33.094719  7183 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:33.137624  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.141099  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:33.142710  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:33.054313  7263 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:25:33.158933  4676 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:25:33.159123  4598 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:33.159336  5348 raft_consensus.cc:2393] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:33.105171  6915 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:33.139698  7219 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.566s	user 0.000s	sys 0.011s
I20260709 12:25:33.169111  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:33.166316  4568 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:33.177300  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 184876us
I20260709 12:25:33.170907  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:33.183214  6959 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:33.183646  6959 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.185163  4568 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:33.185480  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:33.117275  7264 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:33.105249  7275 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:33.115067  6787 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 12:25:33.184005  6959 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:33.150254  7215 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.111392  7270 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:33.209450  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.210453  4679 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:33.158370  7251 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:33.187413  4747 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 1.
W20260709 12:25:33.196684  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 123087us
W20260709 12:25:33.206429  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63797.1us
I20260709 12:25:33.163583  7151 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:33.133791  6992 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 12:25:33.158545  7077 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:33.183655  7267 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:25:33.159622  7177 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:33.160604  7183 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:33.109421  7043 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 12:25:33.111817  7084 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.111526  7250 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.229761  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:33.227501  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:33.206548  7218 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:33.225955  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.226822  4571 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:33.225994  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:33.254463  4679 raft_consensus.cc:2410] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:25:33.255013  7196 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:33.216368  7246 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.255462  7196 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.255834  7196 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:33.255730  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:33.256548  4679 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:33.257072  7246 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:33.257989  4680 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:33.258970  7141 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:33.177526  7243 consensus_queue.cc:1059] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:25:33.251134  7177 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.260011  7243 raft_consensus.cc:3055] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:33.260383  7243 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:33.262063  7177 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:33.228626  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.263443  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.212504  7249 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:33.209903  7215 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:33.196309  6915 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.212047  7275 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:33.211975  7270 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.216361  7251 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:33.274814  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:33.291072  4747 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:33.200186  7153 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.205914  7152 consensus_queue.cc:1059] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:25:33.196175  7219 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:33.299273  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 95302.6us
I20260709 12:25:33.294430  4679 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:33.295753  7281 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.224186  7077 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.312754  4647 leader_election.cc:400] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6.
I20260709 12:25:33.313182  4647 leader_election.cc:403] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:33.314924  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.280107  7280 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:33.311290  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 84739.1us
I20260709 12:25:33.324209  4596 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:33.296685  4604 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:33.224556  7183 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:33.291577  7270 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:33.326607  4681 raft_consensus.cc:2410] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:25:33.298461  6773 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:33.294569  7251 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:33.329934  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:33.303885  7176 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:33.332717  4756 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:33.335532  7248 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.335637  7223 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:33.336311  7223 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.336190  7248 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.336845  7223 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:33.340587  7278 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:25:33.340796  7245 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.163s
I20260709 12:25:33.341756  7256 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6.
W20260709 12:25:33.342518  7273 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:25:33.342903  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:25:33.363163  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.330736  6773 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:33.372092  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:25:33.374203  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.455916  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.333986  7152 consensus_queue.cc:1243] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:33.333173  7282 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.469913  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.338624  7248 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:33.325786  7242 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:33.350973  6801 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:33.319010  7281 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.373984  6773 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:33.368292  7202 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:33.586400  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 476025us
I20260709 12:25:33.371430  7151 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:33.349519  7236 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.587991  7282 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:33.583084  7241 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 12:25:33.660640  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.707243  7282 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:33.577986  7152 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:33.346491  7280 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.705017  7084 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:33.677579  4747 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
W20260709 12:25:33.686769  7279 log.cc:927] Time spent T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.364s	user 0.001s	sys 0.000s
W20260709 12:25:33.726747  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 354677us
W20260709 12:25:33.730743  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 127489us
I20260709 12:25:33.746490  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:33.350852  7193 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:33.615906  7045 consensus_meta.cc:287] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.732s	user 0.008s	sys 0.002s
I20260709 12:25:33.737030  6897 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:33.574633  7196 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:33.728547  5342 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:33.367620  7275 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:33.734508  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.748682  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.782528  7236 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:33.785143  7153 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:33.789731  7077 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:33.794519  5349 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEARNER]: Advancing to term 4
I20260709 12:25:33.806598  7281 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:33.809597  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:33.810911  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:33.811535  4748 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:33.757772  7202 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:33.813548  4748 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:33.814144  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.831205  4673 raft_consensus.cc:2410] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:25:33.832732  4748 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:33.811159  7183 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:33.822268  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 66412.8us
I20260709 12:25:33.835208  4647 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:33.747249  7245 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:33.767434  6897 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:33.820214  6669 consensus_meta.cc:287] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.898s	user 0.012s	sys 0.000s
W20260709 12:25:33.817147  7204 consensus_meta.cc:287] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.889s	user 0.004s	sys 0.004s
I20260709 12:25:33.760413  7045 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:33.838599  6940 consensus_meta.cc:287] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.797s	user 0.008s	sys 0.002s
I20260709 12:25:33.866025  6940 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:33.834159  7263 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:33.866981  7183 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:33.867357  7263 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:33.851286  6959 consensus_meta.cc:287] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.632s	user 0.007s	sys 0.003s
I20260709 12:25:33.874125  6959 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:33.876129  4674 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:33.876520  6959 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:33.853271  7245 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:33.857818  6897 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:33.857860  6669 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:33.828365  7202 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:33.858903  7280 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:33.861716  7246 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:33.862026  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.827553  7242 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:33.864148  7223 consensus_meta.cc:287] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.527s	user 0.004s	sys 0.006s
I20260709 12:25:33.881135  4647 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:33.838469  6788 consensus_meta.cc:287] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.878s	user 0.007s	sys 0.003s
I20260709 12:25:33.883805  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:33.890705  4674 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:33.892514  5349 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
W20260709 12:25:33.886884  7215 consensus_meta.cc:287] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.596s	user 0.008s	sys 0.002s
I20260709 12:25:33.902494  5349 pending_rounds.cc:85] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 2
I20260709 12:25:33.926381  5349 pending_rounds.cc:107] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:25:33.893245  7202 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:33.898214  7264 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.007s	sys 0.037s
I20260709 12:25:33.926685  5349 raft_consensus.cc:2889] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:33.921103  6669 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:33.921893  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.929963  4675 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 12:25:33.930399  4675 pending_rounds.cc:85] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 4
W20260709 12:25:33.899663  7270 consensus_meta.cc:287] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.000s
I20260709 12:25:33.930811  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:33.938088  6669 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:33.899166  6940 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.940656  6669 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:33.940740  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:33.941603  4673 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:33.930657  4675 pending_rounds.cc:107] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.5
W20260709 12:25:33.982556  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 101497us
I20260709 12:25:33.984025  4647 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:33.978372  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:33.932919  7264 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.007s	sys 0.037s
I20260709 12:25:33.990708  7264 raft_consensus.cc:697] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:33.993351  4675 raft_consensus.cc:2917] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Aborting config change with OpId 3.5: Aborted: Op aborted by new leader
I20260709 12:25:33.841457  7151 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.944303  7270 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.918866  7246 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:33.928714  7167 raft_consensus.cc:2764] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:33.925940  7223 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:33.995471  4672 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:34.060884  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:33.935335  7263 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:34.061188  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:34.063953  4672 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:34.002120  7196 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:34.039407  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:34.039163  6773 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:33.936086  7215 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:34.040022  7187 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:34.040338  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:34.066977  4596 raft_consensus.cc:2393] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 3: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:33.994042  6940 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:33.991801  7264 consensus_queue.cc:237] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:34.043035  7184 raft_consensus.cc:993] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:34.006891  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:34.045975  7249 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:25:34.004261  7301 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:34.050254  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:34.003849  7097 consensus_meta.cc:287] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.091s	user 0.010s	sys 0.003s
I20260709 12:25:33.995255  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:34.080071  7184 raft_consensus.cc:1081] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:25:34.092365  7193 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:34.096458  6940 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:34.098066  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:34.099026  5349 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:34.100735  4644 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:34.102489  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 109188us
I20260709 12:25:34.103600  4647 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:34.104674  7167 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:34.105644  7167 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:34.107261  6788 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:34.107803  6788 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.108197  6788 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:33.998530  4756 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:33.995850  7191 ts_tablet_manager.cc:1916] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260709 12:25:34.000885  7045 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:33.999542  6897 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:34.115512  4750 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:25:34.062318  7251 consensus_meta.cc:287] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.715s	user 0.009s	sys 0.001s
I20260709 12:25:34.080075  5344 raft_consensus.cc:2410] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEARNER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:25:34.080982  4605 raft_consensus.cc:2393] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:34.121387  4674 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:34.105162  4647 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:33.987947  7299 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 12:25:34.125218  5346 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 4: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:34.062911  4675 raft_consensus.cc:2889] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:34.138724  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:34.146503  4596 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:34.148512  4672 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:25:34.076400  7187 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:34.126513  6897 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:34.160709  4672 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:34.173418  7235 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:34.174520  7256 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:34.175508  7235 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:34.176621  4674 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
W20260709 12:25:34.158313  6801 consensus_meta.cc:287] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.001s
I20260709 12:25:34.120843  7193 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:25:34.127898  7251 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:34.166527  6897 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.216763  6897 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:34.217433  7215 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:34.139982  7301 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:25:34.218533  7270 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:34.214545  7187 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:34.156426  7249 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:25:34.214959  7251 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:34.202025  6801 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:34.246634  4596 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:34.249595  4645 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:34.251674  7270 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:34.252275  7270 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:34.154287  7246 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:34.252687  7223 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:34.185979  7167 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:25:34.205123  7245 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:34.275498  4596 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:34.248001  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.276641  4681 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:34.243587  6773 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:34.278493  4672 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:34.277247  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 102155us
I20260709 12:25:34.202978  7202 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:34.284547  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2383) took 1058 ms. Trace:
I20260709 12:25:34.286020  4672 rpcz_store.cc:276] 0709 12:25:33.226594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.250702 (+1024108us) service_pool.cc:224] Handling call
0709 12:25:34.278136 (+ 27434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.284314 (+  6178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.284495 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:34.283805  7306 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:25:34.253167  7270 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:34.297919  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:34.299969  4603 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:34.300105  5345 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:34.283887  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:34.304471  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2384) took 1025 ms. Trace:
I20260709 12:25:34.304821  4672 rpcz_store.cc:276] 0709 12:25:33.279063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.295416 (+1016353us) service_pool.cc:224] Handling call
0709 12:25:34.304042 (+  8626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.304201 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.304225 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:34.304265 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:34.304301 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.304347 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.304426 (+    79us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:34.306113  4678 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:34.319111  7219 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:34.321363  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1516) took 1197 ms. Trace:
I20260709 12:25:34.321856  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2322) took 1062 ms. Trace:
I20260709 12:25:34.321802  4675 rpcz_store.cc:276] 0709 12:25:33.124216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.923774 (+799558us) service_pool.cc:224] Handling call
0709 12:25:33.924068 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.273092 (+349024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.321137 (+ 48045us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b540019fd54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:34.321302 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12539264}
I20260709 12:25:34.322535  4748 rpcz_store.cc:276] 0709 12:25:33.259984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.848524 (+588540us) service_pool.cc:224] Handling call
0709 12:25:33.894025 (+ 45501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.894254 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.894284 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.901917 (+  7633us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.116658 (+214741us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:34.116708 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.116791 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.321553 (+204762us) raft_consensus.cc:1685] finished
0709 12:25:34.321597 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.321776 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":127336,"threads_started":1,"wal-append.queue_time_us":336691}
I20260709 12:25:34.323422  7151 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:34.324653  7264 raft_consensus.cc:3055] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:25:34.334791  7310 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 12:25:34.333950  7264 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:34.463871  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2310) took 1368 ms. Trace:
I20260709 12:25:34.934489  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:34.791824  4680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1511) took 1674 ms (client timeout 1506 ms). Trace:
I20260709 12:25:34.951334  4751 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:25:34.953338  4680 rpcz_store.cc:269] 0709 12:25:33.117435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.117622 (+   187us) service_pool.cc:224] Handling call
0709 12:25:34.791572 (+1673950us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54000b27d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:34.791744 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282302848}
W20260709 12:25:34.791323  4681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1520) took 1666 ms (client timeout 2052 ms). Trace:
I20260709 12:25:34.847770  5345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 391) took 1011 ms. Trace:
W20260709 12:25:34.940902  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 467821us
W20260709 12:25:34.794684  4605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1499) took 1706 ms (client timeout 2052 ms). Trace:
I20260709 12:25:34.792915  4500 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 162) took 5128 ms. Trace:
I20260709 12:25:34.859282  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1509) took 1749 ms. Trace:
W20260709 12:25:34.378309  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:34.798108  5348 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 423) took 1806 ms (client timeout 1848 ms). Trace:
I20260709 12:25:34.619374  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2317) took 1520 ms. Trace:
W20260709 12:25:34.716634  4676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48198 (request call id 363) took 1890 ms (client timeout 1828 ms). Trace:
I20260709 12:25:34.639719  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2307) took 1544 ms. Trace:
W20260709 12:25:34.955587  4605 rpcz_store.cc:269] 0709 12:25:33.087827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.088053 (+   226us) service_pool.cc:224] Handling call
0709 12:25:34.794420 (+1706367us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b5400034154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:34.794605 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2271312128}
W20260709 12:25:34.859830  4604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1502) took 1695 ms (client timeout 2039 ms). Trace:
I20260709 12:25:34.958318  4750 rpcz_store.cc:276] 0709 12:25:33.094834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.095630 (+   796us) service_pool.cc:224] Handling call
0709 12:25:33.095881 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.116068 (+1020187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.639394 (+523326us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001f1654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:34.639638 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2343936384}
W20260709 12:25:34.474236  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 1.14932e+06us
I20260709 12:25:34.957100  4747 rpcz_store.cc:276] 0709 12:25:33.099425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.306909 (+207484us) service_pool.cc:224] Handling call
0709 12:25:34.619194 (+1312285us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54000721d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:34.619310 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796244864}
W20260709 12:25:34.956665  5348 rpcz_store.cc:269] 0709 12:25:32.991757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.991897 (+   140us) service_pool.cc:224] Handling call
0709 12:25:34.797824 (+1805927us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54007d1254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:34.798020 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245542272}
W20260709 12:25:34.959131  4604 rpcz_store.cc:269] 0709 12:25:33.164473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.165517 (+  1044us) service_pool.cc:224] Handling call
0709 12:25:34.859614 (+1694097us) spinlock_profiling.cc:243] Waited 99.6 ms on lock 0x7b540032b9d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:34.859758 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229149056}
I20260709 12:25:34.955953  4678 rpcz_store.cc:276] 0709 12:25:33.110031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.110151 (+   120us) service_pool.cc:224] Handling call
0709 12:25:33.148272 (+ 38121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.306500 (+1158228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.858863 (+552363us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54001d29d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:34.859157 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2661231360}
W20260709 12:25:34.958187  4676 rpcz_store.cc:269] 0709 12:25:32.826235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.826364 (+   129us) service_pool.cc:224] Handling call
0709 12:25:34.716395 (+1890031us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54002dfd54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:34.716544 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":743537408}
I20260709 12:25:34.950731  4749 rpcz_store.cc:276] 0709 12:25:33.095773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.096054 (+   281us) service_pool.cc:224] Handling call
0709 12:25:33.114909 (+ 18855us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.260371 (+145462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.260403 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.260961 (+   558us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.319696 (+ 58735us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:33.320018 (+   322us) log.cc:844] Serialized 14 byte log entry
0709 12:25:34.463439 (+1143421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.463541 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.463604 (+    63us) raft_consensus.cc:1685] finished
0709 12:25:34.463623 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.463791 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1143118,"prepare.queue_time_us":161,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":1217981,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":170,"thread_start_us":203359,"threads_started":2,"wal-append.queue_time_us":58752,"wal-append.run_cpu_time_us":2513,"wal-append.run_wall_time_us":1003215}
W20260709 12:25:34.954607  4681 rpcz_store.cc:269] 0709 12:25:33.125402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.469604 (+344202us) service_pool.cc:224] Handling call
0709 12:25:34.791057 (+1321453us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b540007c454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:34.791238 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1587760896}
I20260709 12:25:34.954746  5345 rpcz_store.cc:276] 0709 12:25:33.835895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.836026 (+   131us) service_pool.cc:224] Handling call
0709 12:25:33.836303 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.300485 (+464182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.847450 (+546965us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b5400280854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:34.847673 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1066158464}
I20260709 12:25:34.793249  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:25:34.976666  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.976998  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:34.740041  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.979804  5346 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEARNER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:34.983876  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:34.984563  4749 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:34.985735  4748 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:25:34.637754  7291 log.cc:927] Time spent T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.941s	user 0.000s	sys 0.004s
W20260709 12:25:34.474416  7314 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
W20260709 12:25:34.459410  7284 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.876s	user 0.000s	sys 0.003s
W20260709 12:25:34.650986  6788 consensus_meta.cc:287] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.542s	user 0.012s	sys 0.000s
I20260709 12:25:35.020079  6788 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:35.023195  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1892) took 1916 ms. Trace:
I20260709 12:25:35.025995  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2303) took 1934 ms. Trace:
I20260709 12:25:35.026530  4754 rpcz_store.cc:276] 0709 12:25:33.091896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.092021 (+   125us) service_pool.cc:224] Handling call
0709 12:25:33.092309 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.142159 (+ 49850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.142208 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.142755 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.284587 (+141832us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:33.295452 (+ 10865us) log.cc:844] Serialized 14 byte log entry
0709 12:25:34.458706 (+1163254us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:34.462652 (+  3946us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b180002dca0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007effeb3cba40 00007effeb3c64e4 00007effeb3c6455 00007effee4ae060 00007effee4c0716 00007effee4d1287 00007effee4d1208
0709 12:25:34.462872 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.462954 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.025669 (+562715us) raft_consensus.cc:1685] finished
0709 12:25:35.025700 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.025913 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11893,"prepare.queue_time_us":11534,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":1307,"raft.queue_time_us":1257811,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":239,"spinlock_wait_cycles":34636544,"thread_start_us":184267,"threads_started":2,"wal-append.queue_time_us":225230}
W20260709 12:25:34.470803  7316 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
W20260709 12:25:34.471386  7311 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
W20260709 12:25:34.987304  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 153792us
I20260709 12:25:34.975528  7283 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.472877  7287 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:34.417521  7271 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.474977  7176 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:34.968848  7041 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:34.911586  7097 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.374828  7219 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.515296  7250 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:34.574856  7235 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:34.994160  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:35.064715  4747 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:25:34.772841  7184 raft_consensus.cc:3055] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:34.789821  7141 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:34.474812  7077 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:34.460793  6915 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:35.003971  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.012229  4498 catalog_manager.cc:5697] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:25:35.014510  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:35.019333  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:34.972924  7302 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:34.973127  7280 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.081446  7235 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.082154  4756 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:35.081875  7280 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.083839  7235 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:35.084347  7235 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:35.084760  7280 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:35.048060  7283 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:34.932555  6773 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.023789  4602 rpcz_store.cc:276] 0709 12:25:33.106611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.106748 (+   137us) service_pool.cc:224] Handling call
0709 12:25:33.136258 (+ 29510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.136734 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.136781 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.170740 (+ 33959us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.326761 (+156021us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:33.326800 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.326887 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.807762 (+480875us) spinlock_profiling.cc:243] Waited 5.03 ms on lock 0x7b4800080058. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff33dfaa1 00007efff33d6b37 00007efff33d69ca 00007efff34ce26f 00007efff34cdf37 00007efff34e255c 00007efff34e2508 00007efff34e2488 00007efff34e2443 00007efff34e16ec 00007efff240ba96
0709 12:25:34.636982 (+829220us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:35.022676 (+385694us) raft_consensus.cc:1685] finished
0709 12:25:35.022709 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.022953 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12094,"raft.queue_time_us":646,"raft.run_cpu_time_us":8878,"raft.run_wall_time_us":659782,"spinlock_wait_cycles":18175616,"thread_start_us":109238,"threads_started":1,"wal-append.queue_time_us":272065}
I20260709 12:25:34.981637  7282 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.931977  7296 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:25:34.474612  7084 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.528s
I20260709 12:25:34.955772  4500 rpcz_store.cc:276] 0709 12:25:29.665194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.667755 (+  2561us) service_pool.cc:224] Handling call
0709 12:25:29.857031 (+189276us) spinlock_profiling.cc:243] Waited 90.2 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee420090 00007effee4bef3a 00007efff4274e9c 00007efff437304e 00007efff28342a4 00007efff2834232 00007efff2834161 00007efff28340dc 00007efff2833392 00007effec7f148c 00007effec7f08c6
0709 12:25:34.792827 (+4935796us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:31.856256 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:31.862069 (+  5813us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:31.862122 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:25:31.862141 (+    19us) tablet.cc:662] Decoding operations
0709 12:25:31.862775 (+   634us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:31.862826 (+    51us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:31.862851 (+    25us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:25:31.863317 (+   466us) tablet.cc:701] Row locks acquired
0709 12:25:31.863358 (+    41us) write_op.cc:260] PREPARE: finished
0709 12:25:31.863531 (+   173us) write_op.cc:270] Start()
0709 12:25:31.863676 (+   145us) write_op.cc:276] Timestamp: P: 1783599931863468 usec, L: 0
0709 12:25:31.863692 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 12:25:31.864324 (+   632us) log.cc:844] Serialized 1674 byte log entry
0709 12:25:32.919591 (+1055267us) op_driver.cc:464] REPLICATION: finished
0709 12:25:33.224686 (+305095us) write_op.cc:301] APPLY: starting
0709 12:25:33.251624 (+ 26938us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:33.294066 (+ 42442us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:33.308631 (+ 14565us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:33.361044 (+ 52413us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:33.361259 (+   215us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:25:33.361305 (+    46us) write_op.cc:312] APPLY: finished
0709 12:25:33.361543 (+   238us) log.cc:844] Serialized 41 byte log entry
0709 12:25:33.371968 (+ 10425us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:33.473800 (+101832us) write_op.cc:454] Released schema lock
0709 12:25:33.578526 (+104726us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:33.588138 (+  9612us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:33.588182 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":8,"spinlock_wait_cycles":207573760,"child_traces":[["op",{"apply.queue_time_us":275763,"apply.run_cpu_time_us":15594,"apply.run_wall_time_us":372736,"num_ops":4,"prepare.queue_time_us":390737,"prepare.run_cpu_time_us":18032,"prepare.run_wall_time_us":696721,"raft.queue_time_us":157096,"raft.run_cpu_time_us":21498,"raft.run_wall_time_us":175969,"replication_time_us":1055728,"thread_start_us":1062135,"threads_started":4,"wal-append.queue_time_us":687782,"wal-append.run_cpu_time_us":43356,"wal-append.run_wall_time_us":2072768}]]}
I20260709 12:25:35.046073  4751 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
W20260709 12:25:35.036197  6897 consensus_meta.cc:287] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.819s	user 0.006s	sys 0.006s
I20260709 12:25:35.057891  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.059514  4755 raft_consensus.cc:3055] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:34.990814  7232 consensus_queue.cc:1059] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 12:25:34.453189  7243 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.068193  7184 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:35.072196  4748 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:34.492980  7264 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.081643  7302 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:34.970052  7236 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:34.973228  7159 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.068962  7250 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.063499  4749 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:35.101778  4755 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:25:35.089942  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 112341us
I20260709 12:25:35.092890  6773 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.096006  4498 catalog_manager.cc:5697] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 1 to 2, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 2 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:25:35.106884  4747 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:25:35.111970  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.090148  7280 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:35.113963  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:35.081642  6915 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.115803  4751 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:35.047573  7271 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.050092  7287 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:35.051123  7176 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:35.057121  7097 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:35.066290  7219 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.108564  4756 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:25:35.099829  7202 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:35.117687  7243 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:35.122210  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.138931  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:35.111478  7084 consensus_queue.cc:1243] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:25:35.125993  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:35.086576  7141 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.084782  7235 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.139158  4749 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:35.089395  7077 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:35.139679  4747 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:35.122700  4498 catalog_manager.cc:5697] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 7 to 8, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 8 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
W20260709 12:25:35.140435  7321 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:25:35.140538  7325 raft_consensus.cc:3055] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:35.122078  6915 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.138550  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:35.142351  7271 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:35.133786  7201 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.145691  7184 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:35.115000  7236 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:35.112975  7159 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:35.149920  7280 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:35.151041  7323 raft_consensus.cc:1064] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:35.152436  7283 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:35.114668  7250 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.165037  7325 raft_consensus.cc:740] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:35.161226  4571 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:35.111272  6897 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.134323  7232 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.170065  4751 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:35.180461  4747 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
W20260709 12:25:35.187592  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.187407  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:35.197803  4749 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:35.101363  7282 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.160329  7084 consensus_queue.cc:1059] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:25:35.172649  6788 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:35.185348  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:35.199735  4754 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:35.200004  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 370) took 1428 ms. Trace:
I20260709 12:25:35.203377  4748 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:35.186311  7141 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:35.202328  7328 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.188159  7184 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:35.185264  7329 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:35.203557  4672 rpcz_store.cc:276] 0709 12:25:33.771185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.326268 (+555083us) service_pool.cc:224] Handling call
0709 12:25:34.333807 (+  7539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.334167 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.334201 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.335947 (+  1746us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.449416 (+113469us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:34.449472 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.449586 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.199793 (+750207us) raft_consensus.cc:1685] finished
0709 12:25:35.199813 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.199946 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":2107,"thread_start_us":72161,"threads_started":1,"wal-append.queue_time_us":675469}
I20260709 12:25:35.209390  7141 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:35.220391  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.221095  4748 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:35.182435  7201 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:35.229092  4754 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:35.191463  7283 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.209649  4749 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:35.240782  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.243012  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:35.243697  4752 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:35.259294  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 223903us
I20260709 12:25:35.260346  7327 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:35.260952  7327 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.263783  5345 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:35.264098  7327 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:35.265199  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:35.267421  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.268268  4749 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:35.205840  6915 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:35.259593  7335 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.273826  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:35.274416  4680 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:35.274487  5348 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:35.274955  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:35.276468  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 115310us
I20260709 12:25:35.277715  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:35.278534  4680 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:35.282783  4681 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:25:35.285274  4676 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:25:35.288477  4752 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:35.288668  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:35.201226  7232 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.280498  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.266074  5343 raft_consensus.cc:2393] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:35.294754  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.291437  4571 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:35.292408  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:35.295634  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:35.222075  7282 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:35.292408  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:35.302238  4749 raft_consensus.cc:2393] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:35.302630  4678 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:35.293874  4748 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:35.271114  7335 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.296465  4752 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:35.309262  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.310035  4749 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:35.296300  4681 raft_consensus.cc:2368] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10.
W20260709 12:25:35.296751  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 163474us
I20260709 12:25:35.307538  4680 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:35.312196  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:35.313191  7282 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:35.314831  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:35.315790  7282 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:35.316243  7282 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.316656  7282 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:35.321105  4680 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:35.318140  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:35.321725  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 120077us
I20260709 12:25:35.322324  4749 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:35.324339  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 49512.9us
I20260709 12:25:35.293038  6915 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:35.238795  7141 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:35.327584  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.334705  4752 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:25:35.336701  4749 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:35.338019  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.309219  7235 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:35.338654  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.269785  7283 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:35.342928  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.343688  4749 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:35.345031  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:35.330775  7301 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:35.346169  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.222847  7329 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:35.219583  7328 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.327196  6915 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:35.361224  6915 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:35.279760  7325 consensus_queue.cc:260] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.324523  7334 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:25:35.334441  7270 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:35.338783  7219 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:35.350242  7326 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:25:35.352743  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:35.354861  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:35.355377  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.349201  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.330603  7332 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 12:25:35.348719  4749 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:35.339591  7302 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:35.359925  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:35.561834  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 270469us
I20260709 12:25:35.564498  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:35.566174  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:35.567142  4749 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:35.568130  4676 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 4: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:35.346774  7077 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:35.562862  4748 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:35.378805  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:35.432533  4755 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.573757  4755 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:35.574918  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:25:35.579524  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 200790us
I20260709 12:25:35.571812  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:35.355103  7301 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.638137  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:35.638978  4679 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:35.551024  7235 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.637259  4676 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:35.637856  7325 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:35.640630  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 64360.1us
I20260709 12:25:35.641160  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.641870  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.642472  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:35.637967  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.361894  6897 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:35.570122  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 247231us
I20260709 12:25:35.422864  7246 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.363512  7159 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:35.424508  7236 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:35.645964  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.727067  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:35.730618  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:35.730769  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:35.731194  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 775783us
W20260709 12:25:35.731241  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 759938us
I20260709 12:25:35.727068  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.646202  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:35.736960  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.736594  4604 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:35.569149  6788 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:35.643720  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:35.564685  7283 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:35.740837  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:35.738770  4601 raft_consensus.cc:2410] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:35.741832  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:35.738564  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.741700  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.564420  7250 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:35.578716  7334 consensus_queue.cc:1243] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:35.500336  7249 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:35.742113  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:35.733448  7184 consensus_meta.cc:287] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.000s
I20260709 12:25:35.565503  7232 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:35.561919  7270 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:35.759877  4749 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:25:35.643764  7301 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:35.422421  7282 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.362813  7339 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:35.764209  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.780154  4755 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:25:35.770825  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.782605  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.424930  6915 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:35.783865  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 48309.7us
I20260709 12:25:35.791006  4681 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:35.762669  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:35.784502  5312 connection.cc:256] RPC call timeout handler was delayed by -1.3199899810001625s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
I20260709 12:25:35.788646  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:35.793615  4601 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:35.794036  4755 pending_rounds.cc:85] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 5
I20260709 12:25:35.639839  7328 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:35.738850  7167 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:35.754387  7223 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.786820  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.788650  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:35.638422  7346 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:35.793962  5312 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:35.569231  7327 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.183s
I20260709 12:25:35.792392  5314 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:35.742034  7159 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:35.786260  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:35.750607  7329 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:35.630924  7219 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.157s
I20260709 12:25:35.422713  7343 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.401016  7342 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.532014  7331 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.362886  7323 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:35.750869  6788 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.753880  7246 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:35.751562  7235 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:35.794528  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:36.355063  4601 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:36.355569  7191 ts_tablet_manager.cc:1929] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 4.4
I20260709 12:25:35.738277  7347 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:36.130450  4676 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:36.012086  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1899) took 1840 ms. Trace:
I20260709 12:25:36.113961  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:36.387727  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.832896  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:36.388713  4720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.013s	sys 0.013s
I20260709 12:25:36.389632  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:35.875586  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:35.913748  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:36.356281  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:35.794549  4755 pending_rounds.cc:107] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:25:35.795506  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:36.114677  5344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53808 (request call id 407) took 1045 ms. Trace:
I20260709 12:25:36.391026  4755 raft_consensus.cc:2889] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:36.389683  4597 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:35.790223  4678 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
W20260709 12:25:36.389660  4720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.013s	sys 0.013s
I20260709 12:25:36.391968  4676 raft_consensus.cc:2393] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:36.393070  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1882) took 1334 ms. Trace:
I20260709 12:25:36.393189  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:36.387292  4599 rpcz_store.cc:276] 0709 12:25:34.171716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.171898 (+   182us) service_pool.cc:224] Handling call
0709 12:25:34.860003 (+688105us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.040451 (+180448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.040496 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.042063 (+  1567us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.108577 (+ 66514us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:35.108620 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.108690 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.940852 (+832162us) raft_consensus.cc:1685] finished
0709 12:25:35.940883 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.011681 (+ 70798us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b540032b9d0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:36.011989 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1841,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":138,"spinlock_wait_cycles":1581472128,"thread_start_us":216871,"threads_started":2,"wal-append.queue_time_us":119931}
W20260709 12:25:36.393627  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 599702us
W20260709 12:25:36.396014  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 603686us
W20260709 12:25:36.432574  4720 connection.cc:256] RPC call timeout handler was delayed by -1.2562961319999886s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
I20260709 12:25:36.432663  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:36.433246  5314 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:36.394245  4674 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:36.434543  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:36.435083  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:36.391144  4601 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:36.435272  4603 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:36.394167  4755 rpcz_store.cc:276] 0709 12:25:35.058999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.059106 (+   107us) service_pool.cc:224] Handling call
0709 12:25:35.059370 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.431984 (+372614us) spinlock_profiling.cc:243] Waited 8.46 ms on lock 0x7b4c003d4a80. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effec782a20 00007effec793a29 00007effec759644 00007effec780f2d 00007effec780b93 00007effee4b118b 00007effee4b1cb5 00007effee4acf30 00007effee4ae7de 00007effee4b8b49 00007effee4b9416
0709 12:25:36.392854 (+960870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.393023 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19456256}
I20260709 12:25:36.391737  4600 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:36.435977  7260 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:36.439134  7045 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:36.439504  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:36.440598  4754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1889) took 1379 ms (client timeout 1527 ms). Trace:
I20260709 12:25:36.441120  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:36.441061  4754 rpcz_store.cc:269] 0709 12:25:35.061483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.113718 (+ 52235us) service_pool.cc:224] Handling call
0709 12:25:36.440440 (+1326722us) spinlock_profiling.cc:243] Waited 81.1 ms on lock 0x7b54000726d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:36.440540 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186467712}
I20260709 12:25:36.441934  4596 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:36.442977  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:36.443584  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:36.443641  4596 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:36.444605  7191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b7b0c6c811e455abe898cb4d21acad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:36.444836  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:36.445225  7173 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:36.446030  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:35.785259  7270 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:36.446738  4596 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:36.440703  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:36.365411  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.392453  5344 rpcz_store.cc:276] 0709 12:25:35.069158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.069364 (+   206us) service_pool.cc:224] Handling call
0709 12:25:35.072766 (+  3402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.118646 (+ 45880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.118675 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:35.118719 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:35.118761 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.114389 (+995628us) spinlock_profiling.cc:243] Waited 45.7 ms on lock 0x7b54006fbdd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:36.114449 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.114609 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105051520}
I20260709 12:25:35.914189  7332 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.791882  7335 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:36.447700  4678 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:36.450462  4681 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
W20260709 12:25:36.395633  4680 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.018s	user 0.008s	sys 0.019s
I20260709 12:25:36.450505  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:36.433686  7159 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:36.453310  7246 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:36.454836  7235 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:36.345988  7352 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 12:25:35.834364  7302 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.621s
I20260709 12:25:36.454695  6788 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:36.389153  7347 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:36.457934  4571 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.010s	sys 0.005s
I20260709 12:25:36.395077  4597 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:36.458787  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:35.864428  7141 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:36.385807  7346 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:36.436251  7280 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:25:36.009526  7264 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:36.461272  4644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.000s	sys 0.022s
W20260709 12:25:36.537750  4644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.000s	sys 0.022s
W20260709 12:25:36.538131  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 703448us
I20260709 12:25:36.461714  4755 raft_consensus.cc:2955] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:25:35.791266  7339 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:36.443667  7333 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.095s	user 0.013s	sys 0.005s
I20260709 12:25:36.455452  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:36.467937  4749 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.004s	sys 0.015s
W20260709 12:25:36.556025  4749 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.015s	sys 0.022s
I20260709 12:25:36.449461  7250 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:36.462031  4601 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:36.433908  7342 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:36.464118  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.462110  4680 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.085s	user 0.013s	sys 0.021s
I20260709 12:25:36.520202  7347 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:36.467769  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:36.459885  7202 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:36.519796  4603 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
W20260709 12:25:36.519917  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 61222.3us
W20260709 12:25:36.466392  7184 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.014s	sys 0.007s
I20260709 12:25:36.394359  7223 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:36.462415  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:35.785918  7249 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:36.467526  7341 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.009s	sys 0.025s
I20260709 12:25:36.432128  7331 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:36.432315  7167 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:36.436825  7296 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:36.519155  7235 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:25:36.517841  4571 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.013s	sys 0.013s
I20260709 12:25:36.462857  7324 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:36.531147  7077 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:36.467190  4569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.003s	sys 0.016s
I20260709 12:25:36.467162  7332 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:36.388697  7343 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:36.461138  7345 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.121s
I20260709 12:25:36.840888  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 1910) took 1054 ms. Trace:
I20260709 12:25:36.842095  4603 rpcz_store.cc:276] 0709 12:25:35.786836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.432112 (+645276us) service_pool.cc:224] Handling call
0709 12:25:36.840824 (+408712us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:36.839438  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:36.843510  4569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.080s	user 0.007s	sys 0.028s
W20260709 12:25:36.844350  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 1.10318e+06us
I20260709 12:25:36.845309  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2395) took 1506 ms. Trace:
I20260709 12:25:36.845847  4680 rpcz_store.cc:276] 0709 12:25:35.338975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.339097 (+   122us) service_pool.cc:224] Handling call
0709 12:25:35.339450 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.339710 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:36.567050 (+1227340us) log.cc:844] Serialized 14 byte log entry
0709 12:25:36.837935 (+270885us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:36.838019 (+    84us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:36.838089 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.838177 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.838330 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3829,"prepare.queue_time_us":455284,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":6522,"raft.queue_time_us":807,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":152,"thread_start_us":1460210,"threads_started":2}
I20260709 12:25:36.686942  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 373) took 2729 ms. Trace:
I20260709 12:25:36.636897  6915 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:36.548673  7282 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:36.567135  7250 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:36.636301  7097 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:36.780623  4748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.818s	user 0.006s	sys 0.003s
I20260709 12:25:36.545737  7339 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:36.529258  7346 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.554452  7246 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:25:36.842876  7341 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.009s	user 0.024s	sys 0.047s
I20260709 12:25:36.558292  7353 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:36.849360  4675 rpcz_store.cc:276] 0709 12:25:33.944653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.336436 (+391783us) service_pool.cc:224] Handling call
0709 12:25:34.347268 (+ 10832us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.348682 (+  1414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.348711 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.897503 (+548792us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.291617 (+394114us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:35.294010 (+  2393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.294108 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.443100 (+1148992us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:36.621114 (+178014us) raft_consensus.cc:1685] finished
0709 12:25:36.621139 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.673554 (+ 52415us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1031,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":2606,"spinlock_wait_cycles":13824896,"thread_start_us":309023,"threads_started":1,"wal-append.queue_time_us":364944}
W20260709 12:25:36.656677  7350 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.587s	user 0.001s	sys 0.000s
W20260709 12:25:36.691277  7354 log.cc:927] Time spent T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:25:36.867281  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:25:36.863864  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 217450us
I20260709 12:25:36.868644  4679 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:36.854804  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 956558us
W20260709 12:25:36.859107  4748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.897s	user 0.007s	sys 0.004s
I20260709 12:25:36.862959  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:36.868747  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:36.915158  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.850080  7184 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.022s	sys 0.018s
I20260709 12:25:36.831442  7342 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:36.651436  7329 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.637159  7336 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:36.931766  4603 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:36.950001  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 107462us
W20260709 12:25:36.937664  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 70460.8us
I20260709 12:25:36.951417  4569 leader_election.cc:400] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10.
I20260709 12:25:36.951814  4569 leader_election.cc:403] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 12:25:36.945961  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 83073.9us
W20260709 12:25:36.936479  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.954484  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:36.955390  7159 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:36.951928  7077 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:36.924402  7358 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:25:36.959157  7332 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:36.924448  7353 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:36.962949  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:36.968724  4603 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
W20260709 12:25:36.970108  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.970479  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 508087us
W20260709 12:25:36.980417  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.961665  7287 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:36.983430  4603 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:36.984304  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:36.985431  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:36.985690  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1541) took 1102 ms. Trace:
I20260709 12:25:36.986188  7360 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:36.987191  7360 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:36.987963  7360 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:36.988270  7360 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.988536  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1893) took 1926 ms. Trace:
I20260709 12:25:36.988667  7360 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.975886  4679 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:36.988934  4749 rpcz_store.cc:276] 0709 12:25:35.062484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.783088 (+720604us) service_pool.cc:224] Handling call
0709 12:25:35.783514 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.783830 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:36.558365 (+774535us) log.cc:844] Serialized 14 byte log entry
0709 12:25:36.961149 (+402784us) spinlock_profiling.cc:243] Waited 905 us on lock 0x7b1800342fa0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effee05e4bc 00007effeb3ac48b 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a
0709 12:25:36.988192 (+ 27043us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:36.988253 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:36.988301 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.988369 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.988479 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26588,"prepare.queue_time_us":901487,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"spinlock_wait_cycles":2081664,"thread_start_us":982481,"threads_started":2,"wal-append.queue_time_us":373755}
I20260709 12:25:36.990059  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1533) took 1262 ms. Trace:
I20260709 12:25:36.994041  4679 rpcz_store.cc:276] 0709 12:25:35.727765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.740611 (+ 12846us) service_pool.cc:224] Handling call
0709 12:25:36.989907 (+1249296us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540038e5d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:36.990006 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2552696576}
I20260709 12:25:36.986795  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1540) took 1195 ms. Trace:
I20260709 12:25:36.997867  4681 rpcz_store.cc:276] 0709 12:25:35.791230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.795278 (+  4048us) service_pool.cc:224] Handling call
0709 12:25:36.986610 (+1191332us) spinlock_profiling.cc:243] Waited 58.9 ms on lock 0x7b54001d1ad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:36.986729 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135378688}
I20260709 12:25:37.002876  4756 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:25:37.003779  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1898) took 1940 ms. Trace:
I20260709 12:25:37.004156  4756 rpcz_store.cc:276] 0709 12:25:35.063608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.002013 (+1938405us) service_pool.cc:224] Handling call
0709 12:25:37.002575 (+   562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.003404 (+   829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.003712 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.012828  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:37.014595  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:37.015317  4604 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:37.022325  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 400) took 2046 ms. Trace:
I20260709 12:25:37.022846  4750 rpcz_store.cc:276] 0709 12:25:34.975943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.976140 (+   197us) service_pool.cc:224] Handling call
0709 12:25:34.976419 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.153167 (+176748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.153211 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:35.153271 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:35.153325 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.022001 (+1868676us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54000a7654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:37.022070 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.022250 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405775744}
I20260709 12:25:37.026396  7176 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:37.028956  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.039311  6773 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:37.040349  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:37.043296  4647 connection.cc:256] RPC call timeout handler was delayed by -1.1944363269999485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
W20260709 12:25:37.043972  4647 connection.cc:256] RPC call timeout handler was delayed by -1.1404363269998612s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
I20260709 12:25:37.047706  7250 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:37.059051  7352 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:37.059722  7352 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:36.990389  7077 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:37.062840  7361 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.063006  7077 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:37.065059  4675 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:36.990741  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:36.991585  7342 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:37.066493  7342 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:37.067121  4749 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:25:36.984298  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1911) took 1197 ms. Trace:
W20260709 12:25:36.992545  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.067867  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2327) took 2003 ms. Trace:
I20260709 12:25:37.067822  4603 rpcz_store.cc:276] 0709 12:25:35.787080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.982866 (+1195786us) service_pool.cc:224] Handling call
0709 12:25:36.983197 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:36.983996 (+   799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.984232 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.068406  4749 rpcz_store.cc:276] 0709 12:25:35.064620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.066387 (+2001767us) service_pool.cc:224] Handling call
0709 12:25:37.066733 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.067584 (+   851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.067807 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:36.992102  4645 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:37.070161  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 78196.5us
I20260709 12:25:37.063434  7361 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:36.988675  7358 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:37.071534  7338 raft_consensus.cc:3055] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:37.072142  7338 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:37.073233  7338 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:37.074762  7338 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:37.077829  7271 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.078514  7271 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:37.072160  7358 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:37.080749  7351 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:25:37.085153  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1913) took 1220 ms. Trace:
I20260709 12:25:37.086139  4603 rpcz_store.cc:276] 0709 12:25:35.864811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.068726 (+1203915us) service_pool.cc:224] Handling call
0709 12:25:37.069024 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.082013 (+ 12989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.082060 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.082125 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:37.082178 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.084842 (+  2664us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b5400141ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:37.084901 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.085077 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29331200}
I20260709 12:25:37.070665  7342 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:37.088481  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:37.096287  4603 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:37.097816  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:37.099272  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:36.863736  7324 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:36.945262  7302 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:37.102140  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:36.868901  7141 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:25:37.106956  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1895) took 2044 ms. Trace:
W20260709 12:25:37.109681  4568 leader_election.cc:336] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Timed out: RequestConsensusVote RPC to 127.3.89.196:39067 timed out after 1.848s (SENT)
I20260709 12:25:36.986052  4678 rpcz_store.cc:276] 0709 12:25:35.883091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.432360 (+549269us) service_pool.cc:224] Handling call
0709 12:25:36.985504 (+553144us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b540019f354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:36.985633 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32847616}
I20260709 12:25:37.100368  4603 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:36.859947  7235 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:37.107685  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:36.940784  7335 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.169s
I20260709 12:25:37.124202  4755 rpcz_store.cc:276] 0709 12:25:35.062953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.451205 (+1388252us) service_pool.cc:224] Handling call
0709 12:25:36.451568 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:36.461557 (+  9989us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:36.557732 (+ 96175us) log.cc:844] Serialized 14 byte log entry
0709 12:25:36.926159 (+368427us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:36.928188 (+  2029us) log.cc:844] Serialized 26 byte log entry
0709 12:25:36.930028 (+  1840us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:37.104260 (+174232us) log.cc:844] Serialized 14 byte log entry
0709 12:25:37.106541 (+  2281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.106656 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.106712 (+    56us) raft_consensus.cc:1685] finished
0709 12:25:37.106731 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.106886 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32934,"raft.queue_time_us":13832,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":29513,"thread_start_us":349489,"threads_started":1,"wal-append.queue_time_us":380289}
I20260709 12:25:37.127187  4603 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:37.136438  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:36.979861  6787 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10.
I20260709 12:25:37.137221  4755 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 4: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:36.983248  7097 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:37.136445  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1896) took 2073 ms. Trace:
I20260709 12:25:37.188187  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:37.105249  7331 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:36.928345  7246 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:37.217595  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1891) took 2155 ms. Trace:
I20260709 12:25:36.962994  7336 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:37.130041  7296 raft_consensus.cc:697] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:37.224187  4749 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:25:36.962713  7329 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:37.133334  7345 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:36.953080  7368 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.271s
W20260709 12:25:37.256449  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 69805.8us
W20260709 12:25:37.230962  7045 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.000s	sys 0.017s
I20260709 12:25:37.182896  7202 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:37.273269  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1909) took 1486 ms. Trace:
W20260709 12:25:37.309939  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 323467us
I20260709 12:25:37.137605  7243 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.119s
W20260709 12:25:37.257700  7369 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:25:37.132778  7302 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:37.126132  7344 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.291190  4752 rpcz_store.cc:276] 0709 12:25:35.062008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.355309 (+293301us) service_pool.cc:224] Handling call
0709 12:25:35.355849 (+   540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.530974 (+175125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.531019 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.569966 (+ 38947us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.938831 (+368865us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:35.938888 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.938988 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.217341 (+1278353us) raft_consensus.cc:1685] finished
0709 12:25:37.217360 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.217514 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":67414,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":128,"thread_start_us":374798,"threads_started":2,"wal-append.queue_time_us":849026}
I20260709 12:25:37.320535  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:37.321976  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:37.230293  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 247682us
I20260709 12:25:37.255875  7263 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:37.277475  7331 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:37.300598  4645 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:37.276832  7348 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:37.312386  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:37.314795  4747 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:37.310124  4600 rpcz_store.cc:276] 0709 12:25:35.786593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.860048 (+1073455us) service_pool.cc:224] Handling call
0709 12:25:36.860434 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:36.860668 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:36.925413 (+ 64745us) log.cc:844] Serialized 14 byte log entry
0709 12:25:37.272322 (+346909us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.272572 (+   250us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:37.272679 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.272814 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.273170 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13825,"prepare.queue_time_us":66415,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"thread_start_us":332575,"threads_started":1}
W20260709 12:25:37.400367  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 88060.5us
I20260709 12:25:37.401787  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:37.390787  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 43508.6us
I20260709 12:25:37.345660  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:37.404299  4752 raft_consensus.cc:2376] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9. Re-sending same reply.
I20260709 12:25:37.405246  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:37.302032  7045 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.000s	sys 0.018s
I20260709 12:25:37.406045  7045 raft_consensus.cc:697] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:37.406926  7045 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:37.293335  7380 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:25:37.383556  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.411978  4752 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:37.413292  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:37.419410  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:37.424695  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:25:37.355597  7260 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.916s	user 0.006s	sys 0.014s
I20260709 12:25:37.431483  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:37.434108  4601 consensus_meta.cc:287] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.869s	user 0.012s	sys 0.000s
I20260709 12:25:37.394016  7223 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:37.435084  4601 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:37.452692  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 1915) took 1350 ms. Trace:
I20260709 12:25:37.453154  4601 rpcz_store.cc:276] 0709 12:25:36.102389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.447791 (+345402us) service_pool.cc:224] Handling call
0709 12:25:37.452602 (+1004811us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:37.399235  7366 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.437s	user 0.004s	sys 0.006s
I20260709 12:25:37.492180  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:37.484344  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:37.495163  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 67281.6us
I20260709 12:25:37.319164  7250 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.135s
W20260709 12:25:37.434648  7260 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.995s	user 0.007s	sys 0.014s
I20260709 12:25:37.422125  7097 raft_consensus.cc:2764] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:25:37.399802  7348 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:37.547241  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:37.450515  7084 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:37.399394  7263 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:37.570593  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.434556  7386 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:37.555931  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:37.489871  7077 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.451061  6773 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:37.434643  7287 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:37.574342  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.552634  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:37.567487  7380 raft_consensus.cc:697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:37.567008  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:37.782020  4752 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
W20260709 12:25:37.782562  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 215636us
W20260709 12:25:37.560115  7367 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.457s	user 0.001s	sys 0.000s
I20260709 12:25:37.394891  7344 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:37.616247  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.555042  7260 raft_consensus.cc:697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:37.569293  7348 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:37.758061  4644 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.196:39067 timed out after 2.128s (SENT) [suppressed 23 similar messages]
I20260709 12:25:37.777802  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2326) took 2713 ms. Trace:
I20260709 12:25:37.574672  7084 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.143040  4754 rpcz_store.cc:276] 0709 12:25:35.063171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.442134 (+1378963us) service_pool.cc:224] Handling call
0709 12:25:36.442402 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:36.442601 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:36.676878 (+234277us) log.cc:844] Serialized 14 byte log entry
0709 12:25:37.136168 (+459290us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.136213 (+    45us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:37.136258 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.136311 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.136397 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":417640,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":55677,"raft.queue_time_us":1043,"thread_start_us":650866,"threads_started":2}
I20260709 12:25:37.860057  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:37.860809  4754 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:37.795661  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:37.861626  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 66071.5us
I20260709 12:25:37.862542  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:37.841418  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:37.843264  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:37.603994  7201 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:37.575796  7384 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:37.847853  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 89819.7us
I20260709 12:25:37.894537  7384 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.895074  7384 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:37.895224  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.847765  4747 rpcz_store.cc:276] 0709 12:25:35.064393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.045437 (+1981044us) service_pool.cc:224] Handling call
0709 12:25:37.045893 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.400105 (+354212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.777425 (+377320us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400309b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:37.777726 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617575680}
I20260709 12:25:37.896482  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:37.897104  4747 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:37.897949  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:37.898334  7365 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.607s	user 0.001s	sys 0.000s
I20260709 12:25:37.898666  4747 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:37.899588  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:37.900087  4747 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:37.900816  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:37.863816  4601 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:37.901872  4747 raft_consensus.cc:2410] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:37.902642  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:37.902779  7374 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.606s	user 0.001s	sys 0.000s
I20260709 12:25:37.903261  4747 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:25:37.904552  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2332) took 2838 ms. Trace:
I20260709 12:25:37.904855  4747 rpcz_store.cc:276] 0709 12:25:35.066134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.903827 (+2837693us) service_pool.cc:224] Handling call
0709 12:25:37.904094 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.904260 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.904285 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.904319 (+    34us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:37.904347 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.904395 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.904494 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.905871  4747 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:25:37.906443  4747 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:37.906746  4747 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.909147  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2329) took 2844 ms. Trace:
I20260709 12:25:37.872901  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:37.909677  4749 rpcz_store.cc:276] 0709 12:25:35.065079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.069336 (+2004257us) service_pool.cc:224] Handling call
0709 12:25:37.091583 (+ 22247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.295007 (+203424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.908836 (+613829us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b540004e054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:37.909073 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":293664128}
I20260709 12:25:37.910831  4752 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.911427  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2335) took 2844 ms. Trace:
I20260709 12:25:37.911736  4752 rpcz_store.cc:276] 0709 12:25:35.066812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.910361 (+2843549us) service_pool.cc:224] Handling call
0709 12:25:37.910647 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.911216 (+   569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.911376 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:37.876248  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:37.886607  4603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51130
I20260709 12:25:37.913273  4603 raft_consensus.cc:606] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:37.779536  6773 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.900591  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1892) took 2838 ms. Trace:
W20260709 12:25:37.888505  7382 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.325s	user 0.000s	sys 0.000s
I20260709 12:25:37.888609  7282 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:37.892222  7381 log.cc:927] Time spent T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.440s	user 0.000s	sys 0.001s
I20260709 12:25:37.892629  7352 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:37.893913  7394 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:25:37.753317  7271 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:37.919324  4747 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:37.901341  7235 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:37.815479  7358 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:37.557111  7202 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:37.934084  4752 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:37.922305  4748 rpcz_store.cc:276] 0709 12:25:35.062256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.636431 (+574175us) service_pool.cc:224] Handling call
0709 12:25:35.742053 (+105622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.779284 (+ 37231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.779314 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.792169 (+ 12855us) log.cc:844] Serialized 26 byte log entry
0709 12:25:36.931818 (+1139649us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:37.398865 (+467047us) log.cc:844] Serialized 14 byte log entry
0709 12:25:37.399134 (+   269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.406985 (+  7851us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.900292 (+493307us) raft_consensus.cc:1685] finished
0709 12:25:37.900320 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.900489 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":262642,"prepare.queue_time_us":288344,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":2476,"raft.queue_time_us":37959,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":33360,"thread_start_us":1181120,"threads_started":2,"wal-append.queue_time_us":1106178}
I20260709 12:25:37.544001  7296 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:38.081159  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1903) took 2812 ms. Trace:
I20260709 12:25:38.081627  4752 rpcz_store.cc:276] 0709 12:25:35.268297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.912466 (+2644169us) service_pool.cc:224] Handling call
0709 12:25:37.933888 (+ 21422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.080898 (+147010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.081101 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:38.067710  4747 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:37.844990  7324 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:37.913300  7287 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:38.076601  7392 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
W20260709 12:25:38.090711  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.096120  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:43864 (request call id 2333) took 3016 ms. Trace:
I20260709 12:25:37.877601  7084 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:37.877928  7077 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.110739  4747 rpcz_store.cc:276] 0709 12:25:35.066359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.905735 (+2839376us) service_pool.cc:224] Handling call
0709 12:25:38.083051 (+177316us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:38.068070  7370 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.796s	user 0.000s	sys 0.001s
I20260709 12:25:38.060735  7282 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:38.058578  7219 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:38.125756  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2325) took 3061 ms. Trace:
I20260709 12:25:38.164657  4750 rpcz_store.cc:276] 0709 12:25:35.064136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.024194 (+1960058us) service_pool.cc:224] Handling call
0709 12:25:37.024621 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.078518 (+ 53897us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.078562 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:37.079158 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.391768 (+312610us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:37.391808 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.391871 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.125430 (+733559us) raft_consensus.cc:1685] finished
0709 12:25:38.125460 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.125672 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":53240,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"thread_start_us":312089,"threads_started":1,"wal-append.queue_time_us":475585}
W20260709 12:25:38.126552  7173 consensus_meta.cc:287] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.681s	user 0.012s	sys 0.000s
I20260709 12:25:37.785274  7386 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:38.133030  7301 consensus_meta.cc:287] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 2.337s	user 0.006s	sys 0.002s
I20260709 12:25:38.117924  7332 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:38.177868  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 403) took 1728 ms. Trace:
I20260709 12:25:38.178329  4750 rpcz_store.cc:276] 0709 12:25:36.449099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.177041 (+1727942us) service_pool.cc:224] Handling call
0709 12:25:38.177351 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.177528 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.177551 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:38.177611 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:38.177659 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.177711 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.177812 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:38.180302  4750 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:25:38.180716  4750 pending_rounds.cc:85] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 6
I20260709 12:25:38.180984  4750 pending_rounds.cc:107] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:25:38.181279  4750 raft_consensus.cc:2889] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:38.137007  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1519) took 2026 ms. Trace:
W20260709 12:25:38.129348  7270 consensus_meta.cc:287] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.673s	user 0.009s	sys 0.000s
I20260709 12:25:38.059547  7375 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 12:25:38.095847  7246 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:38.115279  7324 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:38.201958  7324 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:38.202575  7324 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:38.114563  7361 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:38.110042  7235 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:38.119931  7329 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:38.176118  7264 consensus_meta.cc:287] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.634s	user 0.004s	sys 0.007s
I20260709 12:25:38.176558  7344 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:38.183079  4753 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:25:37.941358  6773 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:38.185969  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1902) took 2454 ms. Trace:
I20260709 12:25:38.186568  4604 rpcz_store.cc:276] 0709 12:25:36.111703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.133657 (+1021954us) service_pool.cc:224] Handling call
0709 12:25:37.139106 (+  5449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.183672 (+ 44566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.183709 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:37.184213 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.512040 (+327827us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:37.512238 (+   198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.512351 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.136797 (+624446us) raft_consensus.cc:1685] finished
0709 12:25:38.136818 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.136946 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43983,"raft.queue_time_us":137840,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":6470,"thread_start_us":231216,"threads_started":1,"wal-append.queue_time_us":587551}
I20260709 12:25:38.189342  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 404) took 1739 ms. Trace:
I20260709 12:25:38.155341  7302 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:38.153290  7097 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:38.153759  7287 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:38.136364  7405 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.158s
I20260709 12:25:38.182089  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1905) took 2855 ms. Trace:
I20260709 12:25:38.177307  7332 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.315110  4753 pending_rounds.cc:85] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 2
W20260709 12:25:38.229105  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.315086  4596 rpcz_store.cc:276] 0709 12:25:35.731741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.463637 (+731896us) service_pool.cc:224] Handling call
0709 12:25:36.519687 (+ 56050us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:36.520054 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:36.520083 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:36.536161 (+ 16078us) log.cc:844] Serialized 52 byte log entry
0709 12:25:36.927296 (+391135us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:37.888049 (+960753us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b18003dcb40. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c9522 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:25:38.012485 (+124436us) spinlock_profiling.cc:243] Waited 58.9 ms on lock 0x7b18003dcb40. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effee05e4bc 00007effeb3ac48b 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c6455 00007efff1eae3c7 00007effee4c09a7 00007effee4d00bf
0709 12:25:38.012664 (+   179us) log.cc:844] Serialized 14 byte log entry
0709 12:25:38.046208 (+ 33544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.046331 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.185688 (+139357us) raft_consensus.cc:1685] finished
0709 12:25:38.185714 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.185896 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70551,"prepare.queue_time_us":53842,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":13867,"raft.queue_time_us":54377,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":21042,"spinlock_wait_cycles":148777984,"thread_start_us":500915,"threads_started":2,"wal-append.queue_time_us":496007}
I20260709 12:25:38.335661  4753 pending_rounds.cc:107] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:25:38.321930  7173 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.336113  4753 raft_consensus.cc:2889] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:38.118911  7380 consensus_queue.cc:237] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:38.117764  7260 consensus_queue.cc:237] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:38.329805  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1907) took 2546 ms. Trace:
I20260709 12:25:38.322791  4748 rpcz_store.cc:276] 0709 12:25:35.327038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.110049 (+2783011us) service_pool.cc:224] Handling call
0709 12:25:38.152728 (+ 42679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.154052 (+  1324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.154084 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:38.154130 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:38.154172 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.181881 (+ 27709us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.182031 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":918,"raft.queue_time_us":31570,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":48511}
W20260709 12:25:38.360904  4646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.073s	sys 0.175s
W20260709 12:25:38.387861  5314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.266s	user 0.000s	sys 0.017s
W20260709 12:25:38.396782  4646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.077s	sys 0.178s
I20260709 12:25:38.314805  4750 rpcz_store.cc:276] 0709 12:25:36.450166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.179571 (+1729405us) service_pool.cc:224] Handling call
0709 12:25:38.179886 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.189017 (+  9131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.189272 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:38.336997  7285 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:38.396842  5314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.276s	user 0.000s	sys 0.017s
I20260709 12:25:38.338061  7332 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:38.396469  4597 rpcz_store.cc:276] 0709 12:25:35.784272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.535898 (+751626us) service_pool.cc:224] Handling call
0709 12:25:36.536379 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:36.536918 (+   539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:36.536959 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:36.840093 (+303134us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.130478 (+290385us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:37.224042 (+ 93564us) log.cc:844] Serialized 14 byte log entry
0709 12:25:37.402665 (+178623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.402956 (+   291us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.172820 (+769864us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b18003dc8a0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c9522 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:25:38.299086 (+126266us) raft_consensus.cc:1685] finished
0709 12:25:38.329505 (+ 30419us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.329723 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10959,"prepare.queue_time_us":179478,"prepare.run_cpu_time_us":8002,"prepare.run_wall_time_us":67776,"raft.queue_time_us":183834,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":37242,"spinlock_wait_cycles":68236032,"thread_start_us":467389,"threads_started":2,"wal-append.queue_time_us":452522}
W20260709 12:25:38.395184  5313 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.086s	user 0.045s	sys 0.103s
W20260709 12:25:38.401919  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 1.28094e+06us
W20260709 12:25:38.399185  7331 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.006s	sys 0.012s
W20260709 12:25:38.406141  5313 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.097s	user 0.045s	sys 0.103s
I20260709 12:25:38.408418  4672 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
W20260709 12:25:38.402344  4721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.003s	sys 0.015s
I20260709 12:25:38.596794  7340 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.597604  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2315) took 5500 ms. Trace:
I20260709 12:25:38.521035  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:38.597966  7340 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:38.598032  4753 rpcz_store.cc:276] 0709 12:25:33.097363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.814683 (+717320us) service_pool.cc:224] Handling call
0709 12:25:33.814955 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.336829 (+4521874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.597286 (+260457us) spinlock_profiling.cc:243] Waited 4.37 s on lock 0x7b540004cc54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:38.597496 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10044606720}
I20260709 12:25:38.477039  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:38.599615  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:38.347168  7173 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:38.570716  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:38.600220  4753 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:25:38.392645  7263 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:38.400992  7363 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.054s	sys 0.090s
W20260709 12:25:38.601066  7363 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.064s	sys 0.090s
I20260709 12:25:38.184655  7077 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:38.603724  7385 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
I20260709 12:25:38.605338  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1897) took 3541 ms. Trace:
W20260709 12:25:38.605708  7395 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.306s	user 0.000s	sys 0.000s
W20260709 12:25:38.462815  7191 consensus_meta.cc:287] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 2.017s	user 0.013s	sys 0.002s
I20260709 12:25:38.337764  7235 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:38.605724  4751 rpcz_store.cc:276] 0709 12:25:35.063386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.993926 (+1930540us) service_pool.cc:224] Handling call
0709 12:25:36.994366 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:36.994833 (+   467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:36.994858 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:36.995433 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.277128 (+281695us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:37.296158 (+ 19030us) log.cc:844] Serialized 14 byte log entry
0709 12:25:37.296367 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.296662 (+   295us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.605064 (+1308402us) raft_consensus.cc:1685] finished
0709 12:25:38.605084 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.605220 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5430,"prepare.queue_time_us":21352,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":3360,"raft.queue_time_us":211,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":10914,"thread_start_us":281172,"threads_started":1,"wal-append.queue_time_us":1251800}
W20260709 12:25:38.596112  4721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.004s	sys 0.015s
I20260709 12:25:38.608150  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:38.608516  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:38.608803  4751 raft_consensus.cc:2376] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8. Re-sending same reply.
I20260709 12:25:38.609207  4753 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:38.508631  7378 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.410109  7404 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:38.577494  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:38.587289  7325 consensus_meta.cc:287] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 2.839s	user 0.013s	sys 0.001s
I20260709 12:25:38.206306  7270 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:38.550144  4569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.389s	user 0.008s	sys 0.010s
I20260709 12:25:38.647423  4755 raft_consensus.cc:2410] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:25:38.647830  4569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.487s	user 0.008s	sys 0.011s
W20260709 12:25:38.648500  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 1.52085e+06us
I20260709 12:25:38.505363  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:38.335521  7287 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:38.550681  5312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.145s	user 0.000s	sys 0.019s
W20260709 12:25:38.652557  5312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.247s	user 0.000s	sys 0.020s
W20260709 12:25:38.653577  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 1.24842e+06us
I20260709 12:25:38.184338  7301 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:38.527628  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:38.413458  7331 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.006s	sys 0.013s
W20260709 12:25:38.549743  4645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.156s	user 0.007s	sys 0.011s
I20260709 12:25:38.338316  7097 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:38.601671  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:38.602774  7376 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:25:38.602953  7173 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:38.677181  7343 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:38.615188  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:38.678198  7336 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:38.678781  7287 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:38.681077  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2352) took 1526 ms. Trace:
I20260709 12:25:38.615725  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1899) took 3551 ms. Trace:
I20260709 12:25:38.605679  7077 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:38.576810  7286 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.156s	user 0.000s	sys 0.018s
W20260709 12:25:38.616974  7396 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:25:38.622143  7339 consensus_meta.cc:287] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 1.697s	user 0.009s	sys 0.003s
I20260709 12:25:38.656663  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:38.413106  7285 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:38.655325  5346 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:38.654488  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:38.617466  7282 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:38.662868  4645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.269s	user 0.008s	sys 0.011s
I20260709 12:25:38.682078  7301 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:38.570226  7387 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 12:25:38.667943  4751 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:38.670929  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:38.651118  7141 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:25:38.705502  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.672344  7235 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:38.652123  7346 consensus_meta.cc:287] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.728s	user 0.008s	sys 0.004s
I20260709 12:25:38.628265  7404 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:38.675011  7097 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:38.611681  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:38.682099  4756 rpcz_store.cc:276] 0709 12:25:35.063836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.005045 (+1941209us) service_pool.cc:224] Handling call
0709 12:25:37.005469 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.085940 (+ 80471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.085978 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:37.086492 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.297244 (+210752us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:37.394961 (+ 97717us) log.cc:844] Serialized 14 byte log entry
0709 12:25:37.395194 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.395286 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.615462 (+1220176us) raft_consensus.cc:1685] finished
0709 12:25:38.615491 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.615653 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17715,"prepare.queue_time_us":97526,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":12529,"raft.queue_time_us":208,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"spinlock_wait_cycles":33502592,"thread_start_us":210310,"threads_started":1,"wal-append.queue_time_us":1050228}
I20260709 12:25:38.681459  4752 rpcz_store.cc:276] 0709 12:25:37.157800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.680244 (+1522444us) service_pool.cc:224] Handling call
0709 12:25:38.680565 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.680756 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.680780 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:38.680823 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:38.680860 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.680914 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.681020 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:38.650830  7250 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.692094  5312 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:25:38.686646  7286 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.289s	user 0.001s	sys 0.018s
I20260709 12:25:38.660130  7331 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:38.694041  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 1.39343e+06us
I20260709 12:25:38.649207  4569 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:38.776619  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 127500us
I20260709 12:25:38.776928  4645 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:38.628034  7378 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:38.782895  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:38.783689  4678 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:38.786171  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:38.786677  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:38.786921  4679 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:38.787451  4676 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:38.788704  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:38.790180  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:38.793987  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:38.708393  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.715039  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2343) took 1596 ms. Trace:
I20260709 12:25:38.795630  4751 rpcz_store.cc:276] 0709 12:25:37.118721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.647212 (+1528491us) service_pool.cc:224] Handling call
0709 12:25:38.667694 (+ 20482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.714654 (+ 46960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.714960 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:38.735191  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:38.768555  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 156982us
W20260709 12:25:38.769397  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 77410.6us
I20260709 12:25:38.685945  7386 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:38.695143  7377 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:38.720718  7332 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
W20260709 12:25:38.803983  4720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.049s	sys 0.124s
W20260709 12:25:38.809144  4720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.049s	sys 0.124s
I20260709 12:25:38.799348  4678 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:38.801230  5349 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:25:38.774118  4571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.613s	user 0.015s	sys 0.010s
I20260709 12:25:38.804471  4680 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:38.779073  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:38.807474  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:38.701839  7282 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:38.693316  7339 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:38.767047  7141 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:38.809216  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:38.888029  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:38.889015  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:39.016191  7345 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.016536  5346 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:39.016819  7345 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:38.859211  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2356) took 1699 ms. Trace:
W20260709 12:25:39.019948  4755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2347) took 1898 ms (client timeout 1575 ms). Trace:
I20260709 12:25:39.019821  4751 rpcz_store.cc:276] 0709 12:25:37.159309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.796621 (+1637312us) service_pool.cc:224] Handling call
0709 12:25:38.858510 (+ 61889us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.858760 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.858797 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:38.858853 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:38.858908 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.858978 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.859132 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:39.020666  4755 rpcz_store.cc:269] 0709 12:25:37.122067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.187850 (+ 65783us) service_pool.cc:224] Handling call
0709 12:25:39.019657 (+1831807us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400263c54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:39.019838 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3350990720}
I20260709 12:25:39.022612  4677 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:39.023813  4755 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:38.850759  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:39.025004  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 1.14877e+06us
I20260709 12:25:39.025277  4674 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:25:39.026088  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 218728us
I20260709 12:25:38.767067  5342 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:39.024562  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1913) took 1806 ms. Trace:
I20260709 12:25:38.767514  7346 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:38.779119  7191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:50638 (request call id 380) took 8119 ms. Trace:
I20260709 12:25:38.829324  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:38.883328  7296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.013s	sys 0.024s
I20260709 12:25:38.774554  7389 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:39.081931  7389 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:39.071738  4750 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.019s	sys 0.022s
W20260709 12:25:39.082644  4750 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.019s	sys 0.022s
W20260709 12:25:38.775715  7236 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.504s	user 0.054s	sys 0.138s
W20260709 12:25:38.900307  7360 consensus_meta.cc:287] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.911s	user 0.011s	sys 0.003s
I20260709 12:25:39.026225  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:39.013311  7393 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.025s	sys 0.115s
W20260709 12:25:39.088061  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 61890.1us
W20260709 12:25:39.079355  7296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.000s	user 0.018s	sys 0.034s
I20260709 12:25:39.067323  5341 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:39.069221  4755 rpcz_store.cc:276] 0709 12:25:37.218512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.023253 (+1804741us) service_pool.cc:224] Handling call
0709 12:25:39.023595 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.024287 (+   692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.024497 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:38.774235  7250 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:38.805750  7416 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:39.098081  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:38.806653  7235 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:39.082406  7389 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:39.072809  7391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.025s	sys 0.050s
I20260709 12:25:39.090898  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:39.101155  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 376478us
I20260709 12:25:38.806860  7301 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.280s
I20260709 12:25:39.087343  7404 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:39.087519  7236 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.829s	user 0.064s	sys 0.163s
I20260709 12:25:39.087843  7328 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.093075  7338 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:25:39.094785  7393 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.026s	sys 0.116s
I20260709 12:25:39.106719  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:38.893543  7331 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:39.095211  7270 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:39.103895  7391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.167s	user 0.026s	sys 0.050s
I20260709 12:25:39.120489  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:39.121222  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:39.121977  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:39.122108  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:39.091115  7360 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:39.124387  7418 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:25:39.131062  5346 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:39.100404  7235 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.112973  7097 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:39.106208  7416 raft_consensus.cc:697] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:39.125157  7338 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.146375  7235 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:39.029047  4571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.868s	user 0.024s	sys 0.025s
I20260709 12:25:39.319749  7077 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.274426  4605 raft_consensus.cc:2393] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:39.169971  7282 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:39.278785  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 390768us
I20260709 12:25:38.809828  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:39.283478  4676 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:39.075984  4675 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:39.080787  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:39.286377  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:39.104991  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:39.293982  4748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.024s	sys 0.019s
W20260709 12:25:39.295121  4672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2403) took 3871 ms (client timeout 2237 ms). Trace:
I20260709 12:25:39.295338  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 405) took 2835 ms. Trace:
I20260709 12:25:39.097216  4751 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:39.297019  4749 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:39.297677  4754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.386s	user 0.032s	sys 0.041s
I20260709 12:25:39.297765  4719 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:39.123803  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:39.122819  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:39.176481  7250 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:39.305761  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:39.307083  6788 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:39.307421  7285 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:39.307874  5342 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:39.398056  7264 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:39.242448  7389 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:39.203641  7416 consensus_queue.cc:237] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:39.145294  7097 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:39.449743  7167 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:39.450526  7097 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:39.318668  7173 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.454165  7173 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:39.456362  4605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1508) took 3814 ms (client timeout 1894 ms). Trace:
W20260709 12:25:39.459100  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 161867us
I20260709 12:25:39.461769  7282 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:39.464104  7422 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:39.131366  7328 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:39.464637  7422 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.465013  4602 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:39.465188  7422 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.168587  7332 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:39.152508  7338 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:39.467808  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1912) took 3680 ms. Trace:
W20260709 12:25:39.276300  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 467020us
I20260709 12:25:39.236405  7339 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:39.469983  7285 raft_consensus.cc:1064] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:39.229769  7340 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:39.470607  7285 raft_consensus.cc:1068] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Unable to promote non-voter dbe960c35c84451faf4e5b5c9233fa39: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:25:39.200475  7345 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:39.175174  7347 consensus_meta.cc:287] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 2.251s	user 0.007s	sys 0.007s
I20260709 12:25:39.474844  7347 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:39.440037  6788 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:39.476754  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1909) took 2260 ms. Trace:
W20260709 12:25:39.234599  7348 consensus_meta.cc:287] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 1.322s	user 0.010s	sys 0.003s
I20260709 12:25:39.477399  6788 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:39.478221  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:39.478440  7348 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:39.075464  7191 rpcz_store.cc:276] 0709 12:25:30.667691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.674923 (+  7232us) service_pool.cc:224] Handling call
0709 12:25:31.149702 (+474779us) ts_tablet_manager.cc:934] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
0709 12:25:32.332481 (+1182779us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b4c000d36c0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effec782a20 00007effec793a29 00007effec79413f 00007effec7582ab 00007effec785f3c 00007effec786461 00007effec786768 00007efff250c2eb 00007efff3431d3b 00007efff342c008 00007efff342b405
0709 12:25:33.930939 (+1598458us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:25:35.145644 (+1214705us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:36.345752 (+1200108us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:36.385689 (+ 39937us) tablet_copy_client.cc:463] Replaced tombstoned tablet metadata.
0709 12:25:38.618874 (+2233185us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:25:38.685282 (+ 66408us) tablet_replica.cc:206] Creating consensus instance
0709 12:25:38.766954 (+ 81672us) ts_tablet_manager.cc:989] Replica registered.
0709 12:25:38.774232 (+  7278us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":131749,"spinlock_wait_cycles":73585408,"tablet-copy.queue_time_us":184591,"thread_start_us":271113,"threads_started":2}
I20260709 12:25:39.468180  4602 rpcz_store.cc:276] 0709 12:25:35.787325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.032550 (+1245225us) service_pool.cc:224] Handling call
0709 12:25:39.464659 (+2432109us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.465478 (+   819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.467512 (+  2034us) spinlock_profiling.cc:243] Waited 2.43 s on lock 0x7b5400060750. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:39.467711 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5592081792}
I20260709 12:25:39.149104  7353 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:39.225798  7360 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:39.184090  7378 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:39.233757  7390 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.744s	user 0.010s	sys 0.000s
W20260709 12:25:39.521062  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 415978us
I20260709 12:25:39.168035  7412 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:39.522588  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:39.523816  7360 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:39.165833  7415 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:39.147745  7260 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.035s	sys 0.043s
I20260709 12:25:39.406253  4677 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:39.401141  5314 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:39.418241  4748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.024s	sys 0.019s
I20260709 12:25:39.422053  7270 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:39.320865  7077 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:39.425981  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 2.28974e+06us
I20260709 12:25:39.426930  4673 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
W20260709 12:25:39.420356  4569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.009s	sys 0.007s
W20260709 12:25:39.429931  4754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.518s	user 0.032s	sys 0.042s
I20260709 12:25:39.431299  7141 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:39.238399  7384 consensus_meta.cc:287] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.343s	user 0.015s	sys 0.000s
I20260709 12:25:39.443449  7264 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:39.563786  7264 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:39.564656  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:39.445267  5344 raft_consensus.cc:2376] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7. Re-sending same reply.
I20260709 12:25:39.445806  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:39.410238  4672 rpcz_store.cc:269] 0709 12:25:35.423336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.572134 (+148798us) service_pool.cc:224] Handling call
0709 12:25:39.294523 (+3722389us) spinlock_profiling.cc:243] Waited 2.69 s on lock 0x7b540017d754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:39.294819 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6187032960}
I20260709 12:25:39.429352  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:39.205694  7346 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:39.484966  7191 tablet_copy_client.cc:806] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
W20260709 12:25:39.456943  4605 rpcz_store.cc:269] 0709 12:25:35.642159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.642248 (+    89us) service_pool.cc:224] Handling call
0709 12:25:39.455965 (+3813717us) spinlock_profiling.cc:243] Waited 3.53 s on lock 0x7b5400060754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:39.456164 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8124535680}
I20260709 12:25:39.477221  4751 rpcz_store.cc:276] 0709 12:25:37.216582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.021664 (+1805082us) service_pool.cc:224] Handling call
0709 12:25:39.075228 (+ 53564us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.474789 (+399561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.476280 (+  1491us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b54001f1154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:39.476562 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49164544}
I20260709 12:25:39.506601  5349 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:39.508235  7389 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:39.508764  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.534166  7380 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:39.457156  7250 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:39.591946  7380 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.592455  7380 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.592281  7250 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:39.595005  7250 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:39.534307  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 407) took 2584 ms. Trace:
I20260709 12:25:39.595733  4748 rpcz_store.cc:276] 0709 12:25:36.949970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.505237 (+1555267us) service_pool.cc:224] Handling call
0709 12:25:38.564135 (+ 58898us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.590062 (+ 25927us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.590095 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.593227 (+  3132us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.533696 (+940469us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:39.533798 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.533929 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.534010 (+    81us) raft_consensus.cc:1685] finished
0709 12:25:39.534020 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.534226 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15032,"raft.queue_time_us":27992,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":21108,"thread_start_us":865540,"threads_started":1,"wal-append.queue_time_us":550530}
I20260709 12:25:39.534648  7270 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:39.537102  7077 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:39.531828  4598 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:39.567759  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:25:39.533085  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 219803us
I20260709 12:25:39.525921  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:39.533519  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:39.526520  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1902) took 4258 ms. Trace:
W20260709 12:25:39.550554  4569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.010s	sys 0.007s
I20260709 12:25:39.524657  7412 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:25:39.531077  7260 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.036s	sys 0.043s
I20260709 12:25:39.515055  7348 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:39.410578  4750 rpcz_store.cc:276] 0709 12:25:36.459637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.398473 (+1938836us) service_pool.cc:224] Handling call
0709 12:25:38.398796 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.403452 (+  4656us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.403484 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.403921 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.083108 (+679187us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:39.083159 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.083239 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.294577 (+211338us) raft_consensus.cc:1685] finished
0709 12:25:39.294623 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.295042 (+   419us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10201,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":71535,"thread_start_us":562187,"threads_started":1,"wal-append.queue_time_us":716824}
I20260709 12:25:39.508687  7421 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:39.532388  5345 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:39.571987  7191 tablet_copy_client.cc:670] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:39.573596  4568 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:39.574287  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:39.574510  7141 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:39.574307  6915 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.575511  7353 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.531625  7249 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:39.675606  4752 raft_consensus.cc:2368] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5.
I20260709 12:25:39.578596  7347 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:39.570421  4570 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.006s	sys 0.005s
I20260709 12:25:39.585865  5347 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:39.559616  7173 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:39.493144  7167 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:25:39.585861  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:39.556277  7338 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:39.705511  7328 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:39.565378  6773 consensus_meta.cc:287] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.237s	user 0.012s	sys 0.008s
I20260709 12:25:39.565462  7264 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:39.709030  4598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1509) took 4066 ms (client timeout 1594 ms). Trace:
W20260709 12:25:39.709587  4598 rpcz_store.cc:269] 0709 12:25:35.642444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.642592 (+   148us) service_pool.cc:224] Handling call
0709 12:25:39.708731 (+4066139us) spinlock_profiling.cc:243] Waited 3.72 s on lock 0x7b54002e4ad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:39.708948 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8562611072}
I20260709 12:25:39.616108  4749 rpcz_store.cc:276] 0709 12:25:35.267643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.910582 (+2642939us) service_pool.cc:224] Handling call
0709 12:25:39.021945 (+1111363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.437637 (+415692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.525425 (+ 87788us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b5400263c54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:39.526113 (+   688us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400263c50. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:39.526379 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3185209984}
W20260709 12:25:39.652608  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 85084.4us
I20260709 12:25:39.614089  7270 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.715458  7235 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:39.679584  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:39.718031  4748 raft_consensus.cc:2376] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8. Re-sending same reply.
I20260709 12:25:39.718495  7338 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:39.682211  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 184637us
W20260709 12:25:39.699019  4570 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.007s	sys 0.006s
I20260709 12:25:39.653250  7412 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.707728  6773 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:39.699736  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 174013us
I20260709 12:25:39.667106  7421 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:25:39.748420  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.735311  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:39.752841  4751 raft_consensus.cc:2368] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10.
I20260709 12:25:39.759938  4599 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:39.760676  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:39.743090  7421 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:39.768999  4647 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:39.768918  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2342) took 4700 ms. Trace:
W20260709 12:25:39.757901  7324 consensus_meta.cc:287] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.347s	user 0.010s	sys 0.018s
W20260709 12:25:39.770764  4571 connection.cc:256] RPC call timeout handler was delayed by -2.4267816820001826s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
W20260709 12:25:39.781688  4571 connection.cc:256] RPC call timeout handler was delayed by -2.3844816820001142s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 12:25:39.782091  4571 connection.cc:256] RPC call timeout handler was delayed by -2.3565816820000691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
W20260709 12:25:39.782405  4571 connection.cc:256] RPC call timeout handler was delayed by -2.3412816820000444s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 12:25:39.782708  4571 connection.cc:256] RPC call timeout handler was delayed by -2.3070816819999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16610000000000003s for a response
W20260709 12:25:39.783104  4571 connection.cc:256] RPC call timeout handler was delayed by -2.2206816820003041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 12:25:39.786978  7440 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:25:39.787096  6773 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:39.788100  7415 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.788741  7415 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:39.789731  7331 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:39.783891  7384 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:39.768661  7235 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:39.791349  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 43932.4us
I20260709 12:25:39.776283  7173 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:39.758639  7422 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:39.773082  4754 rpcz_store.cc:276] 0709 12:25:35.068500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.910463 (+2841963us) service_pool.cc:224] Handling call
0709 12:25:37.910742 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.911052 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.911077 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:37.911511 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.590228 (+1678717us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:39.672576 (+ 82348us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.726421 (+ 53845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.750630 (+ 24209us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.750675 (+    45us) raft_consensus.cc:1685] finished
0709 12:25:39.750695 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.768843 (+ 18148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29874,"raft.queue_time_us":212958,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":36534,"spinlock_wait_cycles":174336,"thread_start_us":1720015,"threads_started":2,"wal-append.queue_time_us":1483310}
I20260709 12:25:39.780326  7338 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:39.819891  4747 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:39.821017  7084 consensus_meta.cc:287] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.596s	user 0.011s	sys 0.011s
I20260709 12:25:39.821535  7084 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:39.822968  4756 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:39.823956  7084 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:39.824285  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:39.825558  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 252100us
I20260709 12:25:39.808698  7173 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.774912  6915 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:39.818101  7338 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:25:39.837221  7423 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
W20260709 12:25:39.849828  4571 connection.cc:256] RPC call timeout handler was delayed by -2.0512481149999076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
W20260709 12:25:39.853245  4571 connection.cc:256] RPC call timeout handler was delayed by -1.766181682000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2262s for a response
I20260709 12:25:39.784245  7324 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:39.851974  7328 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:25:39.855692  4569 connection.cc:256] RPC call timeout handler was delayed by -2.3091655920002268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2237s for a response
I20260709 12:25:39.841737  7173 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:39.860545  7249 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:39.861076  7384 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:39.861191  7249 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:39.860562  7328 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:39.860553  7348 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:39.862283  7348 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.862717  7348 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:39.863885  7328 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:39.861656  7249 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:39.877496  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:39.878490  4751 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:39.879074  4753 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:25:39.892148  4571 connection.cc:256] RPC call timeout handler was delayed by -1.1547407399998519s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
W20260709 12:25:39.892650  4571 connection.cc:256] RPC call timeout handler was delayed by -1.0674407399997108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
I20260709 12:25:39.854235  7338 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:39.895345  7433 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:39.898976  4748 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:39.846861  7347 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:25:39.903399  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 79272.9us
I20260709 12:25:39.912678  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1907) took 4520 ms. Trace:
I20260709 12:25:39.847713  7084 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:39.913115  4747 rpcz_store.cc:276] 0709 12:25:35.391928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.154005 (+2762077us) service_pool.cc:224] Handling call
0709 12:25:38.176689 (+ 22684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.820481 (+1643792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.912423 (+ 91942us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54000735d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:39.912610 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3714750208}
I20260709 12:25:39.927693  4751 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:25:39.923837  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1906) took 4158 ms. Trace:
I20260709 12:25:39.929139  4599 rpcz_store.cc:276] 0709 12:25:35.757985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.462910 (+704925us) service_pool.cc:224] Handling call
0709 12:25:39.750294 (+3287384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.775076 (+ 24782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.914156 (+139080us) spinlock_profiling.cc:243] Waited 3.26 s on lock 0x7b54002e4ad0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:39.916138 (+  1982us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7488677760}
I20260709 12:25:39.892601  7141 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:39.937006  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 181171us
I20260709 12:25:39.943379  7416 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:25:39.946842  7347 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:39.947086  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:39.948525  7345 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:39.949098  7345 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:39.950619  7345 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:39.951650  6915 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:39.953749  7336 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.954555  7336 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:39.956964  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:39.959657  4750 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:39.959882  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:39.961654  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:39.962380  4752 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:39.962311  7141 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:39.963433  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:39.964046  4752 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
W20260709 12:25:39.964432  7444 log.cc:927] Time spent T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:25:39.965983  7331 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:39.966241  7412 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:39.967013  4747 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:39.968143  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:39.968808  4753 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
W20260709 12:25:39.971166  7459 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:25:39.960806  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:39.949179  4571 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:39.904373  7328 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:39.932192  7384 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:39.949407  7347 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:40.022341  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 73274.6us
I20260709 12:25:39.968860  7412 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:40.021678  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:39.976286  7331 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:40.019285  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:40.031291  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:40.034121  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2353) took 2875 ms. Trace:
I20260709 12:25:40.038990  4750 raft_consensus.cc:2376] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5. Re-sending same reply.
I20260709 12:25:40.041148  7141 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:40.034996  4756 rpcz_store.cc:276] 0709 12:25:37.158287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.779767 (+1621480us) service_pool.cc:224] Handling call
0709 12:25:38.801360 (+ 21593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.823359 (+1021999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.995452 (+172093us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001f1654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:40.034029 (+ 38577us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2348227456}
I20260709 12:25:40.042320  7324 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:40.040117  7336 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:40.044689  7380 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.045691  7339 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:40.046191  7339 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.046620  7339 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:40.047855  7380 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:40.051257  4720 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:40.055931  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 99118.5us
W20260709 12:25:40.064276  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.065418  4748 raft_consensus.cc:2393] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:40.071242  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:40.080597  4647 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:40.070382  7097 consensus_meta.cc:287] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.007s
I20260709 12:25:40.067955  7324 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.072060  7380 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:40.085987  4749 raft_consensus.cc:2376] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7. Re-sending same reply.
I20260709 12:25:40.081065  7422 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:40.098717  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.120426  4748 raft_consensus.cc:2393] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:40.089686  7191 tablet_copy_client.cc:538] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:25:40.099818  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.106168  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:40.107791  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:40.108690  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.085098  7336 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:40.122215  7336 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.122840  4756 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:40.124701  7336 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:40.126472  7348 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.129146  7348 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:40.113608  7340 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:40.114447  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 44005.7us
I20260709 12:25:40.114482  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:40.115090  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:40.115903  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:40.116619  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:40.118631  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 67462us
I20260709 12:25:40.118939  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:40.068216  7384 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.104708  7338 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.104521  7331 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.132988  5349 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:40.133003  5347 raft_consensus.cc:2368] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5.
I20260709 12:25:40.072273  7328 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.096225  7084 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:40.152064  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:40.135413  7353 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:40.165239  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 88944.2us
I20260709 12:25:40.167006  4571 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:40.167425  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:40.088603  7097 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.158725  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:40.174757  4749 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:40.142570  7380 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:40.178817  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:40.179575  7461 log.cc:927] Time spent T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:25:40.179778  4674 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:40.180766  4680 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:40.182675  4599 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:40.143507  7416 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:40.200696  7416 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.202973  7384 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:40.204897  7331 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:40.142979  7383 consensus_queue.cc:1059] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:25:40.205902  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:40.206355  7383 raft_consensus.cc:3055] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:40.206746  7383 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:40.207280  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:40.207892  4677 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:40.148142  7415 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.133067  7249 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.208827  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:40.209089  5347 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
W20260709 12:25:40.210355  7455 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:25:40.211102  7301 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:40.212077  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:40.213016  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 59711.7us
I20260709 12:25:40.213251  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:40.213522  7345 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:40.214005  7345 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.214372  7345 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:40.214478  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:40.215755  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:40.216312  4672 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:40.216898  6915 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:40.217478  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:40.217813  7141 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:40.218353  5341 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:40.220351  6915 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:40.220726  6915 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:40.211342  7328 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:40.212230  7301 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:40.184881  7339 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.227855  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:40.231261  4753 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:40.232306  4754 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:40.231017  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:40.154279  7324 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.245555  4599 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:40.246930  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:40.251364  4676 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:40.226423  7301 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:40.257620  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 118199us
I20260709 12:25:40.232091  7084 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:40.261806  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:40.271813  7301 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.273800  7301 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:40.275238  7270 consensus_meta.cc:287] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.003s
I20260709 12:25:40.280356  7270 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.281054  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:40.281625  7084 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.255074  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:40.282505  7270 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:40.283501  4679 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:40.283800  7084 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:40.284729  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:40.285972  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:40.288154  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:40.290025  5349 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:40.290951  5343 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:40.293545  7324 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:40.294045  7339 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:40.295866  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:40.296548  4598 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:40.297410  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:40.297838  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:40.298120  4596 raft_consensus.cc:2376] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10. Re-sending same reply.
I20260709 12:25:40.298237  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:40.298565  4605 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:40.299166  4597 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:40.296555  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:40.300520  4602 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:40.300652  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:40.302659  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:40.271684  7380 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.303546  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:40.311338  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 43569.3us
I20260709 12:25:40.207648  7383 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.313244  4751 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:40.314025  4749 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:40.320304  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:40.289085  7173 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.321487  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:40.277313  7097 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:40.332000  5348 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:40.310246  7338 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:40.342854  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:40.345583  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 129776us
I20260709 12:25:40.346197  4604 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:40.356789  4605 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:40.320773  7445 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:40.358369  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 116477us
I20260709 12:25:40.356868  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:40.359228  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:40.359728  5348 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:40.359840  7173 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:40.360520  7173 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:40.363395  7339 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.364223  7328 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:40.365622  7347 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:40.365990  7347 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:40.366269  7421 consensus_meta.cc:287] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.592s	user 0.007s	sys 0.006s
I20260709 12:25:40.366264  4571 leader_election.cc:400] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5.
I20260709 12:25:40.366946  4571 leader_election.cc:403] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:25:40.364059  7339 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.372766  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:40.373057  4571 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:40.374414  7328 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5.
I20260709 12:25:40.374230  7383 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:40.374317  7416 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:40.389225  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:40.375226  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:40.400527  4568 leader_election.cc:400] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5.
I20260709 12:25:40.402114  4568 leader_election.cc:403] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:40.404784  7468 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:25:40.405196  4568 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:40.386252  7097 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.410838  7097 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.413035  7097 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:40.401121  4569 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:40.366966  7421 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.423615  7331 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:40.424647  7331 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:40.425002  7331 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:40.426102  7331 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.427969  7331 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:40.376976  6915 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.392140  7348 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:40.431579  7097 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:40.432343  7339 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:40.437363  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:40.439672  7415 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:40.441280  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:40.441835  4605 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:40.443871  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:40.444588  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 48917.2us
I20260709 12:25:40.475934  4603 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:40.443961  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:40.449795  7340 consensus_queue.cc:1059] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:40.450373  7353 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:25:40.496591  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 91488.4us
I20260709 12:25:40.476753  4601 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:40.454460  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:40.467921  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:40.472733  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:40.366966  7347 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.498785  4676 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.496742  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:40.503170  7097 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.503839  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.504546  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.505033  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:40.505918  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.506342  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.506645  4749 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:40.506985  4755 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:40.500619  7336 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:25:40.504539  4747 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:40.507566  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.506253  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:40.508280  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.508886  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.509546  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.510238  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:40.512049  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:40.515005  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 129150us
W20260709 12:25:40.517096  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.491163  7328 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:25:40.521365  4750 raft_consensus.cc:2376] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4. Re-sending same reply.
I20260709 12:25:40.532084  4603 raft_consensus.cc:2393] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 7: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:40.532830  4597 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.500089  7353 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:40.534307  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 133258us
I20260709 12:25:40.534579  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.535224  4755 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:40.499771  7348 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:40.536957  4756 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:40.515915  7339 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:40.515949  7336 consensus_queue.cc:1243] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:25:40.497315  7270 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:25:40.497229  7338 consensus_queue.cc:1059] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:25:40.547557  4600 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:25:40.553768  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.531895  7097 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:40.556344  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:40.512571  7472 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:25:40.557442  4747 raft_consensus.cc:2376] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10. Re-sending same reply.
I20260709 12:25:40.576556  4644 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:40.582178  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 209189us
I20260709 12:25:40.545171  7416 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:40.596336  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 51739.4us
I20260709 12:25:40.597723  4680 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:25:40.599663  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.600330  4747 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:40.565621  6915 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:40.560652  7421 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:40.610888  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.576095  7347 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:40.616200  4571 leader_election.cc:400] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10.
I20260709 12:25:40.615797  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:40.601902  7472 raft_consensus.cc:3055] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:40.621269  7472 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:40.622210  7472 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.542101  7340 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:25:40.641798  7338 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:40.642622  7338 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.646306  7348 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.654968  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:40.655946  7270 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:40.656745  7270 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.659382  7270 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:40.660885  7340 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:40.661355  7340 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:40.593740  7416 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:40.661926  7416 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.665861  4569 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:40.667197  7476 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:40.667325  4569 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:40.668696  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:40.668790  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:40.672638  7249 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:40.645304  7338 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:40.634351  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:40.634776  7472 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:40.632009  4747 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:40.691015  4676 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:40.698037  4597 raft_consensus.cc:2410] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:25:40.701915  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:40.702273  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 183946us
I20260709 12:25:40.706846  7191 tablet_bootstrap.cc:492] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:25:40.709791  7332 raft_consensus.cc:2764] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:40.597898  7345 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.713073  7328 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5.
I20260709 12:25:40.713555  7345 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:40.714277  7345 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:40.723659  4602 raft_consensus.cc:2368] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8.
I20260709 12:25:40.727761  4747 raft_consensus.cc:2376] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10. Re-sending same reply.
I20260709 12:25:40.733338  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:40.733685  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:40.734258  4600 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 6: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:40.734555  4754 raft_consensus.cc:2393] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:40.735469  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:40.736208  4754 raft_consensus.cc:2393] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:40.736537  4600 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:40.736951  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.737386  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:40.739171  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:40.740129  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 141803us
I20260709 12:25:40.750653  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.754305  4605 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:40.743541  4754 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:40.756533  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:40.757552  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:40.758901  7270 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.759173  7260 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:40.759557  7270 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.769100  7260 raft_consensus.cc:3055] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:40.769659  7260 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:40.770884  7270 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:40.770717  7260 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:40.771991  7260 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:40.756603  7476 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:40.774933  7375 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:40.775524  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:40.777170  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.761595  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:40.779675  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:40.780435  4754 raft_consensus.cc:2368] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8.
I20260709 12:25:40.781100  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:40.783528  4600 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:40.777962  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:40.785595  4597 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:40.776227  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:40.787225  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:40.781164  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:40.814698  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 53840.5us
I20260709 12:25:40.815519  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:40.816309  7478 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:40.823370  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:40.824172  4747 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:40.824577  5347 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
W20260709 12:25:40.825158  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 268743us
I20260709 12:25:40.776883  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:40.786595  7173 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:40.827778  4569 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:40.839846  4747 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:40.840358  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 63259.7us
W20260709 12:25:40.842551  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 66430.1us
I20260709 12:25:40.843372  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:40.842995  4644 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:40.822856  7421 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.790334  7328 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:25:40.841681  5349 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:39524
W20260709 12:25:40.863746  7301 consensus_meta.cc:287] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.577s	user 0.011s	sys 0.000s
I20260709 12:25:40.864538  7301 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:40.865622  7485 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:40.865804  7479 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:40.872742  7270 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:40.874624  7270 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:40.875015  7270 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.875510  7270 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.841885  7471 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:40.880195  7471 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:40.880702  7471 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:40.881747  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 59261.3us
I20260709 12:25:40.884596  4599 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:40.854799  5349 raft_consensus.cc:606] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:40.896289  7347 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:40.896796  7347 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.897146  7347 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.909718  6915 consensus_queue.cc:1059] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:40.910786  7328 raft_consensus.cc:3055] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:40.911747  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:40.869390  7421 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:40.928261  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:25:40.935072  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 91955.5us
I20260709 12:25:40.911185  7328 raft_consensus.cc:740] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:40.947221  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:40.958457  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 115074us
I20260709 12:25:40.953613  6915 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:40.963114  7173 raft_consensus.cc:1064] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:40.964116  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.964905  4747 raft_consensus.cc:2393] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:40.965660  7202 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 12:25:40.966560  7433 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:40.970480  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:40.945317  7490 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:40.973335  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:40.983343  4747 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:40.984591  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:40.986197  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.960486  7348 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:40.954476  7482 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
W20260709 12:25:41.002269  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 226819us
I20260709 12:25:41.003356  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:41.003890  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 68052.4us
I20260709 12:25:41.024794  4747 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
W20260709 12:25:41.025900  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 65011.1us
I20260709 12:25:41.029134  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:41.030179  4747 raft_consensus.cc:2393] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 4: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:40.997164  7339 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:41.032068  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 44612.7us
I20260709 12:25:41.035804  4647 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:41.036324  4749 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:41.037142  7249 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:41.037529  7249 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:41.037774  4749 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:41.037990  7249 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:41.033314  7348 consensus_queue.cc:1059] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:25:41.040781  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:40.985773  7353 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:41.042742  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:41.044313  4571 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:41.048877  4749 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:41.054203  7302 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:25:41.055383  7302 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:41.055466  4571 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:41.055931  7302 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:41.056720  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:41.054215  7141 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:41.057524  7141 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:41.057925  7141 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:41.057821  7479 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:41.058279  7482 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:41.058773  7479 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:41.059034  7482 raft_consensus.cc:3055] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:41.059299  7479 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:41.059396  7384 raft_consensus.cc:3055] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:41.059851  7384 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:41.060410  7479 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:41.060786  7384 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:41.055661  7380 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:25:41.063337  7491 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:41.053709  4747 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:41.071282  7421 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:41.072151  4677 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:41.059468  7482 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:41.050115  7493 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:41.087917  7380 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:41.094877  7493 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:41.095336  7493 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:41.095413  7380 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:41.095943  7467 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:41.096961  7380 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:41.100355  4755 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:41.100693  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:40.968575  7173 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:41.101370  4752 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:41.085402  7494 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:41.086971  5348 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:41.086647  7270 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:41.087000  4672 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:25:41.071305  4756 pending_rounds.cc:85] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 1
I20260709 12:25:41.056891  7302 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:41.090858  7332 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.092321  4571 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:41.110765  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.074154  7141 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:41.105078  4756 pending_rounds.cc:107] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:25:41.117141  4756 raft_consensus.cc:2889] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:41.113045  7332 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:41.119042  7270 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:41.119788  7332 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:41.116043  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:41.128310  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:41.128997  5348 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:41.129805  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:41.105965  7494 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:41.109054  7467 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:25:41.112975  7384 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:41.266289  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:41.249404  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:41.411651  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 230218us
W20260709 12:25:41.217140  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 143096us
I20260709 12:25:41.414678  4752 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:41.181037  4747 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:41.419116  7494 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:41.421939  5348 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:41.122179  7353 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:41.423904  4569 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:25:41.373674  7499 log.cc:927] Time spent T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.271s	user 0.001s	sys 0.001s
W20260709 12:25:41.393373  7474 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.800s	user 0.005s	sys 0.000s
I20260709 12:25:41.172509  7097 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:41.390205  7472 consensus_meta.cc:287] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.689s	user 0.007s	sys 0.004s
I20260709 12:25:41.417380  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:41.418298  7141 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:41.174594  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:41.444124  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:41.439272  4720 connection.cc:256] RPC call timeout handler was delayed by -1.7921224850001636s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3365s for a response
I20260709 12:25:41.444165  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:41.445489  4720 connection.cc:256] RPC call timeout handler was delayed by -1.583931795999888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 12:25:41.444310  4603 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
W20260709 12:25:41.446228  4720 connection.cc:256] RPC call timeout handler was delayed by -1.5686317959998632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1563s for a response
W20260709 12:25:41.446714  4720 connection.cc:256] RPC call timeout handler was delayed by -1.3922317960000328s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
I20260709 12:25:41.134794  7332 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.297s
I20260709 12:25:41.443513  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:41.217837  7471 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:41.134977  7347 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:41.244164  7461 log.cc:927] Time spent T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 12:25:41.459774  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.434859  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 342572us
I20260709 12:25:41.173619  7338 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:41.280011  7416 consensus_meta.cc:287] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.618s	user 0.006s	sys 0.006s
I20260709 12:25:41.469880  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:41.482010  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:41.482223  7498 log.cc:927] Time spent T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 12:25:41.482841  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:41.432548  7097 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:41.430781  7249 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:41.494625  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:41.495311  4747 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 4: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:41.497637  7476 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:41.498577  7347 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:41.471566  7332 consensus_queue.cc:1243] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:25:41.499652  7331 consensus_meta.cc:287] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.028s	user 0.009s	sys 0.004s
I20260709 12:25:41.503084  4499 catalog_manager.cc:5697] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:41.506515  4680 raft_consensus.cc:2393] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 4: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:41.507048  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:41.521080  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.521705  4498 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52808 (request call id 275) took 14579 ms (client timeout 15000 ms). Trace:
I20260709 12:25:41.521970  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 425) took 1355 ms. Trace:
I20260709 12:25:41.510253  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:41.511250  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:41.514451  4674 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:41.523833  4748 rpcz_store.cc:276] 0709 12:25:40.166430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.166599 (+   169us) service_pool.cc:224] Handling call
0709 12:25:40.230204 (+ 63605us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.242559 (+ 12355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.242593 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:40.252556 (+  9963us) log.cc:844] Serialized 26 byte log entry
0709 12:25:40.351172 (+ 98616us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:40.351238 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.351344 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.141708 (+790364us) spinlock_profiling.cc:243] Waited 8.67 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:41.521657 (+379949us) raft_consensus.cc:1685] finished
0709 12:25:41.521680 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.521898 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6180,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":6066,"spinlock_wait_cycles":19930368,"thread_start_us":35580,"threads_started":1,"wal-append.queue_time_us":257136}
W20260709 12:25:41.515921  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 92142.7us
I20260709 12:25:41.517280  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:41.497408  7505 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:41.508697  7416 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:41.527359  4747 raft_consensus.cc:2393] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 4: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:41.523209  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:41.528822  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:41.527314  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:41.529736  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:41.530350  4752 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:25:41.533512  4747 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:41.541777  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:41.542104  7097 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:41.542660  4747 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:41.542902  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:41.543660  4750 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:41.543831  4747 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:25:41.546010  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:41.546635  7495 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.508s	user 0.003s	sys 0.002s
I20260709 12:25:41.547501  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:41.548300  4679 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:41.549141  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:41.549559  4499 catalog_manager.cc:5697] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:41.551463  4499 catalog_manager.cc:5697] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 6, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 6 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:41.552943  4749 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:41.512893  7338 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:41.555303  7416 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:41.555511  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:41.572656  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 405) took 1118 ms. Trace:
W20260709 12:25:41.558269  7469 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.956s	user 0.002s	sys 0.000s
I20260709 12:25:41.558625  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:41.560343  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.561375  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.562122  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:41.563056  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.563560  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.575071  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:41.563755  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:41.564831  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.565975  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.567665  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:41.556424  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 1942) took 1153 ms. Trace:
I20260709 12:25:41.549823  7328 consensus_queue.cc:260] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:41.602300  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:41.617168  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.617439  4600 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:41.557650  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:41.580392  4679 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:41.567030  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:41.598420  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.601358  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:41.604404  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 85301.9us
W20260709 12:25:41.608832  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 81599.7us
I20260709 12:25:41.613831  4602 rpcz_store.cc:276] 0709 12:25:40.403309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.419977 (+ 16668us) service_pool.cc:224] Handling call
0709 12:25:41.556185 (+1136208us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b540013e7d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:41.556333 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":499360000}
I20260709 12:25:41.621884  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:41.601054  4674 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:41.601738  4605 rpcz_store.cc:276] 0709 12:25:40.454373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.491039 (+ 36666us) service_pool.cc:224] Handling call
0709 12:25:41.572425 (+1081386us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540004f454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:41.572542 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":576876288}
I20260709 12:25:41.580888  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:41.623275  4603 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:41.558763  7167 raft_consensus.cc:3055] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:41.624940  4680 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:41.559046  7505 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:41.641722  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:41.642704  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.651252  4605 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:41.651793  4604 raft_consensus.cc:2410] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:25:41.642287  4748 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:25:41.642062  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:41.613171  4678 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:41.619530  4499 catalog_manager.cc:5697] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 6 to 8, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 8 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:41.642800  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:41.653445  5312 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:41.628504  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.647636  4676 raft_consensus.cc:2393] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:41.654850  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:41.628984  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:41.655938  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.658043  4601 raft_consensus.cc:3055] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:41.659394  4752 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:41.639564  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:41.660115  4674 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:41.661340  4681 raft_consensus.cc:2410] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:25:41.658442  4601 raft_consensus.cc:740] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:41.607807  7097 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:41.666827  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:41.667683  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:41.669014  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.673512  4597 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:41.675288  5343 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:41.674912  4499 catalog_manager.cc:5697] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index -1 to 3, term changed from 6 to 7, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added, no longer has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 7 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:41.662706  7332 consensus_queue.cc:1059] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:25:41.678138  4499 catalog_manager.cc:5697] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:41.679601  4499 catalog_manager.cc:5697] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 2 to 3, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 3 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:41.681622  4499 catalog_manager.cc:5697] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 6, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 6 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:41.681922  7505 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:41.683115  4499 catalog_manager.cc:5697] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 3 to 4, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 4 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:41.684736  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:41.685546  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:41.686151  4598 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:25:41.685428  4499 catalog_manager.cc:5697] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:41.688606  4499 catalog_manager.cc:5697] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 3 to 5, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 5 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:41.692041  7511 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:41.692538  7511 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.692898  7511 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:41.699692  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:41.700585  4677 raft_consensus.cc:2410] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:41.702282  6915 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:41.702785  6915 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:41.705211  7471 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:41.710430  4600 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:41.713294  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.709082  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.598322  7494 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:41.607803  7415 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:25:41.650552  7167 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:41.723332  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:41.724466  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 102640us
I20260709 12:25:41.719867  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:41.726891  4645 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:41.714972  4601 consensus_queue.cc:260] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:41.735129  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:41.736397  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 71796.3us
I20260709 12:25:41.737926  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:41.671093  7202 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:41.742009  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:41.743914  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:41.745478  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:41.741739  5345 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:41.749994  4679 raft_consensus.cc:2410] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:25:41.690703  7353 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:25:41.759338  7301 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:41.759301  7167 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:41.759652  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 96258.8us
I20260709 12:25:41.760496  7141 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:41.761874  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.750716  7339 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:41.786598  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 45383.3us
W20260709 12:25:41.731330  7493 consensus_meta.cc:287] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.002s
I20260709 12:25:41.776584  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:41.729067  7501 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:41.749797  7336 raft_consensus.cc:3055] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:41.776508  4755 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:41.747886  7345 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:41.816872  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:41.791968  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 107345us
W20260709 12:25:41.794338  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 45555.3us
I20260709 12:25:41.818797  7496 raft_consensus.cc:3055] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:41.789316  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:41.819291  7496 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:41.753909  7502 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:41.801057  4598 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:41.820103  7502 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.820499  7502 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:41.820878  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.821687  7510 raft_consensus.cc:2764] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:25:41.801237  4645 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:41.822854  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:41.823686  4645 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:41.825847  7488 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:41.826841  7332 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:41.826924  7249 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:41.827260  7332 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:41.827704  7332 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:41.827893  7249 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:41.828438  7249 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:41.828933  7249 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:41.800278  7097 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:41.829903  7097 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:41.830404  7097 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:41.830752  5347 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53808 (request call id 420) took 1510 ms (client timeout 1878 ms). Trace:
I20260709 12:25:41.814543  7167 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:41.831238  5347 rpcz_store.cc:269] 0709 12:25:40.320340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.321245 (+   905us) service_pool.cc:224] Handling call
0709 12:25:41.830549 (+1509304us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b5400640354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:41.830669 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1019927808}
I20260709 12:25:41.834139  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:41.522109  4498 rpcz_store.cc:269] 0709 12:25:26.938737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.938900 (+   163us) service_pool.cc:224] Handling call
0709 12:25:41.515393 (+14576493us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:35.255770 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:35.334905 (+ 79135us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:35.338713 (+  3808us) write_op.cc:435] Acquired schema lock
0709 12:25:35.340589 (+  1876us) tablet.cc:662] Decoding operations
0709 12:25:35.363315 (+ 22726us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:35.363421 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:35.363451 (+    30us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:25:35.363786 (+   335us) tablet.cc:701] Row locks acquired
0709 12:25:35.363805 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:25:35.363982 (+   177us) write_op.cc:270] Start()
0709 12:25:35.364107 (+   125us) write_op.cc:276] Timestamp: P: 1783599935363903 usec, L: 0
0709 12:25:35.364128 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:25:35.564369 (+200241us) log.cc:844] Serialized 1308 byte log entry
0709 12:25:36.996219 (+1431850us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b1800291340. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:25:38.136104 (+1139885us) op_driver.cc:464] REPLICATION: finished
0709 12:25:39.187596 (+1051492us) write_op.cc:301] APPLY: starting
0709 12:25:39.191639 (+  4043us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:39.202475 (+ 10836us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:39.203388 (+   913us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:39.235802 (+ 32414us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:39.236076 (+   274us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:25:39.236130 (+    54us) write_op.cc:312] APPLY: finished
0709 12:25:39.236359 (+   229us) log.cc:844] Serialized 35 byte log entry
0709 12:25:39.443407 (+207048us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:39.559460 (+116053us) write_op.cc:454] Released schema lock
0709 12:25:39.618769 (+ 59309us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:39.815480 (+196711us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:39.818036 (+  2556us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":809387,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":1007383,"apply.run_cpu_time_us":20285,"apply.run_wall_time_us":588752,"num_ops":3,"prepare.queue_time_us":11032,"prepare.run_cpu_time_us":38230,"prepare.run_wall_time_us":702548,"raft.queue_time_us":792362,"raft.run_cpu_time_us":143946,"raft.run_wall_time_us":1196238,"replication_time_us":2703570,"spinlock_wait_cycles":45208832,"thread_start_us":1531343,"threads_started":5,"wal-append.queue_time_us":604035,"wal-append.run_cpu_time_us":25290,"wal-append.run_wall_time_us":2449598}]]}
I20260709 12:25:41.772883  7301 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:41.778713  7202 raft_consensus.cc:697] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:41.786886  7468 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:25:41.754199  7331 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:41.799751  7336 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:41.846078  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:41.850808  4598 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:41.836165  4644 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:41.818697  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:41.838471  4719 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:41.843546  4597 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:41.817509  7476 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:41.850090  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:41.854009  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.845364  7384 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:25:41.862233  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.862540  7202 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:41.864147  7492 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.855s	user 0.002s	sys 0.001s
W20260709 12:25:41.865974  7497 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.827s	user 0.003s	sys 0.000s
I20260709 12:25:41.867475  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1939) took 1151 ms. Trace:
W20260709 12:25:41.844834  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 109787us
I20260709 12:25:41.867831  4753 rpcz_store.cc:276] 0709 12:25:40.715742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.887145 (+171403us) service_pool.cc:224] Handling call
0709 12:25:40.887527 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.888025 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.888056 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:40.888654 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 12:25:40.993758 (+105104us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:40.994148 (+   390us) log.cc:844] Serialized 14 byte log entry
0709 12:25:40.994348 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.994421 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.865310 (+870889us) spinlock_profiling.cc:243] Waited 5.19 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:41.867207 (+  1897us) raft_consensus.cc:1685] finished
0709 12:25:41.867308 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.867419 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":211,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":82,"raft.queue_time_us":3400,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":9196,"spinlock_wait_cycles":11931904,"thread_start_us":104726,"threads_started":1,"wal-append.queue_time_us":149670}
I20260709 12:25:41.871147  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:41.872123  4597 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:41.872848  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:41.836746  7301 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:41.874389  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 127058us
I20260709 12:25:41.870754  4569 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:41.877002  4568 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:41.802794  7493 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:41.901292  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:41.896680  4597 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:41.895902  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 129918us
W20260709 12:25:41.902730  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 84127.2us
W20260709 12:25:41.911265  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 72856.1us
I20260709 12:25:41.886379  7173 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:41.886229  7501 consensus_queue.cc:1059] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:25:41.930920  4672 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:41.845621  7421 consensus_queue.cc:1059] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:25:41.886337  7353 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:41.924546  7523 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:41.928032  6915 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:41.902359  7191 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:25:41.895973  7331 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:41.935175  4596 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:41.930210  7336 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:41.941237  7328 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:41.944000  7502 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:41.909121  7270 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:41.908337  7338 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:41.907100  7476 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:41.914099  7416 consensus_queue.cc:1059] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:25:41.920166  7506 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:41.956885  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:41.956100  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 120009us
I20260709 12:25:41.974800  5349 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:41.992877  7536 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:42.002843  4677 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:42.004680  5347 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:41.956537  7353 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.005501  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:42.007437  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:42.010658  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 107306us
I20260709 12:25:42.004957  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:42.015740  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:42.016155  4680 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:42.015933  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:42.017314  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 140368us
I20260709 12:25:41.970624  7347 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
W20260709 12:25:42.032055  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 75228.3us
I20260709 12:25:42.004930  7336 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.031020  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:42.037325  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:25:42.040529  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 125595us
I20260709 12:25:42.041711  7173 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.042282  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:42.042476  7173 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:42.044121  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:42.020290  4677 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:41.976279  7421 raft_consensus.cc:3055] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:42.045792  7338 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:42.045915  7421 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:42.046980  7338 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:42.004273  7445 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:41.949522  7097 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:41.998567  7488 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:41.980194  7167 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 12:25:42.048609  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 76297.4us
W20260709 12:25:42.012055  7491 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:25:42.007925  7506 raft_consensus.cc:697] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:25:42.059690  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.030819  4678 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:42.015811  7501 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.060161  7249 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:42.060649  7501 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:42.062493  7249 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:41.979864  7511 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:42.069236  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:42.061040  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:42.055589  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:42.058982  7506 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:42.064999  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:42.007705  7353 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:42.050015  7173 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:42.075405  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:42.035200  7416 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:42.079712  7416 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:42.035043  7345 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.079785  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:42.065794  7526 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:25:42.083232  4597 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:42.047447  7421 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:42.066917  7493 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:42.064677  7339 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:42.059007  7167 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.090368  7532 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:25:42.090873  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:42.091666  6915 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:42.092392  6915 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:42.092506  4604 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:42.092967  6915 raft_consensus.cc:697] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:42.093672  7522 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:42.093920  4604 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:42.094255  7522 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:42.095669  6915 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:42.096428  7332 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:42.096909  7527 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:42.097491  7527 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:42.100401  7527 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:42.096381  7522 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:42.076427  7523 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:42.104403  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:42.104455  7488 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:42.105404  7540 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
W20260709 12:25:42.106907  7477 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 1.509s	user 0.003s	sys 0.000s
W20260709 12:25:42.107851  7479 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.026s	sys 0.051s
I20260709 12:25:42.063488  7501 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:42.112028  4678 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:42.078131  7270 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:42.120530  7540 consensus_queue.cc:1059] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:42.120976  7533 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:42.156028  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:42.105417  7530 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 12:25:42.159435  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 115432us
I20260709 12:25:42.161216  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:42.163703  4605 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:25:42.163939  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:42.167762  7542 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:42.168741  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:42.064283  7340 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:42.089630  7345 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:42.119674  7167 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:42.171718  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 114575us
I20260709 12:25:42.169735  7502 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:42.176338  7332 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:42.153879  7479 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.027s	sys 0.051s
W20260709 12:25:42.189642  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 114357us
I20260709 12:25:42.195358  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:42.173108  7421 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:42.197430  4569 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:42.199963  4602 raft_consensus.cc:2410] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:42.200276  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:42.197875  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:42.187423  7173 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:42.209092  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:42.206594  7340 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:42.210246  7542 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:42.211026  7345 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:42.211062  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:42.204080  7445 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:42.213864  7535 raft_consensus.cc:2764] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:25:42.217363  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:42.217940  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2429) took 2048 ms. Trace:
I20260709 12:25:42.218371  4675 raft_consensus.cc:2410] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:25:42.219014  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1551) took 1109 ms. Trace:
I20260709 12:25:42.208426  7191 tablet_bootstrap.cc:492] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:42.220762  7191 tablet_bootstrap.cc:492] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:25:42.213503  7511 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:42.221550  7191 ts_tablet_manager.cc:1403] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 1.515s	user 0.049s	sys 0.004s
I20260709 12:25:42.222214  4645 leader_election.cc:400] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8.
I20260709 12:25:42.222520  4645 leader_election.cc:403] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:25:42.171985  7202 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
W20260709 12:25:42.224227  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.187887  7416 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:42.225786  4678 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:42.817181  4678 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:42.818615  4678 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:42.219381  4675 rpcz_store.cc:276] 0709 12:25:41.109756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.110340 (+   584us) service_pool.cc:224] Handling call
0709 12:25:42.217167 (+1106827us) spinlock_profiling.cc:243] Waited 6.12 ms on lock 0x7b58000d4798. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:25:42.218922 (+  1755us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14067584}
I20260709 12:25:42.291168  7301 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:42.221733  7348 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:42.342962  4679 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:25:42.821822  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 600389us
I20260709 12:25:42.228204  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:42.822295  4569 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:42.227252  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:42.240772  4748 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:42.240944  4645 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:42.822278  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.821864  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 610814us
I20260709 12:25:42.834769  7527 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.835340  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:42.238008  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:42.835415  7527 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:42.242790  7550 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:25:42.837945  7301 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:42.266214  7097 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:42.835793  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:42.707240  7421 consensus_meta.cc:287] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.001s
I20260709 12:25:42.842280  4675 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:42.218343  4673 rpcz_store.cc:276] 0709 12:25:40.169633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.169766 (+   133us) service_pool.cc:224] Handling call
0709 12:25:40.255212 (+ 85446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.269916 (+ 14704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.269945 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:40.275914 (+  5969us) log.cc:844] Serialized 26 byte log entry
0709 12:25:41.163429 (+887515us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b180025f240. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3972a6 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9
0709 12:25:41.683959 (+520530us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b180025f240. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:25:41.893147 (+209188us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:41.918719 (+ 25572us) log.cc:844] Serialized 14 byte log entry
0709 12:25:41.975358 (+ 56639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.975515 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.975554 (+    39us) raft_consensus.cc:1685] finished
0709 12:25:41.975567 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.217742 (+242175us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b540007bf50. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:42.217849 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30338,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":19887,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":16967,"spinlock_wait_cycles":168692608,"thread_start_us":265209,"threads_started":2,"wal-append.queue_time_us":1677614}
I20260709 12:25:42.846853  5349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53808 (request call id 426) took 1156 ms. Trace:
I20260709 12:25:42.590917  7542 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:42.851038  4675 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:25:42.850726  7542 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:42.851403  7496 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:42.258774  7167 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:42.248108  7538 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8.
I20260709 12:25:42.847661  5349 rpcz_store.cc:276] 0709 12:25:41.690291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.694354 (+  4063us) service_pool.cc:224] Handling call
0709 12:25:41.694646 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.997332 (+302686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.846586 (+849254us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54006fbdd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:42.846766 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":643805440}
I20260709 12:25:42.338709  7173 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:42.249029  7332 consensus_queue.cc:1059] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.600s
I20260709 12:25:42.884910  7332 raft_consensus.cc:3055] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:42.885097  7534 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:42.885376  7332 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:42.885883  7375 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:42.886478  7375 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:42.886574  7332 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:42.887892  7510 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.888451  7510 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:42.870599  7472 raft_consensus.cc:3055] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:42.889745  7472 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:42.889863  7527 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:42.866673  7097 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:42.890738  7097 raft_consensus.cc:697] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:42.890645  7472 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:42.891947  7472 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:42.891984  7097 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:42.873507  7421 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:42.875516  4676 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.010s	sys 0.009s
W20260709 12:25:42.853248  7536 consensus_meta.cc:287] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.836s	user 0.011s	sys 0.000s
I20260709 12:25:42.886116  7534 raft_consensus.cc:2955] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index 2 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:42.888775  7375 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:42.870323  4678 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:42.893249  7445 consensus_meta.cc:287] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.000s
I20260709 12:25:42.226684  7270 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:42.881839  7544 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:42.897226  4676 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.010s	sys 0.009s
W20260709 12:25:42.873129  7353 consensus_meta.cc:287] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.790s	user 0.010s	sys 0.000s
I20260709 12:25:42.865332  7191 raft_consensus.cc:359] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:42.901721  7536 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:42.892874  7542 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:42.909164  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.905184  7191 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:25:42.913259  7191 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:42.916306  7538 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.916847  7538 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:42.904726  7544 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:42.919023  7538 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:42.919586  7544 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:42.920958  5343 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53808 (request call id 423) took 2563 ms (client timeout 1561 ms). Trace:
W20260709 12:25:42.921430  5343 rpcz_store.cc:269] 0709 12:25:40.357624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.357948 (+   324us) service_pool.cc:224] Handling call
0709 12:25:41.406556 (+1048608us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b5800003c98. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:25:42.920707 (+1514151us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54006fca54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:42.920838 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290415104}
I20260709 12:25:42.922699  7510 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:42.903692  7353 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:42.904162  7445 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:42.966630  4681 raft_consensus.cc:2368] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8.
I20260709 12:25:42.925446  7538 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:42.979954  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.981387  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.967024  7191 ts_tablet_manager.cc:1434] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.745s	user 0.021s	sys 0.017s
W20260709 12:25:42.941416  7552 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.013s	sys 0.015s
I20260709 12:25:42.989315  7445 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:42.991197  7527 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:42.998451  7552 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.013s	sys 0.015s
I20260709 12:25:42.990854  7538 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:43.030362  7353 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:42.995997  7202 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:42.996412  7191 ts_tablet_manager.cc:1467] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Tablet startup took 773ms
W20260709 12:25:43.002660  7505 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.317s	user 0.031s	sys 0.037s
I20260709 12:25:43.011623  7416 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:43.048290  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.044620  7505 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.360s	user 0.031s	sys 0.037s
I20260709 12:25:43.032584  7527 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:43.055691  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 220426us
I20260709 12:25:43.033034  7445 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:43.038985  7202 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:43.070335  4678 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:43.070567  5312 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:43.071868  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 830966us
I20260709 12:25:43.068421  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:43.078217  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:43.095124  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:43.096814  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:43.097801  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:43.097811  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 271539us
I20260709 12:25:43.098881  4569 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:43.097306  4720 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:43.103780  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:43.104619  5348 raft_consensus.cc:2410] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:43.104460  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:43.105324  5347 raft_consensus.cc:2376] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4. Re-sending same reply.
I20260709 12:25:43.106880  4569 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:43.108120  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:43.109351  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:43.090325  7538 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:43.113255  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:43.095861  7505 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:43.142395  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 45768us
I20260709 12:25:43.147643  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:43.143365  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:43.143832  4569 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:43.143137  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:43.144317  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:43.149739  4569 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:43.147737  4647 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:43.148891  4674 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:43.102443  7506 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:43.096580  7542 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:43.165802  7553 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:43.111838  7510 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:43.171414  7476 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:43.110049  7270 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:43.216475  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 68755.1us
I20260709 12:25:43.172242  7345 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:43.176086  7537 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:43.142478  7173 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:43.179030  7494 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:43.179268  7471 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:43.180701  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:43.182231  4673 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:43.166208  7328 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:43.188419  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 411) took 1421 ms. Trace:
I20260709 12:25:43.189503  6915 raft_consensus.cc:3055] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:43.189683  7505 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:43.167550  7540 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:43.192345  7202 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:43.142333  7482 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:43.110762  7353 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.195138  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:43.221653  7421 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:43.204020  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:43.112618  7536 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:43.111675  7544 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:43.170231  7561 consensus_queue.cc:1059] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:43.287161  7561 raft_consensus.cc:3055] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:43.287720  7561 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:43.205732  4674 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:43.207489  7568 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:43.208756  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:43.290459  4603 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:43.204873  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:43.219614  7368 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:43.223594  7563 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.229873  4679 rpcz_store.cc:276] 0709 12:25:41.766651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.766791 (+   140us) service_pool.cc:224] Handling call
0709 12:25:41.770945 (+  4154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:42.831360 (+1060415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.188041 (+356681us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b540017eb54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:43.188330 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1314344192}
I20260709 12:25:43.235821  4647 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:43.235459  4675 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:43.216368  7476 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:43.254213  4680 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:43.261960  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2439) took 1260 ms. Trace:
I20260709 12:25:43.352530  4681 rpcz_store.cc:276] 0709 12:25:42.001867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.007188 (+  5321us) service_pool.cc:224] Handling call
0709 12:25:43.261649 (+1254461us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b540017d754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:43.261861 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2191213824}
I20260709 12:25:43.202445  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:43.285058  7482 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:43.293931  4602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1537) took 1727 ms (client timeout 1740 ms). Trace:
I20260709 12:25:43.228642  7471 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:43.301298  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.293498  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 150084us
I20260709 12:25:43.231812  6915 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:43.357775  4602 rpcz_store.cc:269] 0709 12:25:41.566723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.628157 (+ 61434us) service_pool.cc:224] Handling call
0709 12:25:43.262860 (+1634703us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54003539d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:43.293861 (+ 31001us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1212378368}
I20260709 12:25:43.358592  4676 raft_consensus.cc:2376] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5. Re-sending same reply.
I20260709 12:25:43.224102  7542 consensus_queue.cc:1059] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:25:43.257820  7421 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:43.360638  7471 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:43.375332  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 203716us
W20260709 12:25:43.375540  4596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1545) took 1747 ms (client timeout 2024 ms). Trace:
I20260709 12:25:43.292984  7445 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:43.223486  7553 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:43.373616  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:43.288868  7561 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:43.376782  4598 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 12:25:43.376048  4596 rpcz_store.cc:269] 0709 12:25:41.628665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.650839 (+ 22174us) service_pool.cc:224] Handling call
0709 12:25:43.375313 (+1724474us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54001405d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:43.375458 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":515312000}
W20260709 12:25:43.293061  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 222553us
I20260709 12:25:43.398865  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1558) took 1328 ms. Trace:
W20260709 12:25:43.379509  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 145683us
I20260709 12:25:43.402042  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:43.402732  4681 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:43.365093  7557 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 12:25:43.403409  4679 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:43.260913  7353 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:43.399479  4597 rpcz_store.cc:276] 0709 12:25:42.049805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.049929 (+   124us) service_pool.cc:224] Handling call
0709 12:25:42.050250 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:42.084889 (+ 34639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.377234 (+1292345us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b54003534d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:43.377987 (+   753us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75440896}
I20260709 12:25:43.407109  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:43.407511  4569 leader_election.cc:400] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8.
I20260709 12:25:43.407943  4596 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:43.375109  7270 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.409418  4647 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:43.410060  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:43.410568  4720 connection.cc:256] RPC call timeout handler was delayed by -1.2898999480003113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
I20260709 12:25:43.256770  7540 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:43.375385  7421 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:43.365440  7494 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:43.414063  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:43.418846  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:43.415731  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:43.421617  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.430356  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:43.422640  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:43.429461  4597 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
W20260709 12:25:43.039093  7191 ts_tablet_manager.cc:1469] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Trace:
0709 12:25:30.667691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.674923 (+  7232us) service_pool.cc:224] Handling call
0709 12:25:31.149702 (+474779us) ts_tablet_manager.cc:934] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
0709 12:25:32.332481 (+1182779us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b4c000d36c0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effec782a20 00007effec793a29 00007effec79413f 00007effec7582ab 00007effec785f3c 00007effec786461 00007effec786768 00007efff250c2eb 00007efff3431d3b 00007efff342c008 00007efff342b405
0709 12:25:33.930939 (+1598458us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:25:35.145644 (+1214705us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:36.345752 (+1200108us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:36.385689 (+ 39937us) tablet_copy_client.cc:463] Replaced tombstoned tablet metadata.
0709 12:25:38.618874 (+2233185us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:25:38.685282 (+ 66408us) tablet_replica.cc:206] Creating consensus instance
0709 12:25:38.766954 (+ 81672us) ts_tablet_manager.cc:989] Replica registered.
0709 12:25:38.774232 (+  7278us) inbound_call.cc:177] Queueing success response
0709 12:25:40.706256 (+1932024us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:25:42.221932 (+1515676us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:25:42.241417 (+ 19485us) tablet_replica.cc:261] Starting instrumentation
0709 12:25:42.838366 (+596949us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":131749,"mutex_wait_us":6581,"raft.queue_time_us":10000,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":699,"spinlock_wait_cycles":73585408,"tablet-copy.queue_time_us":184591,"tablet-download-pool.queue_time_us":95607,"tablet-download-pool.run_cpu_time_us":2095,"tablet-download-pool.run_wall_time_us":311811,"thread_start_us":271113,"threads_started":2}
I20260709 12:25:43.417435  4603 raft_consensus.cc:2393] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:43.428408  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:43.414088  4720 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.193:45077 timed out after 1.594s (SENT) [suppressed 10 similar messages]
I20260709 12:25:43.433720  4675 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:43.436041  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:43.440094  4602 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:43.358264  7563 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:43.440157  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:43.444406  4678 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:43.444725  4596 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:43.441605  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:43.445925  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:43.447278  4597 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 2.
I20260709 12:25:43.416240  7527 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:43.449069  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:43.453084  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:43.411304  7558 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:25:43.411114  7472 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 12:25:43.464619  7564 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:43.465318  7472 raft_consensus.cc:3055] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:43.465724  7472 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:43.432861  7559 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:43.466591  7472 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:43.464232  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:43.467779  7472 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:43.469058  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.470798  4679 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:43.415320  7565 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:25:43.473747  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:43.474773  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 72059.7us
I20260709 12:25:43.475890  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:43.476117  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:43.421283  7270 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:43.491117  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:43.491915  4753 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:43.447098  7561 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:43.478706  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:43.479539  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:43.484851  4679 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:43.446982  7494 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:43.504498  4596 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:43.511379  4675 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:43.515033  4747 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:43.519999  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:43.520818  4751 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:43.522059  4602 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:25:43.527565  7494 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:43.448493  7573 consensus_queue.cc:1059] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:25:43.536515  7353 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:43.539215  7553 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:43.543473  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.547060  7496 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:43.551088  4604 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:43.552269  5312 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:43.552744  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.555068  7574 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:43.557507  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:43.558043  5349 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:25:43.558633  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:43.606676  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 54497.6us
I20260709 12:25:43.561316  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:43.565784  4601 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:43.570103  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.570473  7345 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:43.572506  5347 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:43.572564  7332 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:43.610934  5348 raft_consensus.cc:2955] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 2 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:25:43.574234  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.637238  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.639585  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:43.575088  4673 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:43.584514  7468 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:43.586199  7576 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:25:43.587005  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:43.594189  4719 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:43.667847  4747 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:25:43.668479  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 74400.8us
I20260709 12:25:43.603125  4602 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:43.604029  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 128404us
I20260709 12:25:43.440024  6915 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:43.609711  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:43.671025  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:43.623620  4598 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:43.624100  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:43.673086  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49066.4us
I20260709 12:25:43.643916  4645 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:43.621918  7574 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:43.600754  7301 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.676519  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:43.672012  4680 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
W20260709 12:25:43.678377  4672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1558) took 2457 ms (client timeout 1868 ms). Trace:
W20260709 12:25:43.683391  4672 rpcz_store.cc:269] 0709 12:25:41.221100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.221255 (+   155us) service_pool.cc:224] Handling call
0709 12:25:43.678190 (+2456935us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b54000b0254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:43.678316 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1185314176}
W20260709 12:25:43.688201  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 44365.4us
I20260709 12:25:43.608321  7505 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:43.689033  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:43.689670  4754 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:43.704744  4647 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:43.612094  7345 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:43.634552  7302 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:43.707544  4677 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:25:43.734306  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:43.734939  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.705457  4598 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:43.693085  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:43.715171  4644 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:43.714071  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:43.738930  4754 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:43.675537  7468 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:43.679482  7558 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:43.718855  4755 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:43.745632  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:43.746235  4755 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:43.746462  4601 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:43.746992  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:43.747233  4601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1529) took 2192 ms (client timeout 2234 ms). Trace:
I20260709 12:25:43.747591  4755 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:43.714725  7340 consensus_queue.cc:1059] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:25:43.720054  7345 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:43.676657  7488 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:43.722127  4748 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:43.762696  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:43.672812  7202 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:25:43.726468  7270 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:43.729934  7571 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 12:25:43.730638  7302 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.719090  7574 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:43.749099  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 56108.8us
I20260709 12:25:43.773690  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:43.676764  7353 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 12:25:43.678870  7445 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:43.706543  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:43.753672  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:43.712390  7476 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:43.756855  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:43.757835  4752 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:43.707175  7301 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:43.747818  4601 rpcz_store.cc:269] 0709 12:25:41.555095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.555239 (+   144us) service_pool.cc:224] Handling call
0709 12:25:43.565341 (+2010102us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b580051f8d0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4120aa 00007effee494447 00007effee4acfe7 00007effee4ae7de 00007effee4ba27c 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c
0709 12:25:43.747159 (+181818us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4121980672}
I20260709 12:25:43.789673  5344 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 4: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:43.759223  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:43.790983  4754 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:43.791426  4691 tablet_copy_service.cc:342] P 3a5aeedd11fe49938079fe86cb01232c: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-1b7b0c6c811e455abe898cb4d21acad4 received from {username='slave'} at 127.0.0.1:48198
I20260709 12:25:43.792013  4691 tablet_copy_service.cc:434] P 3a5aeedd11fe49938079fe86cb01232c: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-1b7b0c6c811e455abe898cb4d21acad4 on tablet 1b7b0c6c811e455abe898cb4d21acad4 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:43.787595  7505 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:25:43.763329  4748 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:25:43.710664  7542 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:25:43.782032  7302 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:43.795521  7332 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:43.796777  4681 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:25:43.714074  7559 consensus_queue.cc:1059] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
W20260709 12:25:43.768062  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 353980us
I20260709 12:25:43.766100  5312 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:43.743916  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:43.785017  7565 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:43.807019  4750 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:43.763895  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:43.774838  4571 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:43.690337  7538 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:43.732563  4749 raft_consensus.cc:2955] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 2 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:43.761890  7488 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.770737  7368 consensus_queue.cc:1059] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:25:43.757761  7540 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:43.768738  7527 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:25:43.758765  7568 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:43.793509  7345 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:43.798856  7445 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:25:43.827659  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.828040  4571 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:43.837404  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:43.801934  7505 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.785812  7380 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:25:43.807070  7572 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:43.842207  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:43.843340  4747 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:43.846350  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 154519us
I20260709 12:25:43.795950  7332 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:43.811530  7565 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.854323  7565 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:43.854286  7332 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:43.816933  7488 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:43.816601  7558 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:43.860338  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 145259us
W20260709 12:25:43.878981  4720 connection.cc:256] RPC call timeout handler was delayed by -1.1494999480000843s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20240000000000002s for a response
W20260709 12:25:43.879566  4720 connection.cc:256] RPC call timeout handler was delayed by -1.119799948000491s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2057s for a response
W20260709 12:25:43.879868  4720 connection.cc:256] RPC call timeout handler was delayed by -1.070299948000411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
I20260709 12:25:43.835971  7568 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:43.835971  7538 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:43.849838  7540 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:43.826421  7563 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:43.881866  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 64442.3us
W20260709 12:25:43.882848  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.883869  4569 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:43.932579  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 262086us
W20260709 12:25:43.937546  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 69610.4us
I20260709 12:25:43.939555  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:43.840360  7494 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:43.848181  7339 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:43.972115  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.945312  4679 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:43.990163  4747 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:43.868094  7485 raft_consensus.cc:3055] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:43.993674  4679 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:25:43.866196  7141 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.001760  4681 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:43.866132  7348 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:44.018406  4673 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:43.866990  7558 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.872783  7536 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.050575  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:43.881013  7580 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:44.052280  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 50642.9us
I20260709 12:25:43.850562  7505 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:43.933434  7568 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:43.841862  7479 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 12:25:43.937698  7353 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:43.989380  7544 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:44.069680  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.993608  7540 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.995889  7339 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.076855  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.100087  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.068264  7378 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:25:44.000079  7485 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:43.990150  7328 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 12:25:44.018131  7506 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.073606  7558 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:44.101081  7339 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:44.104138  7583 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:25:44.057582  7141 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:44.105582  7581 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.265s	user 0.000s	sys 0.000s
W20260709 12:25:44.106840  7586 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:25:44.112190  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.110162  7588 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
W20260709 12:25:44.173719  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.084928  7173 raft_consensus.cc:3055] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:44.123399  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.057586  7415 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:25:44.084323  7348 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.094946  7353 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:44.101207  7540 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:44.095324  7536 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:44.128233  7421 consensus_meta.cc:287] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.688s	user 0.009s	sys 0.001s
W20260709 12:25:44.137009  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 253216us
W20260709 12:25:44.176401  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 192864us
W20260709 12:25:44.143572  7482 consensus_meta.cc:287] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.781s	user 0.012s	sys 0.000s
W20260709 12:25:44.172188  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 79393us
I20260709 12:25:44.171233  4750 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:25:44.076905  7445 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:44.224047  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.217235  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 93926.4us
I20260709 12:25:44.225082  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:44.225277  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:44.177333  7340 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:44.177199  7301 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.128297  7506 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.180145  7584 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:44.179317  7173 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:44.229046  4599 raft_consensus.cc:2410] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:25:44.235309  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 423385us
I20260709 12:25:44.221248  7468 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.218292  7593 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:44.252074  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:44.268292  7544 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:44.268870  7544 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:44.255628  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:44.250576  7494 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.230388  7097 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:44.178761  7538 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.233341  7617 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.221591  7421 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.296866  4647 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:44.282295  7471 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.881s	user 0.011s	sys 0.000s
I20260709 12:25:44.252902  7301 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.250576  7608 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:44.272707  7510 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:44.355207  7590 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.241s	user 0.001s	sys 0.001s
W20260709 12:25:44.379335  7302 consensus_meta.cc:287] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.584s	user 0.008s	sys 0.000s
I20260709 12:25:44.250746  7537 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
I20260709 12:25:44.382961  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.390925  7479 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:44.388319  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.371310  7617 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.357336  7593 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:44.357601  7340 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:44.373168  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:44.398684  7608 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:44.382459  7584 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:25:44.449713  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 66831.9us
I20260709 12:25:44.382426  7609 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:44.443094  7613 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:44.444305  7141 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.406143  4679 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:44.459316  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 177681us
I20260709 12:25:44.356935  7494 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:44.457528  7345 consensus_meta.cc:287] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.617s	user 0.007s	sys 0.003s
I20260709 12:25:44.501211  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.458189  7620 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:44.358701  7097 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:44.518533  7594 log.cc:927] Time spent T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:44.444620  7270 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:44.355798  7353 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.509086  4647 leader_election.cc:400] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8.
W20260709 12:25:44.567800  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 58920.4us
W20260709 12:25:44.510859  7603 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.231s	user 0.000s	sys 0.000s
I20260709 12:25:44.512686  7618 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.406116  7173 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.399436  7301 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.392338  7471 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:44.498935  7505 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:44.404663  7510 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:44.539350  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 314342us
W20260709 12:25:44.546203  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 45084.5us
W20260709 12:25:44.463587  7332 consensus_meta.cc:287] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.609s	user 0.007s	sys 0.003s
I20260709 12:25:44.501058  7608 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:44.515005  7626 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 12:25:44.513546  7496 consensus_meta.cc:287] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.902s	user 0.007s	sys 0.000s
I20260709 12:25:44.544921  7339 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:44.558288  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.558907  5348 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:25:44.614888  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.514334  7596 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
W20260709 12:25:44.499281  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 126106us
I20260709 12:25:44.452766  7302 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.471568  7540 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.460125  7479 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.634596  7540 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.635491  7537 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:44.517937  7536 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.518218  7348 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:44.575564  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.545136  7600 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.591439  7353 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:44.559816  7607 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.598145  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:44.515481  7610 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 12:25:44.612938  4678 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:44.509080  7609 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:44.635926  7611 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:25:44.707899  7301 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.708511  7421 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.709192  7619 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.709786  7619 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.533833  7584 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:44.699306  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 53824.9us
I20260709 12:25:44.623101  7505 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 12:25:44.713464  7505 consensus_queue.cc:1243] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:44.600836  7567 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:44.638373  7628 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:44.519943  7613 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.601670  7332 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:44.612517  7565 consensus_meta.cc:287] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.758s	user 0.005s	sys 0.005s
I20260709 12:25:44.704974  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:44.634618  7479 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.706903  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:44.715996  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 102650us
W20260709 12:25:44.737075  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.714962  7339 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.581769  7618 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.646320  7496 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:44.566279  7620 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.724874  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2398) took 1050 ms. Trace:
I20260709 12:25:44.637977  7623 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
W20260709 12:25:44.727782  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 129718us
W20260709 12:25:44.728551  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 160082us
I20260709 12:25:44.622900  7482 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.093s
I20260709 12:25:44.730793  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:44.705857  7506 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:44.739373  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:44.744656  4681 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:44.725164  7628 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:44.745167  4751 rpcz_store.cc:276] 0709 12:25:43.674346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.688674 (+ 14328us) service_pool.cc:224] Handling call
0709 12:25:43.717276 (+ 28602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.717677 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.717708 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.718190 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 12:25:43.879883 (+161693us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:43.879925 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.879991 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:44.724549 (+844558us) raft_consensus.cc:1685] finished
0709 12:25:44.724585 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.724788 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":858,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":161325,"threads_started":1,"wal-append.queue_time_us":639199}
W20260709 12:25:44.609165  7568 consensus_meta.cc:287] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.000s
I20260709 12:25:44.729837  7567 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:44.743100  7472 consensus_meta.cc:287] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 1.275s	user 0.013s	sys 0.000s
W20260709 12:25:44.751464  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 44885.1us
I20260709 12:25:44.796159  7506 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:25:44.798139  7568 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.799345  7506 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.799947  7506 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.800350  7568 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.801030  7622 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:44.801451  7622 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:44.801837  7622 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:44.801973  7471 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:44.802564  7558 consensus_meta.cc:287] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.000s
I20260709 12:25:44.803015  7558 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.803560  7332 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.804349  7617 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.804414  7620 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.805684  7625 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:44.805879  7141 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.806293  7625 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.806430  7141 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.807430  7619 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:44.808573  7625 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:44.808810  7141 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.809157  7621 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:44.809756  7621 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.805321  7558 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:44.810567  7506 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:44.812083  7621 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:44.813089  7630 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.813760  7630 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.815366  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:44.819936  7534 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.820461  7534 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.818197  4751 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:44.818572  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:44.782701  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.785241  4647 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.785640  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 429) took 1677 ms. Trace:
I20260709 12:25:44.751711  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:44.785935  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 58285us
I20260709 12:25:44.824084  4596 rpcz_store.cc:276] 0709 12:25:43.108186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.540959 (+432773us) service_pool.cc:224] Handling call
0709 12:25:43.541292 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.599393 (+ 58101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.599425 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.600159 (+   734us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.091656 (+491497us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:44.091702 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.091773 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:44.785340 (+693567us) raft_consensus.cc:1685] finished
0709 12:25:44.785373 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.785557 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57715,"raft.queue_time_us":58712,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":491004,"threads_started":1,"wal-append.queue_time_us":576601}
I20260709 12:25:44.703992  7600 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.780680  7479 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.795591  7628 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:44.796685  7580 consensus_meta.cc:287] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.708s	user 0.004s	sys 0.007s
I20260709 12:25:44.854318  7580 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:44.720482  7607 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.855692  7613 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:44.857256  7534 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.830075  7482 raft_consensus.cc:3055] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:44.857849  7619 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.857988  7482 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:44.858455  7619 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.847711  7332 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:44.859045  7482 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.860504  7482 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:44.860639  7607 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:44.861840  7537 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.735726  7565 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:44.862839  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 79425us
I20260709 12:25:44.863360  5343 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:44.850391  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:44.864048  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.863371  5347 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
W20260709 12:25:44.855839  7544 consensus_meta.cc:287] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.587s	user 0.005s	sys 0.005s
W20260709 12:25:44.866133  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.830785  7494 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:44.871727  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 89098.5us
I20260709 12:25:44.872292  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.875339  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.876363  4647 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.880148  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.881152  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:44.883603  4676 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:44.860543  7619 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.886670  4751 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:44.887539  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.888669  5314 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:44.890583  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.891805  4647 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:44.879271  7488 consensus_meta.cc:287] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.997s	user 0.005s	sys 0.006s
I20260709 12:25:44.895920  4569 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:44.859090  7600 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.874636  7332 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:44.893460  4647 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:44.878520  7496 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:44.900238  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:44.735287  7584 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.886314  7494 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:44.862308  7537 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.894184  7472 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:44.896637  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:44.904239  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:44.925923  4676 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:44.901772  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:44.851042  7479 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.892586  7421 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:44.928426  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2464) took 1508 ms. Trace:
I20260709 12:25:44.874351  7544 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.932538  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.892292  7621 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:44.932150  4678 rpcz_store.cc:276] 0709 12:25:43.419634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.565802 (+146168us) service_pool.cc:224] Handling call
0709 12:25:43.566158 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.703592 (+1137434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.926139 (+222547us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540007b554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:44.928358 (+  2219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2406175616}
I20260709 12:25:44.954120  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:44.954288  7537 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.954952  4678 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 6: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:44.898763  7445 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:25:44.955824  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:44.895179  7639 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:44.938293  7613 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:44.942492  7561 consensus_meta.cc:287] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.432s	user 0.012s	sys 0.000s
I20260709 12:25:44.942911  7472 raft_consensus.cc:697] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:44.900614  7506 consensus_queue.cc:1059] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:25:44.966475  7609 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:44.958904  7494 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:44.966698  7472 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.967465  7479 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:44.967902  7494 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.968215  7623 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:44.945842  5342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 429) took 1063 ms. Trace:
I20260709 12:25:44.946825  7622 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.900475  7270 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:44.977023  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 44561.9us
I20260709 12:25:44.978346  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.948464  7544 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:44.950304  7578 log.cc:927] Time spent T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 1.213s	user 0.002s	sys 0.002s
I20260709 12:25:44.982590  7565 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:44.982786  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1967) took 1882 ms. Trace:
I20260709 12:25:44.983167  4605 rpcz_store.cc:276] 0709 12:25:43.100499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.100594 (+    95us) service_pool.cc:224] Handling call
0709 12:25:43.102867 (+  2273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.145963 (+ 43096us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:43.146012 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.169521 (+ 23509us) log.cc:844] Serialized 26 byte log entry
0709 12:25:43.437040 (+267519us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:43.437078 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.437141 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:44.836403 (+1399262us) spinlock_profiling.cc:243] Waited 6.47 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:44.982478 (+146075us) raft_consensus.cc:1685] finished
0709 12:25:44.982500 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.982720 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":137,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":29173,"spinlock_wait_cycles":14883072,"thread_start_us":209108,"threads_started":1,"wal-append.queue_time_us":296877}
I20260709 12:25:44.985450  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2417) took 1117 ms. Trace:
I20260709 12:25:44.986050  4749 rpcz_store.cc:276] 0709 12:25:43.867823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.372910 (+505087us) service_pool.cc:224] Handling call
0709 12:25:44.416523 (+ 43613us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:44.729338 (+312815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.729376 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:44.739219 (+  9843us) log.cc:844] Serialized 52 byte log entry
0709 12:25:44.887731 (+148512us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:44.887770 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.887970 (+   200us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:44.985145 (+ 97175us) raft_consensus.cc:1685] finished
0709 12:25:44.985177 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.985370 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2453,"raft.queue_time_us":155850,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":77872,"thread_start_us":369329,"threads_started":2,"wal-append.queue_time_us":225799}
I20260709 12:25:44.988181  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:44.988725  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:44.990497  7332 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:44.972416  5342 rpcz_store.cc:276] 0709 12:25:43.882571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.882711 (+   140us) service_pool.cc:224] Handling call
0709 12:25:44.071898 (+189187us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.174399 (+102501us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:44.439495 (+265096us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.642221 (+202726us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.740022 (+ 97801us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.746970 (+  6948us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:44.780881 (+ 33911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.787594 (+  6713us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:44.945501 (+157907us) raft_consensus.cc:1685] finished
0709 12:25:44.945520 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.945752 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23236,"prepare.queue_time_us":486461,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":141501,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":40711,"spinlock_wait_cycles":6104960,"thread_start_us":305074,"threads_started":2,"wal-append.queue_time_us":368456}
I20260709 12:25:44.997471  7622 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:44.972651  7623 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.950989  7627 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:44.822119  7567 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:44.898794  7488 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:44.954157  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:44.897425  7607 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:44.959165  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 62654.9us
I20260709 12:25:45.005519  4754 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:45.006729  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:44.956431  4678 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:45.007220  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:45.007516  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:45.008194  4754 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:44.902761  7619 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:44.974608  4676 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:45.009618  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:45.013417  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:45.014456  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 454) took 1209 ms. Trace:
I20260709 12:25:44.972688  7618 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:45.015303  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.900566  7629 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:44.955386  7485 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.019651  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:45.004410  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:44.893823  7301 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:25:45.021052  7496 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:45.022490  7488 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:45.013520  4569 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:45.021049  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.000394  7623 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.018947  4672 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:25:45.001875  4749 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.003993  7627 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.023756  4678 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:45.021157  4647 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:45.033730  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:45.034369  4676 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.008440  7332 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:45.023411  7639 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:45.042186  7631 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:45.042192  7639 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.043208  7639 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.043148  7631 raft_consensus.cc:697] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:45.017916  7619 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.019779  7544 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:45.051905  4673 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:45.050093  7544 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.036207  7627 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:45.039789  7485 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.040503  7618 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:45.033372  7445 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:45.048943  7488 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.045833  7301 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:45.050292  7619 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.058784  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:45.045152  7629 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:45.060634  4569 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:45.040164  7496 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:45.068463  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:45.068967  4678 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:45.057773  7544 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:45.066906  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:45.082916  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 453) took 1309 ms. Trace:
I20260709 12:25:45.071337  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:45.083317  4747 rpcz_store.cc:276] 0709 12:25:43.773323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.991154 (+217831us) service_pool.cc:224] Handling call
0709 12:25:43.991508 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.991756 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:44.151547 (+159791us) log.cc:844] Serialized 14 byte log entry
0709 12:25:45.044313 (+892766us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b18003bdf40. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:25:45.082597 (+ 38284us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:45.082666 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:45.082707 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.082760 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.082862 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":121980,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":3732,"spinlock_wait_cycles":38733696,"thread_start_us":119939,"threads_started":2}
I20260709 12:25:45.085075  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:45.065402  7445 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.090924  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2445) took 3083 ms. Trace:
I20260709 12:25:45.091380  4677 rpcz_store.cc:276] 0709 12:25:42.006797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.071064 (+ 64267us) service_pool.cc:224] Handling call
0709 12:25:43.678299 (+1607235us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.748603 (+ 70304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.090203 (+1341600us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54000b0250. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:45.090357 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3695183104}
I20260709 12:25:45.092634  7445 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:45.073789  7506 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:45.093109  7609 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.093448  7506 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:45.073827  7575 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.098688  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2475) took 1115 ms. Trace:
I20260709 12:25:45.099116  4681 rpcz_store.cc:276] 0709 12:25:43.983331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.002689 (+ 19358us) service_pool.cc:224] Handling call
0709 12:25:44.003044 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.784806 (+781762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.098434 (+313628us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b5400086954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:45.098628 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1705112832}
I20260709 12:25:45.047215  7631 consensus_queue.cc:237] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.087756  4747 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:45.087986  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:45.070343  7565 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:45.089437  7302 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.092674  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.093559  7609 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.106858  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:45.107231  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.107759  4752 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:45.087345  7496 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.096505  7533 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:45.079062  7618 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:45.094005  7506 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:45.109786  4749 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:45.106128  4677 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:45.068126  7622 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:45.121557  7496 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:45.108899  7574 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:45.103070  4676 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:45.126929  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.114627  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.127645  4752 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:45.127767  4681 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:25:45.128450  4605 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:25:45.115200  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:45.065843  7488 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.101390  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.131075  4748 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:45.108310  7565 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.120114  7302 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:45.120820  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:45.103854  7445 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:45.119994  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.063061  7608 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.134100  4747 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:45.164125  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:45.134445  5344 raft_consensus.cc:2410] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:45.136489  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.105762  7567 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:45.137807  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:45.137485  4647 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:45.157985  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.167965  4676 raft_consensus.cc:2410] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:45.120038  4569 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:45.155891  4677 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:45.104673  7644 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:45.126116  7561 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:45.097393  7575 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:45.166786  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.162374  4678 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
W20260709 12:25:45.179886  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 59941.5us
I20260709 12:25:45.180346  4749 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
W20260709 12:25:45.181551  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.181777  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.109422  7629 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.177726  7567 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.194519  4754 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:45.182857  4569 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:45.198040  7348 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:45.198562  7348 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.198959  7348 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:45.202610  7533 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.137703  7496 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:45.162336  7565 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.153944  7574 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:45.203213  7533 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.168565  7445 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.170864  7628 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.208647  7496 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.224354  7496 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:45.224777  7628 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:45.166572  7302 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.207840  7609 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:45.228653  5312 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:45.227099  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:45.229501  5312 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:45.208206  7536 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:45.292469  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:45.293282  4598 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:45.208890  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:45.307480  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 170511us
I20260709 12:25:45.226418  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.189545  7301 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:45.314292  4752 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
W20260709 12:25:45.344935  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.294617  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:45.358124  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 229533us
I20260709 12:25:45.385553  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.386255  4749 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:45.388335  4720 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:45.388607  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:45.389734  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:45.358605  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:45.390442  4602 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:25:45.391988  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:45.392689  4603 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:45.392772  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:45.393767  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:45.395605  4647 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:45.396521  4681 pending_rounds.cc:85] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 5
I20260709 12:25:45.396883  4681 pending_rounds.cc:107] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:25:45.394428  4597 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:45.397127  4681 raft_consensus.cc:2889] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:45.397547  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:45.360451  7632 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:45.356588  7301 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.399669  7301 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.384609  7573 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:45.408898  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:45.409636  4602 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:45.177546  7616 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.289s
I20260709 12:25:45.402951  7494 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:45.394313  7628 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.402143  7632 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.403841  7621 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:45.403276  7623 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:45.408459  7567 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:45.401250  7622 consensus_queue.cc:1059] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:25:45.229254  7618 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.418053  7613 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:45.428406  7559 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:45.428578  7566 raft_consensus.cc:3055] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:25:45.444195  7643 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 12:25:45.444237  7097 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:45.430974  7621 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:25:45.445439  7649 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.311s	user 0.001s	sys 0.001s
I20260709 12:25:45.446008  7629 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.434746  7628 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:45.408705  7574 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.423446  7573 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:45.429265  7632 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:45.412905  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.455646  7608 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:45.466217  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.448005  7609 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:45.428843  7494 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:45.470484  7621 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.471923  7488 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.461237  7622 consensus_queue.cc:1243] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:45.457095  7559 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:45.460438  7566 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:45.481920  4752 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:25:45.439997  7623 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:45.487370  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:45.469429  7575 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.405619  7636 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:25:45.487032  7494 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:45.474738  7627 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.496073  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 426) took 1801 ms. Trace:
I20260709 12:25:45.493247  7644 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:45.471300  7536 raft_consensus.cc:697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:45.483790  7445 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:45.485716  7607 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:45.494463  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 97008.8us
I20260709 12:25:45.512092  4672 rpcz_store.cc:276] 0709 12:25:43.694680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.695165 (+   485us) service_pool.cc:224] Handling call
0709 12:25:43.743147 (+ 47982us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:45.032549 (+1289402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.495731 (+463182us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540017e154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:45.495984 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2933442944}
I20260709 12:25:45.503208  6915 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.500140  7584 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.494210  7580 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.533967  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.534056  7580 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.534763  4752 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:45.535677  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.536248  4752 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:45.536304  7580 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.537149  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.537999  7653 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.538627  7653 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.540894  7653 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:45.508220  7623 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.543998  7623 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:45.545692  7606 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:45.546320  7606 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:45.547384  7606 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.548061  7302 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.548872  7097 raft_consensus.cc:3055] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:45.551436  7619 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.549289  7097 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:45.557348  7097 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.558995  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.559288  7097 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.559772  4676 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:45.560905  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.563397  4672 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:25:45.564528  7627 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.565426  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:25:45.519407  7650 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:25:45.520077  4571 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:45.568197  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 48191.1us
I20260709 12:25:45.568589  7558 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:45.568922  4571 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:45.570370  4571 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:45.487169  7609 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.521903  4747 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:45.522403  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2400) took 1847 ms. Trace:
I20260709 12:25:45.522315  7635 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:45.522634  7575 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:45.526470  7563 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:45.527220  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:45.525009  7496 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.529923  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.531571  4673 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:45.581229  7653 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:45.594426  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.594820  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:45.595134  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:45.578308  7568 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:45.597030  7575 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:45.569195  7558 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.598277  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:45.598925  4748 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.599856  7558 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:45.601159  7653 raft_consensus.cc:697] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:45.602101  7537 raft_consensus.cc:3055] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:45.602645  7537 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:45.602188  7653 consensus_queue.cc:237] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.603586  7537 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.584065  7476 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:45.606264  7575 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:45.606830  5343 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 455) took 2514 ms (client timeout 2199 ms). Trace:
W20260709 12:25:45.607211  5343 rpcz_store.cc:269] 0709 12:25:43.092247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.096690 (+  4443us) service_pool.cc:224] Handling call
0709 12:25:43.561170 (+464480us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b5800003c98. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391eee 000000000036eb39 00007efff34051e0 00007efff34ce0c4 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:25:45.606651 (+2045481us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54004fb0d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:45.606764 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2749759488}
I20260709 12:25:45.529071  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:45.565546  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:45.504436  7533 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:45.608512  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 79580.9us
I20260709 12:25:45.520613  7566 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.549010  7506 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.578075  4676 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:45.570673  7512 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:45.585064  4754 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.585518  4674 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:45.578930  4753 rpcz_store.cc:276] 0709 12:25:43.674912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.690367 (+ 15455us) service_pool.cc:224] Handling call
0709 12:25:43.690813 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.691070 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:43.865861 (+174791us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.170132 (+304271us) log.cc:844] Serialized 14 byte log entry
0709 12:25:45.521853 (+1351721us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b180043ed60. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effee05e4bc 00007effeb3ac48b 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c6455 00007effee4ae060 00007effee4c0716 00007effee4d1287
0709 12:25:45.521984 (+   131us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:45.522039 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:45.522096 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.522190 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.522339 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12095,"prepare.queue_time_us":410779,"prepare.run_cpu_time_us":1486,"prepare.run_wall_time_us":10649,"raft.queue_time_us":547330,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":37815,"spinlock_wait_cycles":24504064,"thread_start_us":537393,"threads_started":4,"wal-append.queue_time_us":401436,"wal-append.run_cpu_time_us":1380,"wal-append.run_wall_time_us":1001427}
I20260709 12:25:45.595146  4673 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:45.573150  4752 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:45.612803  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.614120  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:45.615201  4747 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:25:45.609454  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:45.621583  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:45.619200  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:45.620245  5345 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:45.626968  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.629594  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.630497  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:45.632844  4755 raft_consensus.cc:2393] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:45.632920  4748 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:45.633890  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:45.634326  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:45.636521  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.636735  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.627785  7561 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:45.640039  4753 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:45.634330  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:45.618659  7302 raft_consensus.cc:3055] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:45.644747  7302 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:45.645556  7302 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.646816  7302 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.649969  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:45.650312  4596 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:45.650616  4603 raft_consensus.cc:2376] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6. Re-sending same reply.
W20260709 12:25:45.651767  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.551486  7544 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.617336  7512 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.615994  7636 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:45.666607  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.685428  4568 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:45.584474  7584 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.686709  4568 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:45.660454  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.660678  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:45.660770  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:45.691052  4673 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.659315  4754 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:45.616386  7608 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.661619  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.663882  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.711819  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.711899  4748 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:45.631690  7644 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:45.628624  7607 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:45.665970  7512 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.666563  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:45.666975  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:45.667565  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:45.667794  7544 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.582316  7559 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:45.671509  7634 log.cc:927] Time spent T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.790s	user 0.000s	sys 0.016s
I20260709 12:25:45.652987  7595 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:45.671954  7301 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.672346  7536 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:45.653646  7445 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:25:45.662917  7332 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:45.674609  7632 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.580583  7573 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.638288  7566 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.675753  4598 raft_consensus.cc:2368] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8.
I20260709 12:25:45.676640  7506 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:45.675789  4604 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:45.677850  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.755887  4681 raft_consensus.cc:2393] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:45.735289  7619 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.678155  7496 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.679548  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:45.757838  4677 raft_consensus.cc:2410] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:45.681864  4569 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:45.756186  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 49316.9us
W20260709 12:25:45.758412  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 76636.6us
I20260709 12:25:45.682323  7621 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:45.753989  7565 consensus_meta.cc:287] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.001s
I20260709 12:25:45.687147  7655 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:25:45.685765  7563 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:45.763417  7563 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.706598  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:45.706768  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:45.707795  7471 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:45.686797  7302 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.706643  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.723137  7608 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:45.663352  7533 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.725353  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:45.684059  7636 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:45.708523  7658 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:25:45.712224  7476 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:45.779104  7476 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.779582  7476 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:45.780265  7563 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:45.780948  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2419) took 1413 ms. Trace:
I20260709 12:25:45.685456  7348 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.782696  7301 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.721845  7644 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:45.783197  7348 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.783423  7644 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.786059  7609 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:45.787381  7608 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:45.788053  7639 consensus_meta.cc:287] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.744s	user 0.013s	sys 0.001s
I20260709 12:25:45.728348  7097 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:45.728549  7607 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:45.758936  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.682596  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:45.761762  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.760514  4569 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:45.765712  4752 raft_consensus.cc:2410] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:45.739650  7494 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.748063  7628 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:45.753994  7559 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:45.747828  7595 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.771327  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:45.756713  7553 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.777801  4754 raft_consensus.cc:2393] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 3: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:45.776711  7636 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.789505  4676 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:45.788470  7639 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.762120  7544 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:25:45.713470  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 52994.9us
I20260709 12:25:45.781354  4756 rpcz_store.cc:276] 0709 12:25:44.367528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.448708 (+ 81180us) service_pool.cc:224] Handling call
0709 12:25:44.449087 (+   379us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:44.538387 (+ 89300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.538420 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:44.539182 (+   762us) log.cc:844] Serialized 52 byte log entry
0709 12:25:44.570418 (+ 31236us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:44.570495 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.570605 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.670890 (+1100285us) spinlock_profiling.cc:243] Waited 8.77 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:45.776957 (+106067us) raft_consensus.cc:1685] finished
0709 12:25:45.776982 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.780868 (+  3886us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30781,"raft.queue_time_us":193293,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":24467,"spinlock_wait_cycles":20177792,"thread_start_us":67593,"threads_started":2,"wal-append.queue_time_us":257062}
I20260709 12:25:45.802170  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:45.794809  7621 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:45.775775  7630 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:45.755792  7332 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.766176  7565 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.774647  7302 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.778203  7533 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.171690  7595 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:45.938597  7476 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:45.936625  7632 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.802300  7097 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:46.143891  4673 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:46.211104  7476 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:46.211717  7097 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.212250  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:45.906622  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.213315  4597 raft_consensus.cc:2376] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5. Re-sending same reply.
W20260709 12:25:46.214740  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.998831  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:46.215363  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:46.215775  7416 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.216149  5347 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.216369  7416 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:46.216655  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.218271  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.218737  7416 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:46.219013  4681 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:25:45.905515  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.803200  7348 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:46.223510  7348 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.223991  7348 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:45.934018  7660 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.170s	user 0.001s	sys 0.001s
I20260709 12:25:46.225754  7544 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.226560  7544 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:45.838027  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2474) took 1855 ms. Trace:
I20260709 12:25:46.227751  4679 rpcz_store.cc:276] 0709 12:25:43.983092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.002537 (+ 19445us) service_pool.cc:224] Handling call
0709 12:25:44.002820 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.508152 (+505332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.837720 (+1329568us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b540007ce54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:45.837960 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":927108352}
W20260709 12:25:45.847259  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.229260  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:46.229888  4679 raft_consensus.cc:2376] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8. Re-sending same reply.
I20260709 12:25:46.230767  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.230898  5347 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:46.237548  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:45.825125  4677 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:45.934752  7584 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.800724  7607 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:46.147277  7512 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:45.914101  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:46.246685  4752 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:46.233332  7565 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:45.951170  7494 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:45.995368  7644 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.213801  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.237699  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2490) took 1334 ms. Trace:
I20260709 12:25:46.283069  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.283574  4672 rpcz_store.cc:276] 0709 12:25:44.903621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.527286 (+623665us) service_pool.cc:224] Handling call
0709 12:25:45.527968 (+   682us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:45.564055 (+ 36087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.831689 (+267634us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b54001d2ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:46.237643 (+405954us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80590208}
I20260709 12:25:46.237867  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:46.231112  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.238240  4598 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:46.285463  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.285873  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.286250  4596 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:46.286492  4598 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:46.287037  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.287279  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:46.287779  4598 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:46.287716  4596 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:46.288635  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.181708  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:46.289417  4673 raft_consensus.cc:2368] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8.
W20260709 12:25:46.239585  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:45.814465  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 51625.1us
W20260709 12:25:46.289980  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.219795  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:46.290300  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:46.290704  4681 raft_consensus.cc:2368] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6.
W20260709 12:25:46.242797  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 440681us
I20260709 12:25:46.292182  4569 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:46.288708  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:46.247629  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.293058  4596 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:46.294979  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:46.263330  4568 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:46.242763  7559 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:46.143157  6915 consensus_meta.cc:287] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.003s
I20260709 12:25:46.278932  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:46.238826  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.264125  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:46.334781  4602 raft_consensus.cc:2393] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:45.823441  7553 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.304032  7445 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:46.350510  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.357785  7445 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.358717  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.359424  4681 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:46.338061  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.281443  7476 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:46.264454  7667 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.366153  4596 raft_consensus.cc:2410] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:25:46.316196  4752 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
W20260709 12:25:46.301653  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 60489.6us
I20260709 12:25:46.339709  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.304374  4756 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:46.278306  7595 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:46.293699  7332 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:46.481839  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.500958  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1979) took 1042 ms. Trace:
I20260709 12:25:46.501600  4752 rpcz_store.cc:276] 0709 12:25:45.458285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.803087 (+344802us) service_pool.cc:224] Handling call
0709 12:25:46.500894 (+697807us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.503899  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:46.504753  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2438) took 1547 ms. Trace:
I20260709 12:25:46.508772  4752 rpcz_store.cc:276] 0709 12:25:44.958723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.503623 (+1544900us) service_pool.cc:224] Handling call
0709 12:25:46.504696 (+  1073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.335307  7584 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:46.296928  7565 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:46.420893  7544 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.506904  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.512746  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 466) took 1637 ms. Trace:
I20260709 12:25:46.424255  7445 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:46.297227  7494 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:46.419678  7587 log.cc:927] Time spent T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 2.352s	user 0.012s	sys 0.012s
I20260709 12:25:46.370863  7476 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:46.345742  7639 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:46.510692  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.511847  7635 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.521102  4747 rpcz_store.cc:276] 0709 12:25:44.862633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.210921 (+348288us) service_pool.cc:224] Handling call
0709 12:25:46.499735 (+1288814us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b5400183154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:46.499887 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":671375104}
I20260709 12:25:46.426725  7302 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:46.421769  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.539017  4681 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
W20260709 12:25:46.424314  7566 consensus_meta.cc:287] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.667s	user 0.008s	sys 0.001s
I20260709 12:25:46.420096  7173 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:46.530687  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:46.547186  4596 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.530870  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.550124  4601 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:46.551347  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.552079  4601 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:46.553105  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.349287  7512 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:46.477917  7671 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.337805  7479 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.304255  7628 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:46.352272  7542 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:46.424540  7533 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:46.427138  7553 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:46.535029  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:46.470943  7345 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.464205  7621 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:45.014891  4750 rpcz_store.cc:276] 0709 12:25:43.805359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.355800 (+550441us) service_pool.cc:224] Handling call
0709 12:25:44.390712 (+ 34912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.390929 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:44.415159 (+ 24230us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.996358 (+581199us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b180033aba0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:25:45.014118 (+ 17760us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:45.014174 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:45.014230 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.014311 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.014404 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24054,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":4878,"raft.queue_time_us":195,"spinlock_wait_cycles":69004032,"thread_start_us":17476,"threads_started":1}
I20260709 12:25:46.425712  7510 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:46.356571  6915 consensus_queue.cc:1059] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.174s
I20260709 12:25:46.515720  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:46.373639  4598 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
I20260709 12:25:46.590286  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2439) took 1618 ms. Trace:
I20260709 12:25:46.590631  4756 rpcz_store.cc:276] 0709 12:25:44.971465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.515372 (+1543907us) service_pool.cc:224] Handling call
0709 12:25:46.590230 (+ 74858us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:46.422950  7601 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.432s	user 0.004s	sys 0.006s
I20260709 12:25:46.544715  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:46.546559  7635 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.533339  7565 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:46.530365  7584 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:46.609117  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.554723  5345 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:46.535916  7476 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.372912  7559 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.536317  7644 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:46.541654  7332 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.563076  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:46.621032  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 318280us
I20260709 12:25:46.621358  4601 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:46.622356  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.623078  4601 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:46.566409  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1576) took 1274 ms. Trace:
I20260709 12:25:46.567464  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2442) took 1257 ms. Trace:
I20260709 12:25:46.568300  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2496) took 1030 ms. Trace:
I20260709 12:25:46.568248  4596 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:46.624701  4754 rpcz_store.cc:276] 0709 12:25:45.310468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.711527 (+401059us) service_pool.cc:224] Handling call
0709 12:25:46.567268 (+855741us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b5400308c54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:46.567405 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149685120}
I20260709 12:25:46.623615  4604 rpcz_store.cc:276] 0709 12:25:45.292368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.294272 (+  1904us) service_pool.cc:224] Handling call
0709 12:25:46.566233 (+1271961us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b540013e2d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:46.566335 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734282624}
I20260709 12:25:46.629642  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:46.630254  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:46.630678  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:46.631279  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:46.632091  4601 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:46.565188  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:46.521137  7667 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:46.634150  4677 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:46.570636  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2476) took 1832 ms. Trace:
I20260709 12:25:46.576457  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.578042  5344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 463) took 1760 ms. Trace:
I20260709 12:25:46.535133  7494 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.584671  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:46.639629  4749 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:46.586289  4752 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:25:46.637759  5344 rpcz_store.cc:276] 0709 12:25:44.817888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.818338 (+   450us) service_pool.cc:224] Handling call
0709 12:25:46.573075 (+1754737us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54007116d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:46.577974 (+  4899us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":714096512}
I20260709 12:25:46.537308  7595 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:46.591605  4756 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:46.582362  7479 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:46.643754  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 348833us
I20260709 12:25:46.644408  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.422056  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:46.573412  7671 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.583055  7488 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:25:46.615947  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 242588us
I20260709 12:25:46.624794  4676 rpcz_store.cc:276] 0709 12:25:45.538014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.628721 (+ 90707us) service_pool.cc:224] Handling call
0709 12:25:46.568148 (+939427us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400086454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:46.568245 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240833408}
I20260709 12:25:46.626391  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:46.635070  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:46.655560  4677 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:46.589773  7345 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.591116  7621 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.655239  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1582) took 1002 ms. Trace:
I20260709 12:25:46.662307  4681 rpcz_store.cc:276] 0709 12:25:45.653267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.546992 (+893725us) service_pool.cc:224] Handling call
0709 12:25:46.655177 (+108185us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:46.666705  4599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1544) took 5078 ms (client timeout 1594 ms). Trace:
W20260709 12:25:46.667150  4599 rpcz_store.cc:269] 0709 12:25:41.575942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.627515 (+ 51573us) service_pool.cc:224] Handling call
0709 12:25:46.649927 (+5022412us) spinlock_profiling.cc:243] Waited 2.57 s on lock 0x7b5400275bd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:46.654532 (+  4605us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5917704448}
I20260709 12:25:46.641409  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 457) took 2087 ms. Trace:
I20260709 12:25:46.642825  4756 raft_consensus.cc:606] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:46.670099  4756 raft_consensus.cc:614] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:46.597919  4747 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:46.582729  7628 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:46.621055  7598 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:46.607312  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 421) took 4887 ms. Trace:
I20260709 12:25:46.645928  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:46.645051  4604 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:46.684520  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.636579  4680 rpcz_store.cc:276] 0709 12:25:44.738001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.738128 (+   127us) service_pool.cc:224] Handling call
0709 12:25:46.569811 (+1831683us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b58000d4798. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff348ffc0 00007efff347a929 00007efff347a3d2 00007effec9f1bd4
0709 12:25:46.570020 (+   209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.570202 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.570226 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.570274 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:46.570310 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.570365 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.570573 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6219008}
I20260709 12:25:46.686185  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.685140  4604 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:46.633558  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.655292  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.687536  4596 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:46.688508  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:46.688788  4596 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:46.671257  4752 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:25:46.679986  4747 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:25:46.669625  4754 rpcz_store.cc:276] 0709 12:25:44.554448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.640512 (+2086064us) service_pool.cc:224] Handling call
0709 12:25:46.640857 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.641063 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.641091 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.641134 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:46.641172 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.641231 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.641346 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.645601  4603 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:46.684737  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:46.686903  4676 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:46.701010  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:46.701596  4676 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:46.702559  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:46.682593  4600 rpcz_store.cc:276] 0709 12:25:41.710080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.714098 (+  4018us) service_pool.cc:224] Handling call
0709 12:25:41.714490 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:41.728624 (+ 14134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:41.728655 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.359294 (+1630639us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b2c000014c8. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:25:43.368045 (+  8751us) log.cc:844] Serialized 26 byte log entry
0709 12:25:43.828816 (+460771us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:43.828871 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.828975 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.419107 (+2590132us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:46.597268 (+178161us) raft_consensus.cc:1685] finished
0709 12:25:46.597297 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.597495 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11107,"raft.queue_time_us":11287,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":25166,"spinlock_wait_cycles":7465600,"thread_start_us":388337,"threads_started":1,"wal-append.queue_time_us":552905}
I20260709 12:25:46.703485  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.703713  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 460) took 2144 ms. Trace:
I20260709 12:25:46.704154  4681 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:46.704041  4754 rpcz_store.cc:276] 0709 12:25:44.559787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.699680 (+2139893us) service_pool.cc:224] Handling call
0709 12:25:46.703213 (+  3533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.703387 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.703412 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.703453 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:46.703504 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.703560 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.703660 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.704721  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1591) took 1049 ms. Trace:
I20260709 12:25:46.705605  4681 rpcz_store.cc:276] 0709 12:25:45.655578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.685497 (+1029919us) service_pool.cc:224] Handling call
0709 12:25:46.704670 (+ 19173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.706054  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 461) took 2145 ms. Trace:
I20260709 12:25:46.706424  4754 rpcz_store.cc:276] 0709 12:25:44.560705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.705224 (+2144519us) service_pool.cc:224] Handling call
0709 12:25:46.705544 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.705749 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.705775 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.705815 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:46.705845 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.705906 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.706002 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.696985  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2428) took 1823 ms. Trace:
I20260709 12:25:46.698391  4720 leader_election.cc:400] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8.
I20260709 12:25:46.657120  7512 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:46.722282  7675 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:25:46.722944  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.723728  7544 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:46.697302  7565 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.724565  7348 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.726640  7667 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:46.708164  4720 leader_election.cc:403] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:46.640350  7332 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.709342  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.711440  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:46.712069  7302 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:46.708468  7639 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:46.712898  4604 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 8. Ops: []
I20260709 12:25:46.732530  4680 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:46.738047  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:46.713980  7412 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:46.735832  7494 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:46.721899  7512 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:46.708540  4747 rpcz_store.cc:276] 0709 12:25:44.873504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.544299 (+1670795us) service_pool.cc:224] Handling call
0709 12:25:46.696705 (+152406us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.692788  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:50782 (request call id 19) took 1563 ms. Trace:
I20260709 12:25:46.759207  4756 rpcz_store.cc:276] 0709 12:25:45.115243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.591448 (+1476205us) service_pool.cc:224] Handling call
0709 12:25:46.670525 (+ 79077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.729365  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.730341  4677 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:25:46.760928  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.762027  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:46.762539  4677 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
W20260709 12:25:46.696250  7573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.018s	sys 0.034s
I20260709 12:25:46.731686  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 462) took 2008 ms. Trace:
I20260709 12:25:46.767657  4754 rpcz_store.cc:276] 0709 12:25:44.699861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.707162 (+2007301us) service_pool.cc:224] Handling call
0709 12:25:46.707359 (+   197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.707486 (+   127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.707500 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.707525 (+    25us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:46.707548 (+    23us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.707578 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.707645 (+    67us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.735809  7567 raft_consensus.cc:3055] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:46.690052  7644 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.713474  7628 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:46.738745  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1592) took 1082 ms. Trace:
I20260709 12:25:46.772485  4680 rpcz_store.cc:276] 0709 12:25:45.655784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.685674 (+1029890us) service_pool.cc:224] Handling call
0709 12:25:46.738592 (+ 52918us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:46.772682  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 1.20536e+06us
I20260709 12:25:46.773188  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 439) took 1828 ms. Trace:
I20260709 12:25:46.664552  7488 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:46.779345  7348 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:46.767257  7566 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:46.740895  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:46.781006  4720 leader_election.cc:400] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8.
I20260709 12:25:46.781446  5312 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:46.743901  7677 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:46.737504  7302 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.747324  7595 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:46.749725  7584 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:46.726634  7671 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:46.750751  7345 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:46.754186  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1580) took 1119 ms. Trace:
I20260709 12:25:46.798237  4602 rpcz_store.cc:276] 0709 12:25:45.634725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.637618 (+  2893us) service_pool.cc:224] Handling call
0709 12:25:46.753979 (+1116361us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b540032afd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:46.754123 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1314145152}
I20260709 12:25:46.803131  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 432) took 2437 ms. Trace:
I20260709 12:25:46.754786  7621 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:46.755229  7639 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:46.757023  7635 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:46.736533  7593 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.743126  7542 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:25:46.767334  4681 raft_consensus.cc:2393] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:46.769316  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2469) took 2890 ms. Trace:
I20260709 12:25:46.770288  7512 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:46.770386  7607 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:46.771611  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2433) took 1888 ms. Trace:
I20260709 12:25:46.989707  4756 rpcz_store.cc:276] 0709 12:25:44.883290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.770858 (+1887568us) service_pool.cc:224] Handling call
0709 12:25:46.771152 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.771316 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.771340 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.771379 (+    39us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:46.771409 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.771462 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.771558 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.991139  4756 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:46.773751  4719 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:25:46.992062  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 218361us
I20260709 12:25:46.774428  4754 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:46.993363  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2427) took 2120 ms. Trace:
I20260709 12:25:46.993755  4754 rpcz_store.cc:276] 0709 12:25:44.873067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.773984 (+1900917us) service_pool.cc:224] Handling call
0709 12:25:46.774257 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.993120 (+218863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.993309 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.995220  4754 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:46.720587  7636 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:46.773591  4677 rpcz_store.cc:276] 0709 12:25:44.945050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.772435 (+1827385us) service_pool.cc:224] Handling call
0709 12:25:46.772704 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.772864 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.772887 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.772928 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:46.772985 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.773035 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.773133 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.776969  7332 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:46.772248  7567 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:46.771946  7573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.018s	sys 0.035s
I20260709 12:25:46.778003  7537 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.782138  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:47.001713  4647 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:25:47.014879  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 232778us
I20260709 12:25:47.015762  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:47.016163  4720 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:46.782537  4603 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:46.787309  5312 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:47.017805  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.019273  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:47.019923  4603 raft_consensus.cc:2376] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5. Re-sending same reply.
I20260709 12:25:47.020887  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:46.789806  4676 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:47.023182  4676 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:25:47.023962  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2498) took 1485 ms. Trace:
I20260709 12:25:47.024355  4676 rpcz_store.cc:276] 0709 12:25:45.538687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.022070 (+1483383us) service_pool.cc:224] Handling call
0709 12:25:47.022719 (+   649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.023606 (+   887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.023900 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.741781  7412 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:46.791116  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49236.2us
W20260709 12:25:46.877092  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.032462  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.885850  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.033638  4748 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:46.803766  4674 rpcz_store.cc:276] 0709 12:25:44.366131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.366335 (+   204us) service_pool.cc:224] Handling call
0709 12:25:44.366875 (+   540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.618496 (+1251621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.802778 (+1184282us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54002df854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:46.803046 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2802115840}
I20260709 12:25:46.767518  7479 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:46.783288  7302 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:46.986343  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 226608us
I20260709 12:25:47.038048  4676 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:47.039844  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2500) took 1497 ms. Trace:
I20260709 12:25:47.040509  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:46.791949  7472 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:46.998503  7677 raft_consensus.cc:2764] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:25:46.796273  7202 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:46.998886  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 440) took 1951 ms. Trace:
I20260709 12:25:47.058022  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.003876  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2436) took 2119 ms. Trace:
I20260709 12:25:47.004323  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1969) took 1965 ms. Trace:
I20260709 12:25:47.059281  4754 rpcz_store.cc:276] 0709 12:25:45.038796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.994637 (+1955841us) service_pool.cc:224] Handling call
0709 12:25:46.995049 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.995565 (+   516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.004267 (+  8702us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.006709  4677 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:47.060616  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 441) took 1577 ms. Trace:
I20260709 12:25:47.060940  4677 rpcz_store.cc:276] 0709 12:25:45.483225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.006269 (+1523044us) service_pool.cc:224] Handling call
0709 12:25:47.006554 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.060370 (+ 53816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.060559 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.986363  4675 rpcz_store.cc:276] 0709 12:25:43.878447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.879293 (+   846us) service_pool.cc:224] Handling call
0709 12:25:43.879625 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.940376 (+ 60751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.940421 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.988800 (+ 48379us) log.cc:844] Serialized 26 byte log entry
0709 12:25:46.235632 (+2246832us) spinlock_profiling.cc:243] Waited 8.91 ms on lock 0x7b180024cb20. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:25:46.618115 (+382483us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:46.618184 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.618367 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.764163 (+145796us) raft_consensus.cc:1685] finished
0709 12:25:46.764182 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.769245 (+  5063us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113548,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":15869,"spinlock_wait_cycles":20502784,"thread_start_us":382018,"threads_started":1,"wal-append.queue_time_us":2676170}
I20260709 12:25:47.026175  4678 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:46.983461  7345 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.030857  4647 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:46.780238  7566 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:47.034731  4674 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:25:47.035308  7412 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:46.997622  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.074203  4602 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:47.040215  4676 rpcz_store.cc:276] 0709 12:25:45.542850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.025222 (+1482372us) service_pool.cc:224] Handling call
0709 12:25:47.025534 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.038430 (+ 12896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.039789 (+  1359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.988345  7593 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.049080  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 443) took 1476 ms. Trace:
I20260709 12:25:47.085536  4673 rpcz_store.cc:276] 0709 12:25:45.573026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.044622 (+1471596us) service_pool.cc:224] Handling call
0709 12:25:47.048593 (+  3971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.048759 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.048785 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:47.048824 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:47.048856 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.048904 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.049024 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.087387  4673 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:47.058351  4681 rpcz_store.cc:276] 0709 12:25:45.047305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.998149 (+1950844us) service_pool.cc:224] Handling call
0709 12:25:46.998426 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.998580 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.998606 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.998645 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:46.998674 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.998726 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.998830 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.058859  4756 rpcz_store.cc:276] 0709 12:25:44.884583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.990662 (+2106079us) service_pool.cc:224] Handling call
0709 12:25:46.990975 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.003634 (+ 12659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.003817 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.047009  7445 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:47.068056  4674 pending_rounds.cc:85] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 5
I20260709 12:25:47.035928  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:47.068488  4677 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:25:47.036911  7332 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:47.040705  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:47.064733  4647 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:47.042892  7472 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:47.097368  7332 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.097654  7472 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:47.097846  7332 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:47.039249  7559 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:47.037426  7595 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:47.039143  7573 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:47.100289  7573 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:47.101161  7573 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.101881  4602 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:47.089072  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.058642  4571 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:25:47.104894  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 46418.4us
I20260709 12:25:47.106108  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:47.107007  7536 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:47.107211  4605 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 7. Ops: [5.6-5.6]
I20260709 12:25:47.108881  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1570) took 1819 ms. Trace:
W20260709 12:25:47.109042  4755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1965) took 2071 ms (client timeout 2087 ms). Trace:
W20260709 12:25:47.109438  4755 rpcz_store.cc:269] 0709 12:25:45.037533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.529676 (+492143us) service_pool.cc:224] Handling call
0709 12:25:47.108873 (+1579197us) spinlock_profiling.cc:243] Waited 39.8 ms on lock 0x7b5400118354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:47.108991 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91651328}
I20260709 12:25:47.110458  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2484) took 2301 ms. Trace:
I20260709 12:25:47.110543  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.028211  7476 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.110898  4678 rpcz_store.cc:276] 0709 12:25:44.809433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.790809 (+1981376us) service_pool.cc:224] Handling call
0709 12:25:47.025823 (+235014us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.062057 (+ 36234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.110292 (+ 48235us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b540019f850. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:47.110405 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":539623168}
I20260709 12:25:47.112792  7476 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:47.046044  7628 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:47.112696  7412 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.113657  7628 raft_consensus.cc:697] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:47.114646  7628 consensus_queue.cc:237] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.093345  4674 pending_rounds.cc:107] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:25:47.117962  4674 raft_consensus.cc:2889] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:47.118647  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2503) took 1546 ms. Trace:
I20260709 12:25:47.118954  4674 rpcz_store.cc:276] 0709 12:25:45.572729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.034265 (+1461536us) service_pool.cc:224] Handling call
0709 12:25:47.034496 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.118400 (+ 83904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.118594 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.093902  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1595) took 1435 ms. Trace:
I20260709 12:25:47.119814  4677 rpcz_store.cc:276] 0709 12:25:45.658238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.067919 (+1409681us) service_pool.cc:224] Handling call
0709 12:25:47.068257 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.093603 (+ 25346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.093847 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.058279  7202 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.120793  7202 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:47.094816  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 54186us
I20260709 12:25:47.125771  7677 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:47.126240  7677 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.126765  7677 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:47.127367  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1584) took 1489 ms. Trace:
I20260709 12:25:47.127733  4597 rpcz_store.cc:276] 0709 12:25:45.637863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.660399 (+ 22536us) service_pool.cc:224] Handling call
0709 12:25:47.127209 (+1466810us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b54003d0254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:47.127309 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1118808960}
I20260709 12:25:47.094901  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:25:47.093677  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57861us
W20260709 12:25:47.132843  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.107192  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:47.113121  4747 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:47.094094  4675 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:47.126125  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:47.109228  4605 rpcz_store.cc:276] 0709 12:25:45.289750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.289985 (+   235us) service_pool.cc:224] Handling call
0709 12:25:45.357092 (+ 67107us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.107697 (+1750605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.108588 (+   891us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b5400140854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:47.108827 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4023961984}
I20260709 12:25:47.074945  7566 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.057991  7683 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:25:47.133447  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.137019  4597 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 12:25:47.142378  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.138623  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 43816us
W20260709 12:25:47.139220  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 351948us
W20260709 12:25:47.126199  4753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 470) took 1577 ms (client timeout 1875 ms). Trace:
I20260709 12:25:47.143601  4568 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:47.143635  4753 rpcz_store.cc:269] 0709 12:25:45.549200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.636439 (+ 87239us) service_pool.cc:224] Handling call
0709 12:25:47.126046 (+1489607us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b540004cc54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:47.126142 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5565184}
I20260709 12:25:47.144842  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:47.145380  4753 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:25:47.074339  7668 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:47.142305  4605 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:25:47.137979  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:47.148073  4645 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:47.096609  7445 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.173925  7533 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:47.091150  7488 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:47.175200  4605 raft_consensus.cc:1240] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:47.136225  7667 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:47.135905  7631 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:47.148437  7595 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:47.075796  7345 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.175297  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:47.197129  4645 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:47.087138  7619 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:25:47.139609  7471 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:47.182029  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:47.141542  7671 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:47.198220  4605 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:47.178901  7575 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:47.166801  7627 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:47.176565  7559 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:47.205147  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2494) took 2232 ms. Trace:
I20260709 12:25:47.178144  7445 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:47.180001  7682 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:47.177294  7683 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:47.170279  7540 consensus_queue.cc:1059] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:25:47.205610  4680 rpcz_store.cc:276] 0709 12:25:44.972410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.773201 (+1800791us) service_pool.cc:224] Handling call
0709 12:25:46.773504 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.783954 (+ 10450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.783998 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.790372 (+  6374us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.143024 (+352652us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:47.143060 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.143124 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.204859 (+ 61735us) raft_consensus.cc:1685] finished
0709 12:25:47.204895 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.205081 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5240,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":10579,"thread_start_us":145354,"threads_started":1,"wal-append.queue_time_us":219334}
I20260709 12:25:47.208071  7671 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:47.175611  7332 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.095897  7639 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.201601  7488 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:25:47.227442  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 96787.9us
I20260709 12:25:47.207415  7471 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:47.237471  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 89650.1us
I20260709 12:25:47.211941  7682 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.226140  7512 raft_consensus.cc:3055] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:25:47.261667  7685 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:25:47.263583  7345 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:47.264685  7695 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:47.225297  7559 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:25:47.212713  7575 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.270098  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 458) took 2715 ms. Trace:
I20260709 12:25:47.289373  4749 rpcz_store.cc:276] 0709 12:25:44.555103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.643565 (+2088462us) service_pool.cc:224] Handling call
0709 12:25:46.668656 (+ 25091us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.669004 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.669032 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.699270 (+ 30238us) log.cc:844] Serialized 26 byte log entry
0709 12:25:46.977765 (+278495us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:46.977808 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.977877 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.269854 (+291977us) raft_consensus.cc:1685] finished
0709 12:25:47.269874 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.270032 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22849,"raft.queue_time_us":496,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":15005,"thread_start_us":232207,"threads_started":1,"wal-append.queue_time_us":345049}
I20260709 12:25:47.242828  7682 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:47.245589  7607 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:47.225740  7202 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.303748  4602 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:47.279366  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.278483  4605 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:47.283528  4720 leader_election.cc:400] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6.
I20260709 12:25:47.304736  4720 leader_election.cc:403] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:25:47.248364  7512 raft_consensus.cc:740] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:47.279902  7595 raft_consensus.cc:697] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:47.278985  7621 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:47.278785  7173 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.722s	user 0.007s	sys 0.004s
I20260709 12:25:47.307686  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.308137  7512 consensus_queue.cc:260] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.308924  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.309144  4751 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:47.309159  7512 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:47.306141  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:47.310025  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.310606  4603 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:47.311026  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.311662  4605 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:47.312490  4753 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:47.312736  4605 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:47.279199  7635 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.313897  7696 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.313828  7635 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.314582  7696 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.316449  7635 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:47.316968  7635 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6.
I20260709 12:25:47.308947  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:47.320466  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:47.321506  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:47.322041  7684 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:25:47.322707  7097 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:25:47.322921  7686 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:25:47.323084  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:47.340545  4749 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.323494  4605 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:47.341025  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.341775  4754 raft_consensus.cc:2393] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:47.324198  7471 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.323788  7593 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:47.323887  7635 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:47.321964  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:47.326184  7472 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.324084  7412 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:47.276222  7683 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.329696  7575 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:47.333093  7607 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:47.347441  7644 consensus_meta.cc:287] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.002s
I20260709 12:25:47.335201  7510 raft_consensus.cc:3055] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:47.335688  7302 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.336782  7621 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.407555  4601 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:25:47.338871  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.342279  4605 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:47.286034  7668 raft_consensus.cc:697] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:47.342523  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:47.344475  4603 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:47.346608  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:47.348254  5344 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:47.349179  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:47.353731  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.357402  4719 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43864 dropped due to backpressure: service queue is full with 50 items
I20260709 12:25:47.327315  7636 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:47.417452  4749 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
W20260709 12:25:47.362484  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.390424  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:47.426424  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:47.427273  4680 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:47.428762  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:47.429545  4672 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:47.392078  5345 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 471) took 1710 ms (client timeout 2217 ms). Trace:
I20260709 12:25:47.393510  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2423) took 2541 ms. Trace:
I20260709 12:25:47.431617  4747 rpcz_store.cc:276] 0709 12:25:44.852531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.775430 (+1922899us) service_pool.cc:224] Handling call
0709 12:25:47.112883 (+337453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.133833 (+ 20950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.393229 (+259396us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b5400118350. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:47.393433 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":743180544}
I20260709 12:25:47.301110  7690 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:47.338711  7696 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:47.304642  7619 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 12:25:47.404734  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 204759us
I20260709 12:25:47.414250  4753 raft_consensus.cc:2393] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:47.358271  7468 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:47.418402  4754 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:47.419179  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 72656.2us
I20260709 12:25:47.433969  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:47.434690  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.436482  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:47.437393  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:47.439870  4681 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:25:47.441344  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:47.446708  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 206982us
I20260709 12:25:47.425691  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:47.430303  5345 rpcz_store.cc:269] 0709 12:25:45.682151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.682336 (+   185us) service_pool.cc:224] Handling call
0709 12:25:47.391857 (+1709521us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b540020dad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:47.392008 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1740184320}
W20260709 12:25:47.447466  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 207924us
I20260709 12:25:47.445426  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:47.451736  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:47.400311  7510 raft_consensus.cc:740] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:47.499604  4676 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:47.500006  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:47.405241  7644 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:47.504113  4645 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:47.508661  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 63298.1us
I20260709 12:25:47.508123  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:47.504247  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:47.504496  4677 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:47.510586  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:47.405519  7097 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:47.506512  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.454324  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:47.506624  4673 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:47.511014  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:47.511847  5340 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.511746  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:47.648579  4677 raft_consensus.cc:2393] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:47.501281  7496 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.513294  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.652410  4601 raft_consensus.cc:2393] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:47.514736  7472 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:47.516412  7567 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.518939  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:25:47.531141  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.577319  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1589) took 1268 ms. Trace:
I20260709 12:25:47.578536  4672 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:47.577301  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2420) took 3208 ms. Trace:
I20260709 12:25:47.577939  4679 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:47.445552  7667 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:47.585213  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 85811us
W20260709 12:25:47.585258  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 77399.3us
I20260709 12:25:47.585628  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:47.401671  7607 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.586710  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1592) took 1346 ms. Trace:
I20260709 12:25:47.730878  4602 rpcz_store.cc:276] 0709 12:25:46.240238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.102738 (+862500us) service_pool.cc:224] Handling call
0709 12:25:47.103026 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.304411 (+201385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.586450 (+282039us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400300a54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:47.586650 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461174016}
I20260709 12:25:47.731434  4676 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:47.500016  7700 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:47.589767  4748 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2441) took 2280 ms (client timeout 2366 ms). Trace:
I20260709 12:25:47.590006  4752 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:47.590673  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:47.444594  7173 raft_consensus.cc:2764] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:47.444608  7635 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:47.592229  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 82519.4us
I20260709 12:25:47.594641  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:47.595901  4599 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:25:47.453840  5342 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:47.498833  7593 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:47.455696  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:47.765682  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:47.765821  5341 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:47.766297  4674 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:47.767113  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:47.599063  4569 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:47.674822  7510 consensus_queue.cc:260] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:47.769934  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 170954us
I20260709 12:25:47.770366  7510 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:47.771076  4569 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:47.772223  4569 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:47.543349  7628 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:47.446372  7690 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:47.646479  4755 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:47.577394  7619 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:47.778964  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.764082  5314 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:47.764662  7635 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:47.755241  7202 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:47.655066  7668 consensus_queue.cc:237] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.646960  4673 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:47.651444  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:47.789882  4677 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:47.652724  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.790334  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:47.791256  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.646322  7625 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.507159  7565 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 12:25:47.674144  7496 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:47.693357  7698 log.cc:927] Time spent T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:25:47.697660  7705 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.655570  4596 rpcz_store.cc:276] 0709 12:25:46.239447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.691972 (+452525us) service_pool.cc:224] Handling call
0709 12:25:46.692264 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.692658 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.692688 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.700894 (+  8206us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.046769 (+345875us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:47.046823 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.046913 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.508135 (+461222us) raft_consensus.cc:1685] finished
0709 12:25:47.508157 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.508294 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13809,"raft.queue_time_us":4385,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":7294,"spinlock_wait_cycles":178048,"thread_start_us":300860,"threads_started":1,"wal-append.queue_time_us":357146}
I20260709 12:25:47.702663  7621 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:47.680750  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.802937  4601 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:47.783610  7505 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:47.644744  7584 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:25:47.675094  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:47.807564  4672 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:47.575819  7595 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:47.692452  7699 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.334s	user 0.000s	sys 0.001s
I20260709 12:25:47.705272  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 439) took 1265 ms. Trace:
I20260709 12:25:47.788408  7635 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:47.683883  5343 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:47.419744  4719 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43864 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2454)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1988)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1993)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1987)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2452)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1991)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2444)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2466)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2463)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2447)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1994)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2443)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2456)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2467)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1990)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2473)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2470)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1989)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2468)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1996)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1984)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2458)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1997)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2455)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1992)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2445)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1976)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1977)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1978)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 468)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1981)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 469)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 472)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2448)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2449)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2450)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2451)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2457)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2460)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2461)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1983)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1985)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1986)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2469)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2471)
I20260709 12:25:47.599728  7488 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:47.656765  4750 rpcz_store.cc:276] 0709 12:25:44.369076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.619414 (+2250338us) service_pool.cc:224] Handling call
0709 12:25:46.644201 (+ 24787us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.644578 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.644605 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.648572 (+  3967us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.069103 (+420531us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:47.069153 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.069231 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.577059 (+507828us) raft_consensus.cc:1685] finished
0709 12:25:47.577082 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.577228 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6719,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":10326,"thread_start_us":379934,"threads_started":1,"wal-append.queue_time_us":478589}
W20260709 12:25:47.700754  7542 consensus_meta.cc:287] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.712s	user 0.018s	sys 0.001s
I20260709 12:25:47.591818  7636 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:47.654264  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:47.826188  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:47.827060  5346 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
W20260709 12:25:47.662243  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.676791  5345 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:25:47.647346  7302 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:47.537113  7468 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.833978  7496 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:47.834694  7559 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:25:47.678237  7701 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 12:25:47.654559  7644 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:47.601444  7671 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
I20260709 12:25:47.822962  7683 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:47.600098  7097 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.729321  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:47.644591  7540 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:25:47.644261  7533 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 12:25:47.450475  7627 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:47.728255  5344 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:47.738512  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.749184  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.751250  4754 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
W20260709 12:25:47.752270  4748 rpcz_store.cc:269] 0709 12:25:45.309524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.724939 (+415415us) service_pool.cc:224] Handling call
0709 12:25:47.589529 (+1864590us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400117954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:47.589701 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2903563648}
I20260709 12:25:47.753892  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:47.871840  4603 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:47.757293  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.872258  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:47.872995  4753 raft_consensus.cc:2410] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:25:47.758073  4751 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 465) took 2918 ms (client timeout 1525 ms). Trace:
I20260709 12:25:47.864171  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:47.802330  7625 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:47.873617  4751 rpcz_store.cc:269] 0709 12:25:44.830067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.888302 (+ 58235us) service_pool.cc:224] Handling call
0709 12:25:47.748036 (+2859734us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b54001b0654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:47.748205 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5478134656}
I20260709 12:25:47.759096  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:47.759248  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.876482  4602 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
W20260709 12:25:47.728191  7566 consensus_meta.cc:287] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.553s	user 0.005s	sys 0.005s
I20260709 12:25:47.715448  7697 consensus_queue.cc:1059] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 12:25:47.777150  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.849830  7606 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:47.648872  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 137919us
I20260709 12:25:47.884917  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:47.886081  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:47.885270  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:47.797756  4674 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:47.888315  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:47.889101  4674 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:47.783768  7619 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.801240  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:47.897244  4596 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:47.853287  7644 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:47.803959  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:47.810416  4597 rpcz_store.cc:276] 0709 12:25:46.440135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.171608 (+731473us) service_pool.cc:224] Handling call
0709 12:25:47.179692 (+  8084us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.180059 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.180087 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.180521 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.421979 (+241458us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:47.422016 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.422094 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.705058 (+282964us) raft_consensus.cc:1685] finished
0709 12:25:47.705079 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.705212 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1564,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":1998,"thread_start_us":235361,"threads_started":1,"wal-append.queue_time_us":406996}
I20260709 12:25:47.590958  7598 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:47.815165  4680 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:47.733712  7667 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:47.760301  7700 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.820518  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.916934  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.918010  4750 raft_consensus.cc:2393] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 7: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:47.780642  7593 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.787521  7575 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:25:47.807045  7505 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:47.820704  7682 consensus_meta.cc:287] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.512s	user 0.005s	sys 0.005s
I20260709 12:25:47.850780  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:47.848402  7542 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8.
I20260709 12:25:47.835484  7468 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:47.593619  7677 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:47.838644  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.910732  7598 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:47.934821  4568 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:47.835654  7445 consensus_meta.cc:287] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.612s	user 0.009s	sys 0.002s
W20260709 12:25:47.865592  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 93237us
I20260709 12:25:47.866834  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 459) took 3307 ms. Trace:
I20260709 12:25:47.868250  4719 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:47.936928  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:47.872020  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:47.937513  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 69373.6us
I20260709 12:25:47.938107  4748 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:47.939006  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.939412  4748 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:47.880115  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.880367  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.939975  4751 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:47.940168  4754 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:47.940816  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.940924  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.941478  4754 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:47.924278  5341 raft_consensus.cc:3055] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:47.793422  5312 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:47.835963  7607 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:47.806245  7705 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:47.874718  4679 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:47.895246  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:47.785471  7536 raft_consensus.cc:2764] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:47.899914  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:47.903600  4674 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:47.913722  4675 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:47.880048  7482 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:25:47.886128  7537 consensus_meta.cc:287] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.868s	user 0.013s	sys 0.000s
I20260709 12:25:47.874557  7627 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:47.883736  7566 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.923398  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.963124  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:47.927474  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 56658.5us
I20260709 12:25:47.927847  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:47.931531  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.899482  7621 raft_consensus.cc:2764] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:47.899915  7619 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:47.905463  7584 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:47.899987  7671 consensus_queue.cc:1243] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:25:47.910890  7644 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:47.899726  7639 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:47.937009  4752 rpcz_store.cc:276] 0709 12:25:44.559066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.691281 (+2132215us) service_pool.cc:224] Handling call
0709 12:25:47.589768 (+898487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.752439 (+162671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.866649 (+114210us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b5400117950. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:47.866776 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2065542912}
I20260709 12:25:47.979858  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.880630  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:47.942377  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.924729  5340 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 12:25:47.927603  7505 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:47.943993  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 150690us
I20260709 12:25:47.930697  7682 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:47.948673  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:47.988286  5312 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:47.988667  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:47.989163  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:47.949468  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:47.951777  4753 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:47.900306  7476 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 12:25:47.953981  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 66635.8us
I20260709 12:25:47.989778  4751 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.990491  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.989756  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:47.956969  4756 raft_consensus.cc:2393] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:47.960633  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:47.991858  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:47.992342  4679 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:47.933627  7488 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:47.947413  7636 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:47.994547  7593 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:47.995411  7711 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:25:47.995836  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:47.997061  7621 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:47.998584  7621 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:47.999879  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:48.001372  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:48.004030  7719 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:25:48.004357  4679 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:47.958066  7704 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:25:48.005682  4679 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:48.006894  7700 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.007958  7097 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:48.009780  7677 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:48.010993  7667 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.011613  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1590) took 1771 ms. Trace:
I20260709 12:25:48.011987  4599 rpcz_store.cc:276] 0709 12:25:46.239743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.693246 (+453503us) service_pool.cc:224] Handling call
0709 12:25:46.693659 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.775896 (+1082237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.011370 (+235474us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b5400343854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:48.011557 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2002606336}
I20260709 12:25:47.977231  7723 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:25:48.015369  7332 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:48.015833  4707 heartbeater.cc:646] Failed to heartbeat to 127.3.89.254:36317 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.3.89.254:36317 timed out after 15.000s (SENT)
I20260709 12:25:48.016086  7682 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:47.977789  7720 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:48.016729  7720 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.017102  7720 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.018811  7566 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:48.021942  7696 consensus_queue.cc:1059] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:48.022856  7696 raft_consensus.cc:3055] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:48.023188  7696 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:48.024076  7696 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:47.986006  7584 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:48.026392  7625 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:47.942975  7445 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:48.026491  7584 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.028515  7445 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:48.029203  7512 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:47.986358  5340 pending_rounds.cc:85] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 5
I20260709 12:25:48.029623  7512 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:48.004093  7713 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:25:48.030524  7512 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.029891  5340 pending_rounds.cc:107] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 5.6
I20260709 12:25:47.969985  4571 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:47.971728  4681 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:48.032316  4749 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:47.974134  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:47.943480  5341 raft_consensus.cc:740] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:47.957443  7468 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:48.005316  7705 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:47.961544  7607 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:47.979187  7537 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:47.979058  5347 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:47.933036  7714 consensus_queue.cc:1059] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:25:47.958518  7482 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:48.042097  7345 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:48.048790  7607 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:48.054347  7345 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.987286  4677 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:48.055130  4605 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:25:48.055406  4677 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:47.970994  7612 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.991494  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:47.957134  4674 raft_consensus.cc:3055] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:48.058173  4674 raft_consensus.cc:740] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:48.030231  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.059315  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.059607  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.060352  4752 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:47.967897  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:48.061825  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 94006.8us
W20260709 12:25:48.032375  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 62569.8us
I20260709 12:25:48.064383  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:48.036494  4751 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:47.946724  7542 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
W20260709 12:25:48.042903  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 54685.7us
W20260709 12:25:48.043908  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 52103.6us
I20260709 12:25:48.066658  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:48.067754  4720 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:48.025365  7696 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:47.980933  4754 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:48.071990  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.072317  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.072553  4603 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.072847  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2455) took 1060 ms. Trace:
I20260709 12:25:48.039393  5340 raft_consensus.cc:2917] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Aborting config change with OpId 5.6: Aborted: Op aborted by new leader
I20260709 12:25:48.069733  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:48.054777  7345 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:48.064361  4571 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:47.999428  7621 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:48.070562  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.103835  4751 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:48.104139  4675 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:48.070489  5314 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:48.046291  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 223302us
I20260709 12:25:48.105633  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:48.107370  4645 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:48.041602  7683 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:48.073247  4748 rpcz_store.cc:276] 0709 12:25:47.012897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.991291 (+978394us) service_pool.cc:224] Handling call
0709 12:25:48.072791 (+ 81500us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:48.108911  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.109051  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:48.109486  4748 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:48.074994  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:48.110313  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.111058  4748 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:25:48.056216  4677 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.112363  4677 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.045605  7097 raft_consensus.cc:2764] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:25:47.977559  4569 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:48.079101  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.100236  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.100446  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.047072  7566 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:48.050766  7512 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:48.079773  4674 consensus_queue.cc:260] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.120240  4674 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:48.120623  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 50242.7us
I20260709 12:25:48.100744  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:48.121794  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 46986.7us
I20260709 12:25:48.122223  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.122556  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:48.058320  7705 raft_consensus.cc:3055] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:48.128803  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:48.129634  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:48.078459  7445 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:48.078302  7714 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:48.114880  4602 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.117110  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:48.135012  4677 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:48.047222  7677 raft_consensus.cc:3055] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:25:48.122997  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 58746.3us
I20260709 12:25:48.100128  5340 raft_consensus.cc:2889] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:48.101449  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.136020  7625 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:48.137689  4752 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:48.137974  7625 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:48.121742  7332 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:48.138392  7625 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.138629  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:48.139794  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.140533  4752 raft_consensus.cc:2410] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:25:48.124293  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 54783.1us
I20260709 12:25:48.141994  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:48.142055  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.142702  7606 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:48.143257  7606 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.143605  7606 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:48.145750  7542 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:48.146183  7542 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.146056  7683 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.147373  7639 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:48.148042  7639 consensus_queue.cc:1243] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:48.058038  7631 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.126091  4603 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:48.126945  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:48.128007  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:48.142102  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 164610us
I20260709 12:25:48.150221  4672 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:48.126899  4647 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:48.130934  4680 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:48.151005  4569 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:48.125697  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2445) took 1115 ms. Trace:
I20260709 12:25:48.152165  4754 rpcz_store.cc:276] 0709 12:25:47.010614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.100499 (+1089885us) service_pool.cc:224] Handling call
0709 12:25:48.125638 (+ 25139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:48.128911  7705 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:48.120088  7704 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:48.154381  7593 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:48.155079  4645 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:48.155224  7631 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.156113  7734 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:48.156731  7734 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.156809  4645 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:48.157135  7734 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.157603  7332 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:48.157998  7332 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.158337  7332 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.076190  7494 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:48.162854  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:48.163537  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:48.163846  4603 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:48.164464  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:48.136173  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:48.142213  7703 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEARNER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:48.142755  4596 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.146586  7542 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:48.156145  7682 raft_consensus.cc:2764] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:48.103478  7621 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:48.130255  4751 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.125758  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:48.160580  4647 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:48.141517  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.180972  4752 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:48.176918  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:48.103071  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 168312us
I20260709 12:25:48.165059  4672 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:48.100248  7612 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:48.182217  4568 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:48.182549  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:48.183614  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:48.184665  7621 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.185755  4598 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:48.164023  4604 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:48.166023  7731 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:48.123980  7726 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:48.130378  7667 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:48.054853  5341 consensus_queue.cc:260] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:48.124250  7202 consensus_queue.cc:1059] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:25:48.204700  5340 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:48.193210  7445 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:48.205348  7302 consensus_queue.cc:1059] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:25:48.171372  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.171999  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:48.210947  4676 raft_consensus.cc:2410] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:48.206169  4752 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50782
I20260709 12:25:48.174005  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:48.212028  4752 raft_consensus.cc:606] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:48.212549  4752 raft_consensus.cc:614] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:48.134294  7482 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.214239  7202 raft_consensus.cc:3055] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:48.214907  4751 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:48.192975  7593 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:25:48.188156  7598 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.134146  7412 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:48.217974  7412 raft_consensus.cc:697] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:48.219123  7598 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.219049  7412 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:48.220659  4751 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 12:25:48.221617  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1978) took 3117 ms. Trace:
I20260709 12:25:48.137328  7595 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.228511  7542 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.170914  7704 raft_consensus.cc:697] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:48.128288  4674 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:48.185881  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:48.208709  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:48.230968  4680 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:48.137229  7677 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:48.187307  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:48.125329  7723 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:25:48.190011  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:48.124547  7717 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
W20260709 12:25:48.242882  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 52930.6us
I20260709 12:25:48.194430  4748 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:48.198191  4596 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:48.202507  4602 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:48.165035  7584 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:48.123371  7619 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:48.222318  7332 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.193778  7726 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:48.177388  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:48.134603  7700 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:48.231921  4754 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:48.230618  4751 rpcz_store.cc:276] 0709 12:25:45.104054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.220035 (+3115981us) service_pool.cc:224] Handling call
0709 12:25:48.220400 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:48.221209 (+   809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.221518 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:48.154342  7705 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:48.237898  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 63950.7us
I20260709 12:25:48.178762  7537 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:48.214732  7482 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.249228  7471 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:48.243841  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1976) took 3182 ms. Trace:
I20260709 12:25:48.293992  7584 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:48.124754  7488 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.309791  4748 rpcz_store.cc:276] 0709 12:25:45.061220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.193928 (+3132708us) service_pool.cc:224] Handling call
0709 12:25:48.194238 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.243610 (+ 49372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.243790 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:48.244517  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 57274.3us
I20260709 12:25:48.311100  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:48.298101  4752 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:48.312091  4647 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:48.244984  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:48.246891  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:48.314641  4596 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:48.315872  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.316705  7598 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:48.317167  7598 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.317593  7598 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:48.247025  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:48.121145  7607 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.242904  7202 raft_consensus.cc:740] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:25:48.320528  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 73611.4us
I20260709 12:25:48.321043  7700 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:48.249188  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 136008us
I20260709 12:25:48.321509  7700 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:48.321930  7700 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.322252  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:48.323298  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:48.302062  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:48.282768  7636 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.282415  7627 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.283414  7445 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.216132  7593 consensus_queue.cc:1243] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:48.285133  5343 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:25:48.286001  7345 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.291275  7505 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.327178  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:48.301012  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 468) took 3095 ms. Trace:
I20260709 12:25:48.323988  7621 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.330255  4748 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:48.243464  7732 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:48.330684  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.299165  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.194104  7682 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:48.332094  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1977) took 3260 ms. Trace:
I20260709 12:25:48.373638  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.374038  4752 rpcz_store.cc:276] 0709 12:25:45.071745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.241025 (+3169280us) service_pool.cc:224] Handling call
0709 12:25:48.297848 (+ 56823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.331803 (+ 33955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.332014 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:48.375089  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:48.375694  4752 raft_consensus.cc:1240] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:48.376359  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 469) took 2881 ms. Trace:
I20260709 12:25:48.352514  7173 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:48.376906  4752 rpcz_store.cc:276] 0709 12:25:45.494512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.375129 (+2880617us) service_pool.cc:224] Handling call
0709 12:25:48.375435 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.376112 (+   677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.376293 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:48.355723  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:48.353127  7636 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.311632  7731 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:48.302765  7628 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.519s	user 0.010s	sys 0.000s
I20260709 12:25:48.365744  4751 rpcz_store.cc:276] 0709 12:25:45.205427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.300243 (+3094816us) service_pool.cc:224] Handling call
0709 12:25:48.300537 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.300715 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.300739 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:48.300780 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:48.300815 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.300863 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.300960 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:48.302608  7537 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.367364  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1981) took 2891 ms. Trace:
I20260709 12:25:48.328691  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:48.305428  4672 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:48.305006  7619 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.329180  7416 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:48.230345  7704 consensus_queue.cc:237] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.360319  7542 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:48.366945  7677 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:48.355265  7097 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:25:48.413870  7537 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.427104  7505 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.427939  4600 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:48.419682  7627 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:48.418315  7635 consensus_meta.cc:287] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.001s
I20260709 12:25:48.387207  4748 rpcz_store.cc:276] 0709 12:25:45.476219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.329455 (+2853236us) service_pool.cc:224] Handling call
0709 12:25:48.330007 (+   552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.367031 (+ 37024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.367293 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:48.371155  4571 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:48.432545  4752 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:48.321147  7202 consensus_queue.cc:260] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:48.352780  7630 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:48.434432  7097 consensus_queue.cc:1243] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:48.437865  7630 raft_consensus.cc:697] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:25:48.438349  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.438979  7630 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:48.372466  7482 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:48.297389  7720 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:48.373203  7345 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.377846  7173 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:48.353598  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.346650  7639 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.355795  7696 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:48.306841  7332 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.310325  7584 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.370496  4602 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:48.429842  7628 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:48.413061  7695 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:48.436707  7202 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.429852  7635 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:48.437191  7627 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:25:48.435320  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 121747us
I20260709 12:25:48.518654  4647 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:48.370023  7732 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:48.466295  4597 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:48.429513  7717 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
W20260709 12:25:48.468822  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 89900.3us
I20260709 12:25:48.383231  7731 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.472352  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.547384  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.433295  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 472) took 2202 ms. Trace:
I20260709 12:25:48.548693  4752 rpcz_store.cc:276] 0709 12:25:46.230614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.430014 (+2199400us) service_pool.cc:224] Handling call
0709 12:25:48.430341 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.433046 (+  2705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.433238 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:48.549461  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:48.550141  5343 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.550304  4752 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:25:48.431584  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 61537.2us
I20260709 12:25:48.387310  7625 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.420092  7636 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:48.434684  7505 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:48.523111  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.441540  7598 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.520447  4647 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:48.535171  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:48.568547  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.467581  7173 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:48.554242  4571 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.510s (SENT) [suppressed 31 similar messages]
I20260709 12:25:48.554785  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2449) took 1543 ms. Trace:
I20260709 12:25:48.574426  4752 rpcz_store.cc:276] 0709 12:25:47.011538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.549750 (+1538212us) service_pool.cc:224] Handling call
0709 12:25:48.550012 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.550739 (+   727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.554729 (+  3990us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:48.444005  7708 consensus_queue.cc:1175] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3a5aeedd11fe49938079fe86cb01232c" responder_term: 4 status { last_received { term: 4 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 4 index: 6 } }
I20260709 12:25:48.510829  7695 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:25:48.572264  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.464973  7644 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.514359  7720 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:48.541370  7332 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:48.575484  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:48.593130  4602 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
W20260709 12:25:48.598994  7737 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:25:48.600718  7635 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:48.548734  7731 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.606396  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:48.543464  7732 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.607708  7625 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:48.509361  7639 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.607913  7734 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.610402  7595 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:48.610997  7743 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:48.547607  7445 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:48.615196  7598 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.616361  7695 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.618216  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:48.619014  4645 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:48.660645  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.661857  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:48.665824  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:48.668216  4603 raft_consensus.cc:2393] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:25:48.671518  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 151203us
W20260709 12:25:48.672125  4598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 447) took 2193 ms (client timeout 1875 ms). Trace:
W20260709 12:25:48.672506  4598 rpcz_store.cc:269] 0709 12:25:46.478654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.590397 (+111743us) service_pool.cc:224] Handling call
0709 12:25:48.671937 (+2081540us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b5400033754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:48.672058 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3623868288}
I20260709 12:25:48.664896  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:48.676167  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:48.677356  5312 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:48.638844  7683 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:48.749657  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 61498.2us
W20260709 12:25:48.754281  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 76987.6us
I20260709 12:25:48.665764  7682 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:48.756016  5343 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
W20260709 12:25:48.757807  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 434710us
W20260709 12:25:48.757891  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 206615us
I20260709 12:25:48.686321  7598 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:48.684052  7696 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:48.765882  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:48.766147  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:48.766728  4598 raft_consensus.cc:2410] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:48.767283  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:48.768538  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:48.771304  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:48.769362  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 215135us
I20260709 12:25:48.774820  7639 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:48.776898  7636 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:48.777117  7726 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:48.822366  7726 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.777791  7690 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:48.823535  7690 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:48.746891  7595 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:25:48.826968  7734 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.779572  7537 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.764107  7682 raft_consensus.cc:697] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:48.747924  7683 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.786783  4752 raft_consensus.cc:2410] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:48.772121  7635 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:48.771937  7744 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.758885  7723 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.790369  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.761395  7717 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:48.796504  5343 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8e62ff81eb34853811314530131292b"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:25:48.818740  7488 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.798095  7628 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:48.770155  7625 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:48.766052  7644 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:48.813508  7621 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.770117  7598 raft_consensus.cc:697] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:48.846238  7621 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.850381  7598 consensus_queue.cc:237] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.851686  7621 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:48.795734  7630 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.856003  7482 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:48.833515  7683 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.833984  4596 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:48.807346  7505 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:48.838008  5343 raft_consensus.cc:606] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:25:48.769053  7606 consensus_meta.cc:287] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.000s
I20260709 12:25:48.865895  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.867762  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.841128  7723 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.868348  4596 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:48.843850  7625 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.846506  7717 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:25:48.882158  4756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2464) took 1810 ms (client timeout 1510 ms). Trace:
W20260709 12:25:48.882658  4756 rpcz_store.cc:269] 0709 12:25:47.072076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.088818 (+ 16742us) service_pool.cc:224] Handling call
0709 12:25:48.876632 (+1787814us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b54000a7154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:48.882086 (+  5454us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1960449408}
I20260709 12:25:48.857650  7537 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.883868  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:48.887873  4596 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:25:48.888367  4756 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:48.889145  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.889778  4596 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:48.839668  7744 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.896565  4596 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:25:48.874915  7625 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.847370  7488 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:48.888643  7700 consensus_meta.cc:287] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.565s	user 0.013s	sys 0.000s
I20260709 12:25:48.898514  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:48.864290  7606 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.896004  7682 consensus_queue.cc:237] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.908830  4596 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:48.911185  7717 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:48.912185  7749 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:25:48.915985  7606 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:48.916131  7733 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.536s	user 0.005s	sys 0.000s
I20260709 12:25:48.923048  4598 raft_consensus.cc:2368] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9.
I20260709 12:25:48.925993  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:48.926637  4747 raft_consensus.cc:2376] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9. Re-sending same reply.
I20260709 12:25:48.927663  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.928359  4747 raft_consensus.cc:2393] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:48.928457  7744 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:48.929557  4571 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:48.930088  7488 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.932024  7723 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.932530  7731 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:48.932981  4755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2446) took 1922 ms (client timeout 1635 ms). Trace:
W20260709 12:25:48.933405  4755 rpcz_store.cc:269] 0709 12:25:47.010838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.110295 (+ 99457us) service_pool.cc:224] Handling call
0709 12:25:48.932813 (+1822518us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b5400117454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:48.932918 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1154484608}
I20260709 12:25:48.934525  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.935217  4755 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 11: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:48.936794  7619 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:48.936939  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2524) took 1424 ms. Trace:
I20260709 12:25:48.937780  4675 rpcz_store.cc:276] 0709 12:25:47.512512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.998093 (+485581us) service_pool.cc:224] Handling call
0709 12:25:47.998411 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.104631 (+106220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.936686 (+832055us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540017f554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:48.936880 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242733312}
W20260709 12:25:48.938896  4681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2520) took 1495 ms (client timeout 1924 ms). Trace:
W20260709 12:25:48.939280  4681 rpcz_store.cc:269] 0709 12:25:47.443814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.511286 (+ 67472us) service_pool.cc:224] Handling call
0709 12:25:48.938750 (+1427464us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b540019f354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:48.938844 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503867264}
I20260709 12:25:48.908340  7700 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:48.942503  7700 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:48.942611  7494 raft_consensus.cc:3055] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:48.943097  7494 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:48.943951  7494 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.945108  7494 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:48.946025  7715 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:25:48.947173  5347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 436) took 1489 ms. Trace:
I20260709 12:25:48.949056  5347 rpcz_store.cc:276] 0709 12:25:47.457712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.457879 (+   167us) service_pool.cc:224] Handling call
0709 12:25:47.458244 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.047469 (+589225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.946929 (+899460us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b5400280854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:48.947110 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1197312896}
I20260709 12:25:48.951846  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:48.953599  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:48.953913  4571 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:48.954615  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:48.955139  4750 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.929301  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:48.967139  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:48.961501  7731 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:48.981045  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:48.983963  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:48.986088  4596 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:48.989997  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:48.990916  4747 raft_consensus.cc:2393] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 7: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:48.992249  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:48.994705  4571 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:48.994801  4596 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:48.995170  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:48.996059  7505 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.996510  4747 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.985416  7630 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:48.982826  7537 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:49.001122  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:49.003683  4596 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:25:49.007267  4596 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:48.998294  7751 log.cc:927] Time spent T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:25:48.961752  7173 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:48.998643  7695 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:49.019861  7695 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:49.021896  7625 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:49.022421  7173 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:49.023108  4748 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:49.031302  4747 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:49.033635  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:49.035131  4747 raft_consensus.cc:2393] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:49.032680  4571 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:49.048062  7726 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.050287  7537 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:49.050887  7537 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:49.051873  7537 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:49.048802  7726 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:49.056872  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:49.057250  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2014) took 1591 ms. Trace:
I20260709 12:25:49.062363  4601 rpcz_store.cc:276] 0709 12:25:47.465890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.803476 (+337586us) service_pool.cc:224] Handling call
0709 12:25:47.803679 (+   203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.803907 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:47.822193 (+ 18286us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.042007 (+1219814us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b18003a7500. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007effeb3cba40 00007effeb3c64e4 00007effeb3c6455 00007efff1eae3c7 00007effee4c09a7 00007effee4d00bf 00007effee4d0038
0709 12:25:49.042231 (+   224us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.056819 (+ 14588us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.056876 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:49.056935 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.057067 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.057192 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1093165,"prepare.queue_time_us":1245975,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":2452,"raft.queue_time_us":50537,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":11847,"spinlock_wait_cycles":9588096,"thread_start_us":118023,"threads_started":3,"wal-append.queue_time_us":135207,"wal-append.run_cpu_time_us":1226,"wal-append.run_wall_time_us":1036162}
I20260709 12:25:49.066356  4747 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:49.025813  4750 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:49.073666  7173 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:49.074311  7173 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:49.077452  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:49.078857  7621 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:49.079504  7621 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:49.054762  7732 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:49.066123  7505 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:49.083269  4596 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:49.084604  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1996) took 2802 ms. Trace:
I20260709 12:25:49.087280  7732 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:49.088547  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2448) took 2077 ms. Trace:
I20260709 12:25:49.090173  4756 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:25:49.088965  4748 rpcz_store.cc:276] 0709 12:25:47.011316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.469011 (+1457695us) service_pool.cc:224] Handling call
0709 12:25:48.472135 (+  3124us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.023521 (+551386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.088305 (+ 64784us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b5400309b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:49.088484 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1266593536}
I20260709 12:25:49.085081  4604 rpcz_store.cc:276] 0709 12:25:46.282614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.134611 (+851997us) service_pool.cc:224] Handling call
0709 12:25:47.147748 (+ 13137us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.186377 (+1038629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.084343 (+897966us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002760d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:49.084531 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2328585088}
I20260709 12:25:49.095949  4498 catalog_manager.cc:5697] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:49.070062  7625 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:49.093207  4603 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:49.102397  7496 raft_consensus.cc:3055] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:49.102159  4603 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:49.099584  4498 catalog_manager.cc:5697] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:49.102859  7496 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:49.105862  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:49.106405  7496 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:49.107028  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:49.107450  7496 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:49.108405  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:49.110553  4603 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:25:49.113823  4747 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:49.115157  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:49.119989  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:49.121208  4747 raft_consensus.cc:2393] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:49.117532  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:49.121964  4750 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:49.123178  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:49.117825  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:49.125128  4755 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:49.125734  4747 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:49.126610  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2451) took 2114 ms. Trace:
I20260709 12:25:49.126870  4755 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:49.127108  4747 rpcz_store.cc:276] 0709 12:25:47.011998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.125056 (+2113058us) service_pool.cc:224] Handling call
0709 12:25:49.125471 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.126309 (+   838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.126533 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.129544  4747 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:49.127704  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2457) took 2114 ms. Trace:
I20260709 12:25:49.133759  4755 rpcz_store.cc:276] 0709 12:25:47.013353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.126121 (+2112768us) service_pool.cc:224] Handling call
0709 12:25:49.126462 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.127422 (+   960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.127640 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.135141  4755 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:49.135656  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2461) took 2094 ms. Trace:
I20260709 12:25:49.135943  4755 rpcz_store.cc:276] 0709 12:25:47.041726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.134555 (+2092829us) service_pool.cc:224] Handling call
0709 12:25:49.134827 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.135426 (+   599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.135606 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.137027  4755 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:49.128417  4750 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:49.137993  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2450) took 2126 ms. Trace:
I20260709 12:25:49.138302  4750 rpcz_store.cc:276] 0709 12:25:47.011756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.124334 (+2112578us) service_pool.cc:224] Handling call
0709 12:25:49.128178 (+  3844us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.137749 (+  9571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.137928 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.139374  4750 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:49.139860  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1985) took 1960 ms. Trace:
I20260709 12:25:49.140151  4750 rpcz_store.cc:276] 0709 12:25:47.179004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.138976 (+1959972us) service_pool.cc:224] Handling call
0709 12:25:49.139224 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.139664 (+   440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.139815 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.141193  4750 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:49.141718  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1986) took 1962 ms. Trace:
I20260709 12:25:49.142019  4750 rpcz_store.cc:276] 0709 12:25:47.179354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.140786 (+1961432us) service_pool.cc:224] Handling call
0709 12:25:49.141031 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.141496 (+   465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.141674 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.143241  4750 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:49.143740  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2469) took 1855 ms. Trace:
I20260709 12:25:49.144025  4750 rpcz_store.cc:276] 0709 12:25:47.288503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.142666 (+1854163us) service_pool.cc:224] Handling call
0709 12:25:49.142909 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.143528 (+   619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.143696 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.154626  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 1983) took 1986 ms. Trace:
I20260709 12:25:49.155478  4755 rpcz_store.cc:276] 0709 12:25:47.168686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.136594 (+1967908us) service_pool.cc:224] Handling call
0709 12:25:49.136836 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.154220 (+ 17384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.154538 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.157755  4755 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:49.158646  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 475) took 1361 ms. Trace:
I20260709 12:25:49.162060  4598 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:49.159155  4755 rpcz_store.cc:276] 0709 12:25:47.797629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.156856 (+1359227us) service_pool.cc:224] Handling call
0709 12:25:49.157276 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.158319 (+  1043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.158573 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:49.168474  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.169750  7566 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:49.173019  4752 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:49.173820  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2476) took 1218 ms. Trace:
I20260709 12:25:49.174225  4752 rpcz_store.cc:276] 0709 12:25:47.957765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.172465 (+1214700us) service_pool.cc:224] Handling call
0709 12:25:49.172811 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.173524 (+   713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.173741 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.175096  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2024) took 1011 ms. Trace:
I20260709 12:25:49.175521  4601 rpcz_store.cc:276] 0709 12:25:48.163855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.070489 (+906634us) service_pool.cc:224] Handling call
0709 12:25:49.100900 (+ 30411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.101894 (+   994us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.101939 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.102449 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.128619 (+ 26170us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:49.128670 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.128734 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.174875 (+ 46141us) raft_consensus.cc:1685] finished
0709 12:25:49.174894 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.175025 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":25790,"threads_started":1,"wal-append.queue_time_us":26101}
I20260709 12:25:49.177783  4601 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:49.175797  4752 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:25:49.179826  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:25:49.180747  7758 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:25:49.180979  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:49.180189  4752 pending_rounds.cc:85] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 3
I20260709 12:25:49.181948  7744 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:49.182106  4752 pending_rounds.cc:107] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:25:49.182427  4752 raft_consensus.cc:2889] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:49.187286  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2477) took 1224 ms. Trace:
I20260709 12:25:49.187697  4752 rpcz_store.cc:276] 0709 12:25:47.962643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.175091 (+1212448us) service_pool.cc:224] Handling call
0709 12:25:49.175416 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.187007 (+ 11591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.187221 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.188127  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:49.189133  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 456) took 1061 ms. Trace:
I20260709 12:25:49.189213  4752 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:49.189554  4678 rpcz_store.cc:276] 0709 12:25:48.127257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.127735 (+   478us) service_pool.cc:224] Handling call
0709 12:25:48.136442 (+  8707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:48.136658 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.136685 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.931338 (+794653us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b2c00010c48. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:25:48.931739 (+   401us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.070718 (+138979us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:49.070754 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.070820 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.188843 (+118023us) raft_consensus.cc:1685] finished
0709 12:25:49.188863 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.189072 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37520768,"thread_start_us":138689,"threads_started":1,"wal-append.queue_time_us":250205}
I20260709 12:25:49.190236  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 479) took 1223 ms. Trace:
I20260709 12:25:49.191654  4752 rpcz_store.cc:276] 0709 12:25:47.966942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.188547 (+1221605us) service_pool.cc:224] Handling call
0709 12:25:49.188862 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.189892 (+  1030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.190174 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.193012  7567 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:49.183027  7644 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:49.193233  7573 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:49.193742  4752 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:49.178260  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 476) took 1380 ms. Trace:
I20260709 12:25:49.212652  4755 rpcz_store.cc:276] 0709 12:25:47.797985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.165157 (+1367172us) service_pool.cc:224] Handling call
0709 12:25:49.165417 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.177930 (+ 12513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.177963 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.178008 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:49.178044 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.178101 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.178209 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.214835  4755 raft_consensus.cc:1240] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:49.182461  7744 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:49.195636  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:49.200846  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.206113  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2460) took 2187 ms. Trace:
I20260709 12:25:49.218290  4747 rpcz_store.cc:276] 0709 12:25:47.018646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.129089 (+2110443us) service_pool.cc:224] Handling call
0709 12:25:49.129414 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.205623 (+ 76209us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:49.205669 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.205731 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:49.205779 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.205868 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.206036 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5170}
W20260709 12:25:49.223889  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.225600  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 480) took 1258 ms. Trace:
I20260709 12:25:49.226164  4752 rpcz_store.cc:276] 0709 12:25:47.967210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.193023 (+1225813us) service_pool.cc:224] Handling call
0709 12:25:49.193355 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.225244 (+ 31889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.225499 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.227878  4752 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:49.230597  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:49.236306  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 481) took 1268 ms. Trace:
I20260709 12:25:49.236871  4755 rpcz_store.cc:276] 0709 12:25:47.967440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.214155 (+1246715us) service_pool.cc:224] Handling call
0709 12:25:49.214561 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.235963 (+ 21402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.236236 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:49.238070  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.243353  7744 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:49.247596  4499 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52812 (request call id 290) took 16355 ms (client timeout 15000 ms). Trace:
I20260709 12:25:49.235435  7644 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:49.258677  7644 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:49.249557  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2006) took 1793 ms. Trace:
I20260709 12:25:49.251830  7732 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:25:49.261909  4605 rpcz_store.cc:276] 0709 12:25:47.456604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.791043 (+334439us) service_pool.cc:224] Handling call
0709 12:25:49.249365 (+1458322us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b540013ecd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:49.249495 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":605473664}
I20260709 12:25:49.263017  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:49.250371  4498 catalog_manager.cc:5697] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:49.264972  4752 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:49.251737  4599 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:49.264695  4498 catalog_manager.cc:5697] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 2 to 5, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 5 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:49.270358  4498 catalog_manager.cc:5697] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:49.291527  4498 catalog_manager.cc:5697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 8 to 9, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 9 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:49.251739  7612 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:49.253131  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:25:49.304211  4499 rpcz_store.cc:269] 0709 12:25:32.892153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.893595 (+  1442us) service_pool.cc:224] Handling call
0709 12:25:33.786914 (+893319us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee420090 00007effee4bef3a 00007efff4274e9c 00007efff437304e 00007efff28342a4 00007efff2834232 00007efff2834161 00007efff28340dc 00007efff2833392 00007effec7f148c 00007effec7f08c6
0709 12:25:49.247511 (+15460597us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:41.859799 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:41.860616 (+   817us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:41.860685 (+    69us) write_op.cc:435] Acquired schema lock
0709 12:25:41.860713 (+    28us) tablet.cc:662] Decoding operations
0709 12:25:41.862069 (+  1356us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:41.862154 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:41.862183 (+    29us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:25:41.863464 (+  1281us) tablet.cc:701] Row locks acquired
0709 12:25:41.863502 (+    38us) write_op.cc:260] PREPARE: finished
0709 12:25:41.863729 (+   227us) write_op.cc:270] Start()
0709 12:25:41.863944 (+   215us) write_op.cc:276] Timestamp: P: 1783599941863626 usec, L: 0
0709 12:25:41.863977 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 12:25:41.864834 (+   857us) log.cc:844] Serialized 4427 byte log entry
0709 12:25:42.274162 (+409328us) op_driver.cc:464] REPLICATION: finished
0709 12:25:43.066927 (+792765us) write_op.cc:301] APPLY: starting
0709 12:25:43.069913 (+  2986us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:43.118463 (+ 48550us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:43.205168 (+ 86705us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:43.465898 (+260730us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:43.466233 (+   335us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 12:25:43.466277 (+    44us) write_op.cc:312] APPLY: finished
0709 12:25:43.524687 (+ 58410us) log.cc:844] Serialized 83 byte log entry
0709 12:25:43.999313 (+474626us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:44.058429 (+ 59116us) write_op.cc:454] Released schema lock
0709 12:25:44.072820 (+ 14391us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:44.118082 (+ 45262us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:44.125347 (+  7265us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2514644,"reported_tablets":27,"spinlock_wait_cycles":383733120,"child_traces":[["op",{"apply.queue_time_us":779522,"apply.run_cpu_time_us":44594,"apply.run_wall_time_us":1043707,"num_ops":11,"prepare.queue_time_us":161703,"prepare.run_cpu_time_us":34803,"prepare.run_wall_time_us":144838,"raft.queue_time_us":38095,"raft.run_cpu_time_us":29307,"raft.run_wall_time_us":806636,"replication_time_us":409796,"thread_start_us":1416627,"threads_started":5,"wal-append.queue_time_us":614460,"wal-append.run_cpu_time_us":46863,"wal-append.run_wall_time_us":2583381}]]}
I20260709 12:25:50.739601  7332 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:49.607244  4498 catalog_manager.cc:5697] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 4 to 5, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 5 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:49.252331  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:50.608309  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.974516  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.975375  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.975927  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.976930  7494 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.374476  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 483) took 1614 ms. Trace:
I20260709 12:25:50.475756  5341 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:50.977922  7494 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:50.978149  4751 rpcz_store.cc:276] 0709 12:25:48.759919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.373259 (+1613340us) service_pool.cc:224] Handling call
0709 12:25:50.373790 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.374056 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.374090 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:50.374137 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:50.374184 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.374250 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.374405 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:50.555279  7173 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:50.459620  7542 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.522853  7567 consensus_meta.cc:287] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 1.328s	user 0.012s	sys 0.004s
I20260709 12:25:50.462699  7621 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:50.570868  7625 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.983160  7542 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:50.984790  7593 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:50.984995  7625 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:50.985836  7593 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:50.545495  7635 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.987273  4603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1626) took 1801 ms (client timeout 1693 ms). Trace:
I20260709 12:25:50.987934  7635 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:50.988619  7593 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:50.503573  7704 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.569415  7677 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:50.559230  7744 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.633128  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.998358  7744 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:50.716216  4605 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:50.989472  7542 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:50.742204  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 1.48888e+06us
I20260709 12:25:50.709043  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:50.745047  4498 catalog_manager.cc:5697] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 3 to 4, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 4 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:50.679335  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 2.48558e+06us
I20260709 12:25:50.758791  7644 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.760821  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 1.55682e+06us
W20260709 12:25:50.820967  4753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2447) took 3809 ms (client timeout 2141 ms). Trace:
I20260709 12:25:51.017532  7644 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:51.017751  4753 rpcz_store.cc:269] 0709 12:25:47.011089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.884954 (+873865us) service_pool.cc:224] Handling call
0709 12:25:50.820362 (+2935408us) spinlock_profiling.cc:243] Waited 51 ms on lock 0x7b54000bf4d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:50.820701 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117329792}
I20260709 12:25:51.017063  4498 catalog_manager.cc:5697] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 4 to 6, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 6 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:51.018951  4752 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: [8.5-11.6]
I20260709 12:25:51.019873  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2494) took 2160 ms. Trace:
I20260709 12:25:51.020080  4753 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:50.827360  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2002) took 2751 ms. Trace:
I20260709 12:25:50.858572  4756 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 12:25:50.863438  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:50.916383  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.917164  7728 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.917943  7720 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:25:50.924401  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 132364us
I20260709 12:25:50.934748  7741 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.936756  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.954511  7755 log.cc:927] Time spent T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.927s	user 0.000s	sys 0.005s
I20260709 12:25:50.943979  7726 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:50.986224  7559 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:50.469359  7705 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.067243  7720 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:50.991043  7704 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:50.981496  4751 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:50.740489  7332 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:51.002481  4600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2010) took 3544 ms (client timeout 2294 ms). Trace:
I20260709 12:25:51.003479  4604 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:51.003930  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2023) took 2826 ms. Trace:
I20260709 12:25:51.003765  7537 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:51.004914  7567 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.005481  7612 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.007437  7635 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.001300  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1621) took 1816 ms. Trace:
I20260709 12:25:51.081974  4605 rpcz_store.cc:276] 0709 12:25:49.184642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.263844 (+ 79202us) service_pool.cc:224] Handling call
0709 12:25:50.715141 (+1451297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.000677 (+285536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.001228 (+   551us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.009673  7625 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.003800  7744 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:50.987802  4603 rpcz_store.cc:269] 0709 12:25:49.186100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.235046 (+ 48946us) service_pool.cc:224] Handling call
0709 12:25:50.987014 (+1751968us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.016501  4569 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:51.076555  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2014) took 2249 ms. Trace:
I20260709 12:25:51.085229  4751 rpcz_store.cc:276] 0709 12:25:48.821785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.980306 (+2158521us) service_pool.cc:224] Handling call
0709 12:25:50.980891 (+   585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.071318 (+ 90427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.071529 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.020349  7644 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:51.027545  4748 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:51.020803  4754 rpcz_store.cc:276] 0709 12:25:48.075619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.152865 (+ 77246us) service_pool.cc:224] Handling call
0709 12:25:50.826790 (+2673925us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b54000e56d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:50.827169 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50188160}
I20260709 12:25:51.029243  4755 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:51.020282  4752 rpcz_store.cc:276] 0709 12:25:48.859798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.017729 (+2157931us) service_pool.cc:224] Handling call
0709 12:25:51.018284 (+   555us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:51.019460 (+  1176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.019766 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.037441  7741 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.038475  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2495) took 2168 ms. Trace:
I20260709 12:25:51.038661  4647 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:51.039803  4601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1623) took 1854 ms (client timeout 2035 ms). Trace:
I20260709 12:25:50.981513  7494 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.027372  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2013) took 2256 ms. Trace:
I20260709 12:25:51.066016  7728 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.065927  7559 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.080909  7704 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:51.030841  7593 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:51.027293  4498 catalog_manager.cc:5697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 1 to 4, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193), now has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:25:51.074149  4600 rpcz_store.cc:269] 0709 12:25:47.457712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.748843 (+291131us) service_pool.cc:224] Handling call
0709 12:25:51.001888 (+3253045us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b540013e7d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:51.002281 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191386368}
I20260709 12:25:51.080942  4602 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:51.100754  4753 rpcz_store.cc:276] 0709 12:25:48.862287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.019087 (+2156800us) service_pool.cc:224] Handling call
0709 12:25:51.019542 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.020523 (+   981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.030652 (+ 10129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.099929  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2489) took 2544 ms. Trace:
I20260709 12:25:51.101608  4756 rpcz_store.cc:276] 0709 12:25:48.771367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.856729 (+2085362us) service_pool.cc:224] Handling call
0709 12:25:50.857891 (+  1162us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:51.026944 (+169053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.027272 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:51.101024  4601 rpcz_store.cc:269] 0709 12:25:49.185182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.239185 (+ 54003us) service_pool.cc:224] Handling call
0709 12:25:51.039701 (+1800516us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.074684  4596 rpcz_store.cc:276] 0709 12:25:48.163632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.008871 (+845239us) service_pool.cc:224] Handling call
0709 12:25:49.009121 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.083685 (+ 74564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.968985 (+1885300us) spinlock_profiling.cc:243] Waited 73.9 ms on lock 0x7b540032c8d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:50.989911 (+ 20926us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170062080}
I20260709 12:25:51.099656  7468 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:51.127485  7720 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.128806  7728 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.131099  7559 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.131283  7741 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.134039  7567 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:51.180362  4750 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.877s	user 0.007s	sys 0.007s
I20260709 12:25:51.182461  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2471) took 3893 ms. Trace:
I20260709 12:25:51.182753  7567 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.183416  7567 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.182871  4750 rpcz_store.cc:276] 0709 12:25:47.288975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.144652 (+1855677us) service_pool.cc:224] Handling call
0709 12:25:49.144899 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.145114 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:49.235262 (+ 90148us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.181645 (+1946383us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.181932 (+   287us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:51.181987 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:51.182088 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.182190 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.182380 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":2002758,"threads_started":2}
I20260709 12:25:51.184516  7567 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.186121  7728 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.187048  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 484) took 2132 ms. Trace:
I20260709 12:25:51.186856  7728 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.187618  7141 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:51.187547  4753 rpcz_store.cc:276] 0709 12:25:49.054206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.185628 (+2131422us) service_pool.cc:224] Handling call
0709 12:25:51.186150 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.186579 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.186611 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:51.186688 (+    77us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:51.186737 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.186811 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.186947 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.188274  7141 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:51.190522  7332 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.084643  7542 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:51.194936  7542 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.196297  7537 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.199085  7537 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.201778  7612 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.202520  4498 catalog_manager.cc:5697] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 4 to 5, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 5 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:51.204632  7612 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.151039  7494 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.193446  7593 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.206492  7494 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:51.207240  7741 raft_consensus.cc:670] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:51.208467  7741 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.139132  4755 rpcz_store.cc:276] 0709 12:25:48.555144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.028225 (+2473081us) service_pool.cc:224] Handling call
0709 12:25:51.028701 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.092673 (+ 63972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.099860 (+  7187us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:51.135172  4571 leader_election.cc:336] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 2.008s (SENT)
I20260709 12:25:51.169808  4605 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:25:51.210035  4605 pending_rounds.cc:85] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 4
I20260709 12:25:51.209589  7741 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.210350  4605 pending_rounds.cc:107] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 7.5
W20260709 12:25:51.210647  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 110449us
I20260709 12:25:51.169476  7468 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.210681  4605 raft_consensus.cc:2889] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:51.215679  4750 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:25:51.217371  4750 pending_rounds.cc:85] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 5
I20260709 12:25:51.229944  4750 pending_rounds.cc:107] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:25:51.230409  4750 raft_consensus.cc:2889] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:51.204845  4498 catalog_manager.cc:5697] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:51.189431  7728 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.212479  7741 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:51.216852  7720 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.266911  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:51.267380  4751 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:51.268267  7728 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:25:51.268908  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 230627us
W20260709 12:25:51.278352  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 262086us
I20260709 12:25:51.269043  7728 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:51.284916  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2497) took 2344 ms. Trace:
I20260709 12:25:51.287019  4751 rpcz_store.cc:276] 0709 12:25:48.940774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.147154 (+2206380us) service_pool.cc:224] Handling call
0709 12:25:51.256918 (+109764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.268121 (+ 11203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.284826 (+ 16705us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.271349  4498 catalog_manager.cc:5697] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 3 to 5, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195), now has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 5 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:51.270306  7774 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.252102  7612 consensus_queue.cc:1059] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:25:51.290571  7774 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.282006  7776 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.293494  7468 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.294140  7776 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.295040  7593 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.295758  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1608) took 3055 ms. Trace:
I20260709 12:25:51.296859  7567 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.295807  7593 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:51.297183  7537 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:51.297801  7593 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:51.299150  7593 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.300684  7773 consensus_queue.cc:1059] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:51.256781  7141 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.302448  7773 raft_consensus.cc:3055] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:51.302834  7494 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.302940  7773 raft_consensus.cc:740] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:51.303416  7494 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.304126  7494 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:51.296747  4599 rpcz_store.cc:276] 0709 12:25:48.240460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.099907 (+859447us) service_pool.cc:224] Handling call
0709 12:25:49.251427 (+151520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.292889 (+2041462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.295325 (+  2436us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b540013ecd0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:51.295524 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347807104}
I20260709 12:25:51.293269  7774 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.304354  7773 consensus_queue.cc:260] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.283001  7720 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.290832  7612 consensus_queue.cc:1243] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:25:51.314385  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 2.13315e+06us
I20260709 12:25:51.296763  7776 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.323247  7612 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:51.322089  7559 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:51.329375  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:51.335198  7717 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.320s	user 0.015s	sys 0.015s
I20260709 12:25:51.335886  4720 leader_election.cc:400] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9.
I20260709 12:25:51.336496  4720 leader_election.cc:403] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:25:51.337323  7644 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:51.338801  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:51.329341  4498 catalog_manager.cc:5697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 1 to 3, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 3 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:51.347172  4533 debug-util.cc:398] Leaking SignalData structure 0x7b0800315ac0 after lost signal to thread 3434
I20260709 12:25:51.350441  5341 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:51.350890  7612 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:51.353756  7720 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:51.335808  7717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.325s	user 0.015s	sys 0.015s
I20260709 12:25:51.330518  7559 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.345239  7734 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:25:51.332923  7728 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.371404  7468 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.406015  7468 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.406479  7494 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.407209  5341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53808 (request call id 452) took 3534 ms. Trace:
I20260709 12:25:51.407704  5341 rpcz_store.cc:276] 0709 12:25:47.872760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.874846 (+  2086us) service_pool.cc:224] Handling call
0709 12:25:47.924133 (+ 49287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.474917 (+2550784us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54004b8320. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee42ba1d 00007effee4121a4 00007effee494447 00007effee4acfe7 00007effee4ae7de 00007effee4b8b49 00007effee4b9416 00007effee4b5d8f 00007effee4b583e 00007efff347a419
0709 12:25:51.360909 (+885992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.407089 (+ 46180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348542208}
I20260709 12:25:51.408293  7468 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.387034  7776 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:51.387379  4647 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:51.389717  7677 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:25:51.390569  7778 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.391444  4498 catalog_manager.cc:5697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:51.396435  7766 raft_consensus.cc:3055] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:25:51.432520  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 97068us
I20260709 12:25:51.386173  7741 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:51.437584  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 97563.2us
I20260709 12:25:51.367978  7537 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 12:25:51.432756  7766 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:51.444586  7771 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9.
I20260709 12:25:51.439584  7468 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.434013  7776 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.453718  7771 consensus_queue.cc:1059] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:51.454850  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:51.455328  7771 raft_consensus.cc:3055] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:51.455862  7771 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:51.457026  7743 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:51.449643  7494 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:51.457896  7765 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:51.457695  7743 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.456929  7771 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.440620  7741 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.459824  7771 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:51.460100  7743 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:51.460793  7778 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:51.478199  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 90966us
I20260709 12:25:51.480729  7705 consensus_queue.cc:1059] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:51.482585  7705 consensus_queue.cc:1059] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:51.461352  7741 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.452214  7776 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:51.463320  7778 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:51.495411  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 44566.9us
W20260709 12:25:51.496001  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.497536  7774 raft_consensus.cc:3055] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:51.498138  7774 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:51.498279  7741 consensus_queue.cc:1059] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:51.498199  4599 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:51.499485  7774 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.500852  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:51.501190  7774 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:51.500051  7743 raft_consensus.cc:3055] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:51.501835  7173 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:51.502287  7766 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.512393  4752 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:51.513531  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:51.512692  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:51.516172  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 171740us
I20260709 12:25:51.489547  7468 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.526042  7705 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:51.509446  7743 raft_consensus.cc:740] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:51.516000  7773 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:51.530714  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.513922  7778 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:51.528908  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.531708  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.538822  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:51.545020  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.544781  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.546586  4752 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
W20260709 12:25:51.547379  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 60894us
I20260709 12:25:51.547652  7141 consensus_queue.cc:1059] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:51.548623  7777 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:51.549413  7777 consensus_queue.cc:1243] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:25:51.549753  7778 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.550340  7734 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:51.552400  7778 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:51.555595  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.555891  7766 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:51.557081  7771 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.557734  7771 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.562554  4753 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:51.567292  7468 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:51.568329  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 485) took 2396 ms. Trace:
I20260709 12:25:51.568876  4748 rpcz_store.cc:276] 0709 12:25:49.171740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.205366 (+2033626us) service_pool.cc:224] Handling call
0709 12:25:51.205985 (+   619us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.211247 (+  5262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.211287 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.212033 (+   746us) log.cc:844] Serialized 26 byte log entry
0709 12:25:51.358702 (+146669us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:51.358760 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.358856 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.568018 (+209162us) raft_consensus.cc:1685] finished
0709 12:25:51.568049 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.568236 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1610,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":28838,"thread_start_us":146102,"threads_started":1,"wal-append.queue_time_us":291278}
I20260709 12:25:51.570863  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.572173  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.573248  4748 raft_consensus.cc:2410] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:25:51.573673  4755 raft_consensus.cc:2393] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:51.574572  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.575222  4755 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:25:51.576033  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.576617  4755 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:51.576833  7567 raft_consensus.cc:3055] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:51.577308  7567 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:51.578497  7567 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.579841  7704 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.580031  7644 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.580376  7765 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:51.580482  7704 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.580708  7644 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.581336  7771 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:51.581143  7765 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.581893  7771 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.582038  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.582417  7771 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.582813  7644 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:51.583444  7765 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.584550  7771 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:51.584995  7743 consensus_queue.cc:260] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.587952  7494 raft_consensus.cc:3055] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:51.564867  7705 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:51.588397  7494 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:51.588874  7705 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.591218  7705 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.589419  7494 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.592949  7494 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:51.593281  7567 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.562479  4751 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:51.598398  7097 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.604929  7767 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:51.545501  7332 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:25:51.612355  4748 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:25:51.619515  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.620203  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1606) took 3380 ms. Trace:
I20260709 12:25:51.620589  4604 rpcz_store.cc:276] 0709 12:25:48.239906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.095671 (+855765us) service_pool.cc:224] Handling call
0709 12:25:51.002957 (+1907286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.080378 (+ 77421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.620006 (+539628us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b540013e7d0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:51.620144 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4385412480}
I20260709 12:25:51.621469  4677 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:51.613723  7743 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:51.602104  7097 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.629437  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.616186  7593 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.632488  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.632705  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.633010  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.633157  5345 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:51.634514  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.636248  7704 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:51.638922  7494 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.616573  7788 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:51.639530  7494 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:51.618124  7705 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.625965  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:51.651731  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 102280us
I20260709 12:25:51.652666  4748 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:51.640228  7788 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.649907  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:51.670321  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:51.671833  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.673282  4677 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:51.674543  5345 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:51.676208  5314 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:51.666371  7677 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.677944  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:51.678522  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:51.679026  4678 raft_consensus.cc:2410] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:51.679121  4679 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:51.679939  4597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 459) took 3583 ms (client timeout 1822 ms). Trace:
W20260709 12:25:51.680321  4597 rpcz_store.cc:269] 0709 12:25:48.060494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.060768 (+   274us) service_pool.cc:224] Handling call
0709 12:25:51.637545 (+3576777us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b5400352fd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:51.643799 (+  6254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709153152}
I20260709 12:25:51.667691  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.668737  7141 consensus_queue.cc:1059] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:25:51.682052  7141 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:51.679054  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1622) took 2494 ms. Trace:
I20260709 12:25:51.683876  4605 rpcz_store.cc:276] 0709 12:25:49.184912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.083083 (+1898171us) service_pool.cc:224] Handling call
0709 12:25:51.083512 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.234976 (+151464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.678764 (+443788us) spinlock_profiling.cc:243] Waited 55.8 ms on lock 0x7b5400060754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:51.678964 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128396544}
I20260709 12:25:51.683534  7141 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:51.685678  7704 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:51.691293  7631 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.691635  7704 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.692250  7779 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:51.691921  7141 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.692716  7779 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.693136  7779 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:51.693984  5342 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:51.694216  7704 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:51.693248  7141 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:51.694934  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:51.669606  7705 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.682528  7767 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.697950  7173 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.701911  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:51.702490  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 482) took 3699 ms. Trace:
I20260709 12:25:51.668452  7593 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.683645  7494 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:51.711213  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 166436us
I20260709 12:25:51.709422  4747 rpcz_store.cc:276] 0709 12:25:48.003233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.219859 (+1216626us) service_pool.cc:224] Handling call
0709 12:25:51.701588 (+2481729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.701885 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.701925 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:51.701977 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:51.702028 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.702099 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.702239 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:51.714213  4568 leader_election.cc:336] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Timed out: RequestConsensusVote RPC to 127.3.89.196:39067 timed out after 2.217s (SENT)
I20260709 12:25:51.715430  4568 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:51.715378  7787 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:51.716313  7777 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:51.717888  7631 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:51.718397  7717 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:25:51.698050  7332 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.731134  5314 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:51.730193  7767 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.735077  7173 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.739077  7631 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:51.740815  7173 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.741809  4678 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:51.741490  7677 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.743216  4748 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:51.741410  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.744524  5342 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:51.746207  7332 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.746953  7705 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.747335  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.748364  7780 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:51.748812  7780 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:51.749189  7780 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:51.751266  7332 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.740883  7767 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:51.755733  4677 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
W20260709 12:25:51.753417  4720 connection.cc:256] RPC call timeout handler was delayed by -1.4004839809999794s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
W20260709 12:25:51.771328  4720 connection.cc:256] RPC call timeout handler was delayed by -1.3723884119999639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
W20260709 12:25:51.771792  4720 connection.cc:256] RPC call timeout handler was delayed by -1.3534841640002924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2348s for a response
W20260709 12:25:51.772131  4720 connection.cc:256] RPC call timeout handler was delayed by -1.152983981000034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
W20260709 12:25:51.772528  4720 connection.cc:256] RPC call timeout handler was delayed by -1.0135748009997769s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1693s for a response
I20260709 12:25:51.766264  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.737031  7704 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.741364  7788 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.743803  7677 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.782208  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.783941  7782 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:51.784029  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.784516  7782 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:51.784771  5345 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
W20260709 12:25:51.785176  4533 debug-util.cc:398] Leaking SignalData structure 0x7b08001050c0 after lost signal to thread 4536
I20260709 12:25:51.784078  4598 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:51.786806  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:25:51.789944  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 75794.4us
I20260709 12:25:51.793800  7704 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.759414  7494 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:51.796018  7704 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:51.796267  7494 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.796797  7542 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.798465  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2031) took 3134 ms. Trace:
I20260709 12:25:51.798834  4598 rpcz_store.cc:276] 0709 12:25:48.663654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.162952 (+499298us) service_pool.cc:224] Handling call
0709 12:25:51.783635 (+2620683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.786040 (+  2405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.798285 (+ 12245us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003552d0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:51.798408 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2357433600}
W20260709 12:25:51.799587  4749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 1988) took 4618 ms (client timeout 1806 ms). Trace:
W20260709 12:25:51.800001  4749 rpcz_store.cc:269] 0709 12:25:47.181446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.738171 (+556725us) service_pool.cc:224] Handling call
0709 12:25:51.799216 (+4061045us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54001f2054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:51.799410 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370612480}
W20260709 12:25:51.800863  4533 debug-util.cc:398] Leaking SignalData structure 0x7b0800281aa0 after lost signal to thread 4630
I20260709 12:25:51.801087  7332 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.811868  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:51.814494  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:51.801505  4602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1604) took 3539 ms (client timeout 2348 ms). Trace:
I20260709 12:25:51.765831  7771 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:51.817122  4602 rpcz_store.cc:269] 0709 12:25:48.239316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.665465 (+426149us) service_pool.cc:224] Handling call
0709 12:25:51.765496 (+3100031us) spinlock_profiling.cc:243] Waited 2.33 s on lock 0x7b54003552d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:51.778868 (+ 13372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5360481664}
W20260709 12:25:51.818663  4533 debug-util.cc:398] Leaking SignalData structure 0x7b080039f3e0 after lost signal to thread 4706
I20260709 12:25:51.775270  7741 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:51.785125  7559 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.818718  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:51.798497  7494 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.835933  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.776772  7705 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.835346  4568 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:51.841828  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:51.842839  4672 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:51.843570  7771 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.845945  7771 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:51.846211  4677 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:51.846814  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.822834  7734 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:25:51.848371  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.849146  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.837558  7788 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.850339  7788 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.852728  7788 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.832273  7741 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.859516  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:25:51.859966  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.853869  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.862269  4753 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:51.863441  4679 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:51.837873  7593 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:51.793429  7782 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.858843  7741 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:51.885685  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 133224us
W20260709 12:25:51.887045  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 68402.8us
I20260709 12:25:51.885653  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:51.893419  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:51.894645  4749 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:51.885102  7173 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:51.839159  7332 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.901364  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:51.884230  7494 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.903324  4677 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:51.891433  7728 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.877458  7773 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.865325  7780 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.912355  4750 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:51.913264  4679 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:51.915501  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.900766  7787 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:51.945647  5312 connection.cc:256] RPC call timeout handler was delayed by -2.4035611070003142s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18220000000000003s for a response
W20260709 12:25:51.949817  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 114547us
W20260709 12:25:51.906050  7789 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:25:51.915421  7559 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.959314  7728 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.959749  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.959671  7559 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.960469  5346 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:51.953903  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:51.906946  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.963878  5347 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
W20260709 12:25:51.965937  7792 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 12:25:51.966727  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.900331  7705 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.944898  7780 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.970575  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:51.939785  7791 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.947068  7332 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:51.908881  7593 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.906756  7631 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:25:51.972406  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 61179.2us
I20260709 12:25:51.979413  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.982851  4719 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:51.970045  7567 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.967989  7468 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.941169  7734 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:25:51.955461  7494 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:51.989650  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 103818us
I20260709 12:25:51.989488  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:51.968529  7173 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.007153  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.008000  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.010039  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.968779  7773 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:51.975666  7787 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.994719  7332 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:52.019214  5345 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:52.014441  4571 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:52.006771  7768 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:51.994488  5341 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:52.029369  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:51.989233  7766 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:25:52.041013  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.041706  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.038262  7777 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:52.042740  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:52.023558  7332 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:52.043948  4673 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:52.044387  4676 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:52.044480  7773 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:52.046113  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.046541  4675 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:25:52.021888  7704 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.995225  7593 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:51.994222  7801 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:25:52.031821  7768 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.991715  7791 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:52.009604  7468 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.085737  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.092859  5345 raft_consensus.cc:2393] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:52.087772  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.089797  7805 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:52.016026  7567 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.060537  7332 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:52.055838  7777 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.095741  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.101874  4677 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:52.095541  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:52.104472  4673 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:52.104384  4678 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:52.090790  7768 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.103614  4675 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:25:52.110795  4672 raft_consensus.cc:2410] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:52.094719  7728 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:52.038787  7173 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:52.112545  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:52.104897  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:52.133042  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:52.113294  5347 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:52.141126  5346 raft_consensus.cc:2376] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8. Re-sending same reply.
I20260709 12:25:52.135738  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.149561  4602 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:52.106789  7777 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:52.110430  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.158218  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.159772  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.194653  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.195432  4601 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 5: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:52.195782  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.196303  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.159796  4676 raft_consensus.cc:2393] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 5: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:52.161044  7787 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:52.168368  4673 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
W20260709 12:25:52.172969  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 202474us
I20260709 12:25:52.173323  5345 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:52.156876  7776 consensus_meta.cc:287] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.656s	user 0.006s	sys 0.007s
I20260709 12:25:52.181236  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.193919  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.194110  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:52.196381  4681 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:25:52.160584  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.218187  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.136384  7704 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:52.208489  4677 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:52.197032  4601 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:52.109115  7805 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:52.197368  4600 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:52.189620  7705 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:52.218127  5348 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:52.238692  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:52.238818  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.239167  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:52.237867  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.247618  4676 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:52.240907  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.239801  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:52.194550  7800 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:52.197974  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.244453  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.246760  4681 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:52.290530  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:52.291872  4604 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:52.279112  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:52.364014  4673 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 3.
W20260709 12:25:52.366412  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.380810  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.385588  4675 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
W20260709 12:25:52.306347  4533 debug-util.cc:398] Leaking SignalData structure 0x7b080009aa80 after lost signal to thread 4780
I20260709 12:25:52.240798  7468 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:52.276772  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:52.247006  4596 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:52.268250  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.286406  7791 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:52.286619  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.237839  7776 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:52.240948  7141 consensus_meta.cc:287] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.542s	user 0.012s	sys 0.000s
W20260709 12:25:52.395459  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 155745us
W20260709 12:25:52.263926  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.308950  7173 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:52.248180  7802 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:25:52.294818  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.293376  7097 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:52.281502  7805 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:52.248564  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.413878  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:52.414050  4601 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:52.414796  4647 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:52.415431  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.416260  4601 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:25:52.248594  7494 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:52.429679  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:52.430117  7494 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:52.430940  4747 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:25:52.355156  4603 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:52.356776  7704 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:52.358246  7593 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:52.359215  7567 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:52.363023  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 51968.8us
I20260709 12:25:52.295877  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.433521  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:52.391326  4533 debug-util.cc:398] Leaking SignalData structure 0x7b080027bd60 after lost signal to thread 4781
I20260709 12:25:52.390640  4676 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:52.417439  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:52.396797  4752 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 3.
I20260709 12:25:52.397259  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.397464  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.397601  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.398102  4599 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:52.399356  7811 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.409415  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.409896  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.410324  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.410391  7801 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:52.407877  4598 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 3.
I20260709 12:25:52.414826  4602 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 5: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:52.417317  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.420812  4674 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
W20260709 12:25:52.427767  7741 consensus_meta.cc:287] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.528s	user 0.004s	sys 0.007s
I20260709 12:25:52.428813  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:52.353778  7705 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.394672  7800 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:52.434870  4756 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:52.435209  4596 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:52.438987  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.433876  4605 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:52.441226  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:52.460549  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:52.441813  4533 debug-util.cc:398] Leaking SignalData structure 0x7b08002482a0 after lost signal to thread 5373
I20260709 12:25:52.444314  4601 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:52.462502  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.463063  4601 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:52.441327  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:52.453949  4604 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:52.454918  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:52.464474  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.456277  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.456692  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.465559  4599 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 5: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:25:52.455497  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.462272  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.467525  4673 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:52.463714  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.461634  4754 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:52.400738  7768 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.470316  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:52.471153  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:52.463615  4533 debug-util.cc:398] Leaking SignalData structure 0x7b0800325840 after lost signal to thread 5374
I20260709 12:25:52.472453  4571 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:52.473227  4533 debug-util.cc:398] Leaking SignalData structure 0x7b08002ca340 after lost signal to thread 7575
I20260709 12:25:52.471740  4600 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
W20260709 12:25:52.464184  4645 connection.cc:256] RPC call timeout handler was delayed by -1.9287792029999764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24680000000000002s for a response
W20260709 12:25:52.464699  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 47834.5us
I20260709 12:25:52.475555  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.476799  4604 raft_consensus.cc:2410] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:52.447118  7801 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:52.446959  7805 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:52.477195  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.478992  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.481776  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.459401  7800 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.458241  7494 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:52.467371  7779 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.774s	user 0.013s	sys 0.000s
I20260709 12:25:52.454270  5341 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:52.483184  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:52.491345  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.491550  4756 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:52.492201  4601 raft_consensus.cc:2410] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:25:52.492687  4533 debug-util.cc:398] Leaking SignalData structure 0x7b080036f0a0 after lost signal to thread 7667
I20260709 12:25:52.492623  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:52.492945  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.479267  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:52.494341  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.494460  5314 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:52.487084  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:52.493270  4533 debug-util.cc:398] Leaking SignalData structure 0x7b080037e880 after lost signal to thread 7686
I20260709 12:25:52.495155  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.493363  4756 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:52.496158  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.489250  4681 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:52.456557  7741 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.498797  7741 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:52.469976  7811 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:52.502185  4598 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:52.482010  7801 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.505741  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.505389  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.505402  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:52.481871  7805 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:52.489045  7494 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:52.508112  4533 debug-util.cc:398] Leaking SignalData structure 0x7b08001f38a0 after lost signal to thread 7698
I20260709 12:25:52.504329  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.488708  7141 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:52.518275  4674 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 6: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:25:52.519754  4533 debug-util.cc:398] Leaking SignalData structure 0x7b08003522c0 after lost signal to thread 7700
I20260709 12:25:52.473750  7705 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.522354  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2020) took 1006 ms. Trace:
I20260709 12:25:52.523021  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:52.523402  4673 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:52.522753  4750 rpcz_store.cc:276] 0709 12:25:51.515817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.519692 (+  3875us) service_pool.cc:224] Handling call
0709 12:25:51.534480 (+ 14788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.912924 (+378444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.522119 (+609195us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54000735d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:52.522298 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":868010368}
I20260709 12:25:52.523350  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:52.521075  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.525297  4752 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.507632  7791 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.507786  7741 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:52.526161  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.526294  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:52.533113  4679 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:52.529289  4603 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:52.533145  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:52.530006  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 88762.1us
I20260709 12:25:52.536531  4647 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:52.513767  7494 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.538378  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:52.538831  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.518505  7743 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:52.518131  7141 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.539623  4674 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:25:52.538833  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.543835  4597 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:52.552258  4753 raft_consensus.cc:2410] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:25:52.552912  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:52.556735  4681 raft_consensus.cc:2410] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:25:52.565331  4750 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:52.537654  7741 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.560456  4672 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:52.531773  7774 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:52.570909  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.576555  4603 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:52.578236  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.579125  4679 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:25:52.579180  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.580142  4674 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:52.551487  7743 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.563195  7141 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:52.588778  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 78137.7us
I20260709 12:25:52.495241  7779 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.590056  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.537428  7791 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:52.576887  7741 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.590330  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.599287  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.597851  4750 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:52.594859  4672 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:52.589403  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:52.594218  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:52.602355  4674 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.581970  7717 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:25:52.586059  7743 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.594331  7593 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:52.590723  7824 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:52.590046  7705 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:52.625052  4750 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:52.610428  7768 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:52.635134  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.597906  7778 raft_consensus.cc:3055] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:52.636446  4750 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.619562  7795 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:52.645679  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.659809  4681 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:25:52.652053  5348 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:52.624100  7593 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.643851  7778 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:52.666141  7778 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:52.666873  7778 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:52.635298  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:52.678730  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.636857  7824 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:52.680604  4597 raft_consensus.cc:2410] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:52.681921  7820 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:52.676473  5342 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:52.672914  7593 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:52.690397  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 79123.5us
I20260709 12:25:52.633790  7811 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:52.632817  7468 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:25:52.686115  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:52.692202  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:52.632370  7818 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.172s
I20260709 12:25:52.695559  4681 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:52.687623  7820 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:52.703601  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 175536us
I20260709 12:25:52.685134  7824 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:52.691520  5312 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:52.706269  4597 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:25:52.707883  7816 log.cc:927] Time spent T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:25:52.626201  7805 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.697762  7795 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:52.713523  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:52.715231  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.705101  7820 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:52.754577  7814 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.467s	user 0.000s	sys 0.003s
I20260709 12:25:52.758275  7785 raft_consensus.cc:3055] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:52.758739  4751 pending_rounds.cc:85] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 5
I20260709 12:25:52.759039  4751 pending_rounds.cc:107] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 5.6
I20260709 12:25:52.759372  4751 raft_consensus.cc:2917] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Aborting config change with OpId 5.6: Aborted: Op aborted by new leader
I20260709 12:25:52.759914  4751 raft_consensus.cc:2889] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:52.758750  7785 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:52.709630  7537 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:25:52.765515  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:52.714480  7332 consensus_meta.cc:287] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.577s	user 0.012s	sys 0.000s
I20260709 12:25:52.729014  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.767892  4750 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:52.768231  4679 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:52.732417  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.733860  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.734227  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.769693  4598 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:52.734709  4680 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 12:25:52.736907  7807 raft_consensus.cc:2764] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:25:52.737865  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 64374.9us
I20260709 12:25:52.740188  7791 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:52.744746  7782 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:52.745895  4674 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:52.775492  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.747848  7832 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 12:25:52.748405  7815 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.460s	user 0.002s	sys 0.000s
I20260709 12:25:52.776463  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.777071  4598 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:25:52.749830  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:52.750677  4644 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:52.777709  4750 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 12:25:52.751854  7830 log.cc:927] Time spent T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:25:52.771059  4747 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:52.783318  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.783048  4645 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:52.785669  4675 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:52.786194  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.786792  4674 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:25:52.713294  7795 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:52.747437  7779 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:52.716471  7811 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:52.761057  7785 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.779268  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.779417  4644 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:52.798710  4679 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:52.799886  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.800565  4679 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:52.798260  4644 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:25:52.786916  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 95530.8us
I20260709 12:25:52.782909  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.774128  4568 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:52.807077  4755 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:25:52.790843  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:52.791224  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 87319.7us
I20260709 12:25:52.819139  4569 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:52.820290  4569 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:52.776400  7468 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:52.821501  7468 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:52.821959  7468 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:52.803676  4644 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:52.775419  7332 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.826699  4678 raft_consensus.cc:2393] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:52.827605  7332 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:52.828248  7332 raft_consensus.cc:2764] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:25:52.828866  7332 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:52.804854  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:52.829639  4675 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:52.831022  4681 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:52.775236  7494 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:52.833671  7494 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:52.791791  7795 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.834694  7801 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.837826  4754 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:52.838059  7787 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:52.838438  7141 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:52.839905  7787 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:52.840425  7787 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:52.840672  7141 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:52.809407  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:52.817169  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.793051  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:52.837870  4676 raft_consensus.cc:2393] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:52.843549  4605 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:52.843473  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.796648  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:52.844578  4679 raft_consensus.cc:2410] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:25:52.791447  7791 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:52.846683  4597 raft_consensus.cc:2393] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:52.845777  4750 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:52.845495  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 292634us
I20260709 12:25:52.848958  4679 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:52.853659  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.841712  7793 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:52.862421  4755 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:52.841441  7141 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:52.864152  4569 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:52.805529  7805 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.855122  4568 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:52.863184  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 45509.7us
I20260709 12:25:52.805991  7774 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:52.842084  7807 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:52.895040  5314 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:52.866010  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 62424.4us
I20260709 12:25:52.828259  7741 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.869905  4674 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:52.904718  4644 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:52.866557  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57227.9us
I20260709 12:25:52.891713  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 495) took 1004 ms. Trace:
I20260709 12:25:52.892640  4679 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:52.893839  4675 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:52.886634  4681 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:52.888199  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:52.870927  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:52.854319  5342 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:52.909015  4680 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:52.911989  4675 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:25:52.935184  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:52.876664  7743 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.936942  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:52.938047  4674 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:25:52.938576  4674 pending_rounds.cc:85] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 5
I20260709 12:25:52.938951  4674 pending_rounds.cc:107] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:25:52.939383  4674 raft_consensus.cc:2889] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:52.864540  7141 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:52.889108  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:52.918092  7777 consensus_meta.cc:287] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.673s	user 0.009s	sys 0.002s
I20260709 12:25:52.901834  7774 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:52.945633  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:52.956308  4602 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.943315  4681 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:25:52.948741  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 60611us
I20260709 12:25:52.919104  7791 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:52.905844  4756 rpcz_store.cc:276] 0709 12:25:51.887363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.557029 (+669666us) service_pool.cc:224] Handling call
0709 12:25:52.570930 (+ 13901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:52.571213 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:52.588437 (+ 17224us) log.cc:844] Serialized 14 byte log entry
0709 12:25:52.751238 (+162801us) spinlock_profiling.cc:243] Waited 7.77 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:52.891385 (+140147us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:52.891435 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:52.891485 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.891546 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.891661 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":265282,"prepare.queue_time_us":16823,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":31,"raft.queue_time_us":282499,"spinlock_wait_cycles":17870208,"thread_start_us":37268,"threads_started":1,"wal-append.queue_time_us":76877}
I20260709 12:25:52.902390  7807 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.869529  7717 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:52.946326  4675 pending_rounds.cc:85] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 2
I20260709 12:25:52.918450  7832 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:52.949436  7771 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:52.918202  7537 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:52.947857  7141 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.986001  4675 pending_rounds.cc:107] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:25:52.997012  4675 raft_consensus.cc:2889] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:52.983512  7791 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.950461  7779 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:52.887526  7593 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:52.853790  7787 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.991448  4752 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:52.992868  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:52.983451  4750 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:25:52.989917  7807 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:52.994786  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 59705.6us
I20260709 12:25:52.883081  7800 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:52.946352  7793 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:52.950481  7097 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.952288  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:52.967159  7774 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:53.010140  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 139266us
I20260709 12:25:52.988106  7717 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:53.013218  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 164749us
W20260709 12:25:53.018533  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 163296us
I20260709 12:25:53.021937  4569 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:53.004130  7791 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:53.022915  4569 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:53.022838  4674 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:53.021937  4568 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:53.025355  4677 raft_consensus.cc:2368] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7.
I20260709 12:25:53.027240  4673 raft_consensus.cc:2410] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:25:53.028689  7173 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:53.029271  7173 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:25:53.030597  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 47083.3us
I20260709 12:25:53.002802  7832 raft_consensus.cc:697] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:52.966464  7777 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.030197  7173 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.038149  4601 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:25:53.012674  7704 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.039866  7820 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.046639  4645 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:53.049270  4674 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:53.061097  7704 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:53.063326  7810 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.759s	user 0.002s	sys 0.000s
I20260709 12:25:53.033540  7717 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:53.063400  7704 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:53.071183  4603 raft_consensus.cc:2393] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:53.072955  4602 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
W20260709 12:25:53.051083  7841 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:25:53.077893  4680 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:53.082708  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:53.024564  7097 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.073865  7766 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:25:53.089591  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2513) took 1244 ms. Trace:
I20260709 12:25:53.089736  7850 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:53.086966  7818 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:25:53.095166  5312 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:53.096228  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:53.096735  7807 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.097908  4678 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:53.098376  7809 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.741s	user 0.003s	sys 0.000s
I20260709 12:25:53.083001  7720 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:53.175889  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2514) took 1329 ms. Trace:
W20260709 12:25:53.210893  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 115798us
W20260709 12:25:53.103965  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57463.2us
I20260709 12:25:53.078174  7793 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:53.098600  7820 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:53.229179  4673 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:53.082038  7779 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:53.114639  4749 rpcz_store.cc:276] 0709 12:25:51.845764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.898628 (+ 52864us) service_pool.cc:224] Handling call
0709 12:25:51.910389 (+ 11761us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.920858 (+ 10469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.920906 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.967672 (+ 46766us) log.cc:844] Serialized 26 byte log entry
0709 12:25:52.139707 (+172035us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:52.139744 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.139823 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.062637 (+922814us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:53.089238 (+ 26601us) raft_consensus.cc:1685] finished
0709 12:25:53.089264 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.089487 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3492,"raft.queue_time_us":3641,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":5860,"spinlock_wait_cycles":112162176,"thread_start_us":147830,"threads_started":1,"wal-append.queue_time_us":191214}
I20260709 12:25:53.150921  7849 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:53.170832  7855 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:53.172469  7593 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:53.211261  7850 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.190047  7097 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:53.168252  7567 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.220520  7780 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:53.243762  7855 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.244271  7778 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.215276  7720 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.213506  4748 rpcz_store.cc:276] 0709 12:25:51.854262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.530814 (+676552us) service_pool.cc:224] Handling call
0709 12:25:52.562559 (+ 31745us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:52.890576 (+328017us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:52.890845 (+   269us) log.cc:844] Serialized 14 byte log entry
0709 12:25:53.070743 (+179898us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.076697 (+  5954us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.076836 (+   139us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:53.076866 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.076937 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.175660 (+ 98723us) raft_consensus.cc:1685] finished
0709 12:25:53.175679 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.175826 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":303474,"prepare.queue_time_us":10987,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":10987,"raft.queue_time_us":310832,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":5941,"spinlock_wait_cycles":13156096,"thread_start_us":179532,"threads_started":1,"wal-append.queue_time_us":250276}
I20260709 12:25:53.249987  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:53.151633  7854 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.261336  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:53.261730  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:53.272518  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:53.242192  7856 consensus_queue.cc:1059] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:25:53.270912  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:53.301306  4747 raft_consensus.cc:2410] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:53.275487  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:53.275868  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:53.279084  4672 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:25:53.282965  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 244278us
I20260709 12:25:53.278534  4569 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:53.282125  7800 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:53.290910  4751 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:25:53.294559  5349 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:53.263415  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:53.271270  7850 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.272737  7849 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.270005  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:53.297922  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:53.294618  7847 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:53.303460  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:53.304625  4645 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:53.302403  4749 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:25:53.318068  5344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 491) took 1258 ms. Trace:
I20260709 12:25:53.318542  7820 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:53.320062  7854 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.294596  7720 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:53.242065  7837 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:25:53.301144  7793 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:25:53.298545  7593 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:53.284327  7780 raft_consensus.cc:697] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:53.171288  7832 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.321401  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:53.339354  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 317451us
W20260709 12:25:53.341411  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 63000.2us
I20260709 12:25:53.341845  7865 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:53.296763  7559 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:53.351828  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:53.283314  7567 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.284822  4719 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:53.303810  7777 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:53.304512  7807 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:53.357671  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 54308.9us
I20260709 12:25:53.360785  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:53.368661  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:53.369751  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1640) took 1067 ms. Trace:
I20260709 12:25:53.402333  4672 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.402804  4755 raft_consensus.cc:2410] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:25:53.370954  4674 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:53.371744  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2557) took 1016 ms. Trace:
I20260709 12:25:53.372021  4678 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:53.371759  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1646) took 1064 ms. Trace:
I20260709 12:25:53.372403  7765 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:53.402573  4676 rpcz_store.cc:276] 0709 12:25:52.302668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.455639 (+152971us) service_pool.cc:224] Handling call
0709 12:25:53.369517 (+913878us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b5400085554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:53.369698 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865719040}
I20260709 12:25:53.391875  7593 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.412554  7593 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:53.411674  7780 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:53.412971  7844 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.384s	user 0.000s	sys 0.002s
I20260709 12:25:53.292264  7778 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.403491  4675 rpcz_store.cc:276] 0709 12:25:52.355760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.931551 (+575791us) service_pool.cc:224] Handling call
0709 12:25:52.946116 (+ 14565us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:53.019586 (+ 73470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.019635 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:53.035170 (+ 15535us) log.cc:844] Serialized 52 byte log entry
0709 12:25:53.249982 (+214812us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:53.250022 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.250090 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.371508 (+121418us) raft_consensus.cc:1685] finished
0709 12:25:53.371526 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.371664 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2039,"raft.queue_time_us":3985,"spinlock_wait_cycles":2921856,"thread_start_us":175693,"threads_started":1,"wal-append.queue_time_us":253786}
W20260709 12:25:53.379261  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 74701us
W20260709 12:25:53.384896  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 111905us
I20260709 12:25:53.385087  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:53.386405  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:53.358237  7824 consensus_meta.cc:287] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.641s	user 0.009s	sys 0.005s
I20260709 12:25:53.389626  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:53.393234  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:53.296613  7779 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.392863  4748 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:53.395838  4747 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.397182  4569 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:25:53.397321  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 112528us
I20260709 12:25:53.454015  4752 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:53.357904  5344 rpcz_store.cc:276] 0709 12:25:52.047897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.049737 (+  1840us) service_pool.cc:224] Handling call
0709 12:25:52.050158 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:52.050767 (+   609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.050807 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:52.051354 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:25:52.151762 (+100408us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:52.151807 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.151869 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.089318 (+937449us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:53.299595 (+210277us) raft_consensus.cc:1685] finished
0709 12:25:53.299630 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.306696 (+  7066us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":142,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":137,"spinlock_wait_cycles":6950272,"thread_start_us":92037,"threads_started":1,"wal-append.queue_time_us":305763}
I20260709 12:25:53.403685  4677 rpcz_store.cc:276] 0709 12:25:52.307123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.732161 (+425038us) service_pool.cc:224] Handling call
0709 12:25:53.371572 (+639411us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54001bbf54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:53.371702 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":589047552}
W20260709 12:25:53.377261  7468 consensus_meta.cc:287] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.001s
I20260709 12:25:53.404701  7846 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:53.355582  7849 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:53.410717  7845 log.cc:927] Time spent T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:25:53.411381  7542 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:53.411972  7862 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:53.467962  7862 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:53.468369  7862 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.392920  7793 consensus_queue.cc:1243] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:53.405169  7559 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.470036  7559 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.472261  7559 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:53.416518  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:25:53.473316  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 56911.9us
W20260709 12:25:53.474082  7828 consensus_queue.cc:1175] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49277a7653524787a715232ab908bbc6" responder_term: 6 status { last_received { term: 3 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: [5.6-5.6]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:53.475219  7734 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:53.476094  4645 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:53.418994  4750 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.406157  7840 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:53.415808  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:53.445322  7778 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:53.445123  7771 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:25:53.450420  4755 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:25:53.460755  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 45403.5us
I20260709 12:25:53.488165  4568 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:53.487059  4672 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:53.452399  7567 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:53.463721  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:53.463689  7777 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:25:53.467154  4749 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:53.481979  4747 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:53.483304  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:53.485621  4748 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:25:53.493161  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 77427.2us
I20260709 12:25:53.495203  4647 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:25:53.495718  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 101298us
I20260709 12:25:53.498087  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:53.500695  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:53.488576  7559 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:53.502063  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:53.502069  4750 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:53.463850  7542 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.504156  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:53.486375  7840 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.502636  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:53.453891  7173 consensus_queue.cc:1059] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:25:53.503731  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:53.504711  4755 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:53.508167  7559 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:53.508967  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:53.509251  7818 raft_consensus.cc:3055] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:53.509012  7559 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:53.477743  7854 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:53.488590  7793 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:53.510914  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:53.511308  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:53.463143  7468 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.506328  7734 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:53.509763  7768 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:53.512321  7567 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:53.504738  7850 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.496975  7865 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:53.501796  7782 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:53.504654  4672 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:53.461584  7824 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.496605  7097 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:53.530795  4750 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.504261  7820 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:25:53.534595  4568 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:53.501518  7788 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:53.530421  7559 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.510308  7542 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.542411  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2029) took 1569 ms. Trace:
I20260709 12:25:53.511715  7840 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:53.530148  7717 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.524214  7818 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:53.535354  7793 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.496682  7855 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:53.558459  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.538952  7768 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.552515  7097 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.582923  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:53.584239  5344 raft_consensus.cc:2368] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7.
I20260709 12:25:53.580087  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:53.585721  7832 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:53.586928  7818 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.588178  7818 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.511406  7141 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.591183  7141 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:53.569725  4753 rpcz_store.cc:276] 0709 12:25:51.973187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.574446 (+601259us) service_pool.cc:224] Handling call
0709 12:25:52.574895 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:52.888428 (+313533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.888462 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:52.889010 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.014603 (+125593us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:53.014645 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.014708 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.542127 (+527419us) raft_consensus.cc:1685] finished
0709 12:25:53.542155 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.542329 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":312999,"raft.queue_time_us":314958,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":125221,"threads_started":1,"wal-append.queue_time_us":191701}
I20260709 12:25:53.592298  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:53.593027  4753 raft_consensus.cc:2410] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:53.594130  4753 pending_rounds.cc:85] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 3
I20260709 12:25:53.594362  4753 pending_rounds.cc:107] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:25:53.506652  7843 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:25:53.572889  4675 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:53.547988  7704 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:53.566606  4750 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:53.579991  7734 raft_consensus.cc:1064] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:25:53.569314  7559 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:53.580579  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:53.576378  7793 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:53.581601  7717 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:53.586161  7867 log.cc:927] Time spent T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:25:53.594671  4753 raft_consensus.cc:2889] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:53.567754  7778 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:53.553656  7854 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.045s
W20260709 12:25:53.570900  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 68334us
I20260709 12:25:53.569502  7777 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:53.545060  7859 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:25:53.608362  5349 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
W20260709 12:25:53.608331  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 104251us
I20260709 12:25:53.614193  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:53.615516  4645 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:53.598187  7097 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.577387  7782 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:53.611726  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 108047us
I20260709 12:25:53.595319  7768 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:53.621201  4645 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:53.621584  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:53.621783  7874 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:53.622063  7788 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:25:53.622429  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 84975.3us
I20260709 12:25:53.600798  7704 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:53.645130  7704 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.625319  4647 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:53.645192  7871 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:53.647306  5349 raft_consensus.cc:1240] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:53.641829  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:53.640607  7717 raft_consensus.cc:3055] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:53.647682  7874 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:53.658043  7795 consensus_meta.cc:287] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.823s	user 0.012s	sys 0.000s
W20260709 12:25:53.681283  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 60185.2us
I20260709 12:25:53.678197  4571 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:53.682349  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:53.657884  7717 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:53.642666  7782 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.695585  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:53.674979  7849 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:53.703706  4598 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:53.706391  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:53.710582  5349 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:53.712652  7717 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.714304  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:53.692852  7843 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:53.716080  7843 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:53.716533  4597 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:53.718020  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:53.719659  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:53.681645  7874 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:53.699280  7788 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:53.696830  7820 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:53.727658  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 68657.3us
I20260709 12:25:53.728085  4600 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:53.711203  7567 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:53.730958  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:53.734108  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:53.734555  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:53.688453  7795 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:53.739430  7791 consensus_meta.cc:287] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.701s	user 0.010s	sys 0.001s
I20260709 12:25:53.739949  7791 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.740746  7795 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:53.740859  7875 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:25:53.741679  7863 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:53.742226  7863 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:53.742645  7863 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.742832  7875 raft_consensus.cc:2764] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:53.743544  7875 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:53.743883  7875 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.744197  7875 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:53.734540  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:53.726871  7468 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:53.688112  7862 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.763326  4599 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:25:53.770282  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 259018us
I20260709 12:25:53.771456  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:53.772323  4600 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.740787  5341 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:53.737041  7824 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:53.779075  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:53.787055  7866 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:25:53.787201  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:53.788028  4673 raft_consensus.cc:2393] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 6: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:53.788369  7873 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.789013  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:53.788869  7873 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.790755  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2558) took 1434 ms. Trace:
I20260709 12:25:53.777305  4569 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:53.764124  7567 raft_consensus.cc:697] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:53.793313  7097 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.791148  4681 rpcz_store.cc:276] 0709 12:25:52.356129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.962924 (+606795us) service_pool.cc:224] Handling call
0709 12:25:53.000488 (+ 37564us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.009526 (+  9038us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.009584 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.023705 (+ 14121us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.396228 (+372523us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:53.396286 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.396412 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.790538 (+394126us) raft_consensus.cc:1685] finished
0709 12:25:53.790563 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.790693 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10179,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":12291,"thread_start_us":355226,"threads_started":1,"wal-append.queue_time_us":425713}
I20260709 12:25:53.819747  4645 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:53.824875  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:53.829523  4600 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:53.745898  7877 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.114s
W20260709 12:25:53.839047  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 271471us
I20260709 12:25:53.827294  7864 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:53.858173  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:53.858652  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:53.864431  5349 raft_consensus.cc:2393] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:53.861176  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:53.865377  5346 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.838088  7808 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:53.867898  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:53.868875  4599 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:53.821708  7849 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
W20260709 12:25:53.878829  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 83649.9us
I20260709 12:25:53.846011  7850 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:53.832885  7824 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
W20260709 12:25:53.881637  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 50322.5us
I20260709 12:25:53.856159  7791 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:53.884038  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:53.879998  7872 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:25:53.869474  7776 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:53.887439  4599 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:53.882164  4569 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:53.894717  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 465) took 1478 ms. Trace:
I20260709 12:25:53.895234  4674 rpcz_store.cc:276] 0709 12:25:52.416286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.075760 (+659474us) service_pool.cc:224] Handling call
0709 12:25:53.370773 (+295013us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.403413 (+ 32640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.894524 (+491111us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b5400085550. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:53.894660 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":650978048}
W20260709 12:25:53.905694  7805 consensus_meta.cc:287] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.002s	user 0.010s	sys 0.003s
I20260709 12:25:53.906822  4672 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:25:53.909299  7876 log.cc:927] Time spent T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:25:53.909688  7704 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.911345  7741 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:53.886083  7864 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:25:53.915119  7097 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:53.895177  7567 consensus_queue.cc:237] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.898423  7782 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:53.919823  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:53.924723  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.926065  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 44009.5us
W20260709 12:25:53.936209  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.941488  7877 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:53.943274  7877 consensus_queue.cc:1059] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:53.944173  7877 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:53.945801  7704 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:53.946035  7873 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:53.946733  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:53.946969  7818 consensus_queue.cc:1059] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:53.948064  7741 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:53.949503  7886 raft_consensus.cc:3055] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:53.949898  7889 raft_consensus.cc:3055] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:53.950162  7886 raft_consensus.cc:740] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:53.950397  7889 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:53.894945  7768 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.951279  7886 consensus_queue.cc:260] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:53.951413  7889 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.953064  7768 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:53.945655  7864 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:53.952988  7859 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:53.949601  7542 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:53.894717  7850 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.951272  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:53.913120  7593 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:53.956524  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 331256us
I20260709 12:25:53.958267  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:53.978451  4752 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.952780  7889 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.960716  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:53.945832  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63483.7us
I20260709 12:25:53.921296  7849 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:53.987133  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:53.975822  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:53.982213  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:53.989300  4676 raft_consensus.cc:2393] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:53.983345  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:53.989903  4750 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:25:53.990753  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:53.991307  4750 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:53.992065  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:53.985903  4755 raft_consensus.cc:2368] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7.
I20260709 12:25:53.993912  4645 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:53.952787  7886 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.994992  4753 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:53.954010  7892 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:53.985317  7542 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:54.003355  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:54.004012  4753 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
W20260709 12:25:53.980873  7720 consensus_meta.cc:287] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.000s
W20260709 12:25:54.000468  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 266029us
I20260709 12:25:54.005483  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:54.012979  4647 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:53.985020  7863 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:54.035368  4647 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:54.036341  4647 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:54.040359  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:54.043985  5349 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:54.002342  7856 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:54.051561  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57718.6us
W20260709 12:25:54.055415  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 44013.8us
I20260709 12:25:54.018867  7787 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:54.019459  7824 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:54.063715  4752 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:54.034214  7782 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:54.023384  7862 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:54.064392  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:54.071336  5349 raft_consensus.cc:2368] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9.
W20260709 12:25:54.056500  7774 consensus_meta.cc:287] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.987s	user 0.012s	sys 0.001s
I20260709 12:25:54.073735  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:54.009225  7776 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 12:25:53.999096  7850 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:54.059185  7808 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:54.058908  7705 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:54.012327  7720 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:54.127156  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.127677  7768 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:54.066623  7840 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:54.231613  7850 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:54.170253  7890 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:25:54.115597  7774 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:54.172016  7863 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:54.201342  7791 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:25:54.172919  7593 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:54.235450  7895 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.211s
I20260709 12:25:54.233604  4677 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:54.183104  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:25:54.235785  7893 log.cc:927] Time spent T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 12:25:54.170645  7883 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:54.187031  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:54.174192  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:54.185789  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:54.186077  7901 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.188s
I20260709 12:25:54.256901  7765 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:54.257910  7765 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:54.249496  7705 raft_consensus.cc:697] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:54.269126  7795 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.269944  7795 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:54.270794  7705 consensus_queue.cc:237] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.272531  7795 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:54.273797  4756 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:54.155619  7820 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.278123  5346 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:54.278414  4677 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
W20260709 12:25:54.279524  4720 connection.cc:256] RPC call timeout handler was delayed by -1.0745608869997341s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
I20260709 12:25:54.280371  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:54.073485  7856 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:54.282310  4673 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:25:54.281949  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 304679us
I20260709 12:25:54.175334  7808 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.285583  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:54.286747  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:54.283144  4569 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:54.212991  7835 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.294605  7898 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.186110  7782 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:54.294926  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:54.374680  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.393635  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:54.358558  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 70330.8us
W20260709 12:25:54.393585  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 106949us
I20260709 12:25:54.393828  4749 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:54.396438  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:54.342633  7097 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:54.400722  7097 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:54.401422  7097 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:54.402546  7874 consensus_meta.cc:287] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.000s
I20260709 12:25:54.402964  7874 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.403482  7765 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:54.395720  7849 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:54.405130  7874 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:54.405263  7849 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:54.406446  7793 consensus_meta.cc:287] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.779s	user 0.012s	sys 0.000s
I20260709 12:25:54.406884  7793 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.408965  7793 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:54.410287  7559 consensus_meta.cc:287] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.788s	user 0.005s	sys 0.014s
I20260709 12:25:54.410779  7559 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.186187  7875 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:54.413092  7559 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:54.414422  7704 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:25:54.416668  4599 consensus_meta.cc:287] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.522s	user 0.007s	sys 0.000s
W20260709 12:25:54.424620  4601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1641) took 1912 ms (client timeout 1507 ms). Trace:
I20260709 12:25:54.424710  4599 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
W20260709 12:25:54.358685  7899 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:25:54.425073  4601 rpcz_store.cc:269] 0709 12:25:52.512599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.513235 (+   636us) service_pool.cc:224] Handling call
0709 12:25:54.424426 (+1911191us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b540004f454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:54.424540 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1178593152}
W20260709 12:25:54.295907  7903 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:25:54.425807  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:54.430469  4599 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:54.428669  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:54.357823  7894 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:54.310789  7808 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:54.285054  7863 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:54.431193  4601 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:54.360076  7840 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:54.296198  7864 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:54.433332  4752 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:25:54.431903  7862 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.434612  7704 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:54.284123  7820 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.434923  7097 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.438949  7807 raft_consensus.cc:3055] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:54.440168  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:54.414422  7875 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:54.423180  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:54.392891  7720 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:54.461061  7801 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:54.434981  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2566) took 1350 ms. Trace:
I20260709 12:25:54.486127  4675 rpcz_store.cc:276] 0709 12:25:53.084937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.845441 (+760504us) service_pool.cc:224] Handling call
0709 12:25:53.867245 (+ 21804us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.890986 (+ 23741us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.891020 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.897679 (+  6659us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.036613 (+138934us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:54.036653 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.036721 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.434771 (+398050us) raft_consensus.cc:1685] finished
0709 12:25:54.434791 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.434923 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2630,"raft.queue_time_us":8012,"spinlock_wait_cycles":5990272,"thread_start_us":85741,"threads_started":1,"wal-append.queue_time_us":243877}
I20260709 12:25:54.476009  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:54.445034  7894 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:54.491334  7894 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:54.491834  7097 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:54.476966  7704 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:54.398396  7835 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:54.495924  7835 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:54.496356  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.483247  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 482) took 1571 ms. Trace:
I20260709 12:25:54.497080  4596 rpcz_store.cc:276] 0709 12:25:52.911740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.911865 (+   125us) service_pool.cc:224] Handling call
0709 12:25:52.946294 (+ 34429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.923373 (+977079us) spinlock_profiling.cc:243] Waited 948 us on lock 0x7b18003f8440. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effee05e4bc 00007effeb3ac48b 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c6455 00007effee4ae060 00007effee4aebfb 00007effee4b964b
0709 12:25:53.923554 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.923585 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.924127 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.005212 (+ 81085us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:54.005243 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.005305 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.477444 (+472139us) raft_consensus.cc:1685] finished
0709 12:25:54.477471 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.483157 (+  5686us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58020,"raft.queue_time_us":122212,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":10196,"spinlock_wait_cycles":2180992,"thread_start_us":204592,"threads_started":2,"wal-append.queue_time_us":191388}
I20260709 12:25:54.499224  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:54.500147  4596 raft_consensus.cc:2410] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:25:54.500978  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:54.488552  4601 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:54.501201  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:54.493144  7704 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:54.502447  7704 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:54.502758  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:54.503376  7788 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.502879  7704 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:54.503947  7788 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:54.506393  7788 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:54.507133  4596 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:54.511745  4679 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:54.477001  7840 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.464349  7765 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:54.479373  7768 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:54.358650  7856 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:54.481073  7807 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:54.481472  7864 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:54.535099  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:54.537680  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:54.538441  4675 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:54.540457  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:25:54.541586  4753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.011s	sys 0.026s
I20260709 12:25:54.479348  7862 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.546931  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:54.547398  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:54.529680  4599 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:54.550354  7788 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:54.428592  7898 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:54.456547  7863 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:54.547689  4753 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:54.561111  4681 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.008s	sys 0.033s
W20260709 12:25:54.571184  4681 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.008s	sys 0.033s
I20260709 12:25:54.574631  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:54.574999  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2567) took 1489 ms. Trace:
I20260709 12:25:54.575397  4681 rpcz_store.cc:276] 0709 12:25:53.085165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.860320 (+775155us) service_pool.cc:224] Handling call
0709 12:25:53.882704 (+ 22384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.888985 (+  6281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.889007 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.896549 (+  7542us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.009406 (+112857us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:54.574044 (+564638us) log.cc:844] Serialized 14 byte log entry
0709 12:25:54.574666 (+   622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.574748 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.574790 (+    42us) raft_consensus.cc:1685] finished
0709 12:25:54.574803 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.574941 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6259,"prepare.queue_time_us":40742,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":3454,"raft.queue_time_us":7547,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":21969,"thread_start_us":558192,"threads_started":2,"wal-append.queue_time_us":201228}
I20260709 12:25:54.577634  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:54.529282  7840 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:54.581964  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 158875us
I20260709 12:25:54.510984  7782 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.585397  4603 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 6: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:54.586659  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:54.590129  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:54.591500  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:54.592114  4681 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:54.596345  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:54.581777  7854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.000s	sys 0.024s
I20260709 12:25:54.598387  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:54.599753  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:54.600185  4675 raft_consensus.cc:2368] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9.
I20260709 12:25:54.601101  4750 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:54.603643  5314 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:54.606801  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:54.607574  4603 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:54.608798  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:54.611665  7743 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:54.595835  7771 raft_consensus.cc:3055] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:25:54.597807  7854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.000s	sys 0.025s
I20260709 12:25:54.616858  7922 raft_consensus.cc:2764] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:25:54.619086  7840 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:54.623320  7862 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:54.625110  4749 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 4.
I20260709 12:25:54.630204  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:54.630839  4675 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:54.601480  7875 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:54.636083  4569 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:54.637148  4756 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:54.616194  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:54.639503  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:54.640617  5349 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:54.641291  4569 leader_election.cc:400] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9.
I20260709 12:25:54.641970  4569 leader_election.cc:403] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:25:54.642256  4750 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:54.642896  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:54.646354  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:54.647671  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:54.609208  7796 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:54.615715  7771 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:54.681129  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:54.683530  4675 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:54.687233  4601 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:54.657403  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:54.651659  7849 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9.
I20260709 12:25:54.651872  7840 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:54.659058  7863 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:54.690341  7863 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:54.662900  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2571) took 1384 ms. Trace:
I20260709 12:25:54.691202  4677 rpcz_store.cc:276] 0709 12:25:53.277995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.454756 (+176761us) service_pool.cc:224] Handling call
0709 12:25:54.662625 (+1207869us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b54002dfd54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:54.662789 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1695059328}
I20260709 12:25:54.691596  7840 raft_consensus.cc:697] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:54.692384  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:54.692543  7840 consensus_queue.cc:237] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:54.665515  4644 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:54.693810  7923 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:54.695873  4644 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:54.640045  7875 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.696791  7875 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:54.701535  4569 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:54.702513  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:54.703279  7805 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:54.704062  7805 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:54.704403  7805 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.704752  7805 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:54.652639  7796 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.627243  7864 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.705796  7796 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:54.706748  7704 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.709190  7704 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:54.649786  7862 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:54.717660  4603 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:25:54.652064  7922 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.693099  4677 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 6: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:54.709754  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:54.709934  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:54.711747  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:54.722035  4679 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:54.722366  4601 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:54.722944  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:54.710731  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:54.672417  7768 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:54.664723  4753 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:25:54.733098  4679 raft_consensus.cc:2393] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:54.723508  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2065) took 1007 ms. Trace:
I20260709 12:25:54.733996  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:54.734089  4601 rpcz_store.cc:276] 0709 12:25:53.715570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.721844 (+1006274us) service_pool.cc:224] Handling call
0709 12:25:54.722209 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.722913 (+   704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.723386 (+   473us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:54.724217  4749 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:54.734753  4753 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:25:54.725020  4681 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:54.726940  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:54.688069  7820 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:54.712374  7887 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:54.739773  4679 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:25:54.718183  7849 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:54.721088  7862 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.748289  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:54.744091  4677 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:54.718616  7854 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.739467  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:54.747175  4750 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:54.750344  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:54.752053  4681 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:25:54.712308  7923 consensus_queue.cc:1059] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:25:54.754894  4675 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
W20260709 12:25:54.755417  7847 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.007s	sys 0.007s
I20260709 12:25:54.701299  7808 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.739674  7778 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.737424  7919 raft_consensus.cc:2764] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 12:25:54.759886  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 90168.9us
I20260709 12:25:54.764189  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:54.766111  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.752178  7862 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:54.751825  7887 raft_consensus.cc:697] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:54.761037  7847 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.007s	sys 0.007s
I20260709 12:25:54.753913  7849 raft_consensus.cc:697] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:54.771337  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:54.774034  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:54.751354  7567 consensus_queue.cc:1059] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.018s
W20260709 12:25:54.774955  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 120091us
I20260709 12:25:54.778268  4720 leader_election.cc:400] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7.
I20260709 12:25:54.778497  4720 leader_election.cc:403] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:54.770440  7743 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:54.782361  4604 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:25:54.784597  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:54.762739  7800 consensus_queue.cc:1059] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:25:54.751616  7771 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.737924  7922 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:54.757771  7854 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.804018  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:54.805373  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:54.767232  7778 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.780728  7782 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:25:54.804134  5312 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.886s (SENT) [suppressed 22 similar messages]
I20260709 12:25:54.781173  7788 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:54.784741  7743 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.784052  7768 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7.
I20260709 12:25:54.786237  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:54.790019  7704 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:54.805325  7864 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:25:54.855547  4680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48198 (request call id 476) took 1921 ms (client timeout 1886 ms). Trace:
I20260709 12:25:54.780998  7923 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
W20260709 12:25:54.858625  4680 rpcz_store.cc:269] 0709 12:25:52.911448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.931267 (+ 19819us) service_pool.cc:224] Handling call
0709 12:25:54.832826 (+1901559us) spinlock_profiling.cc:243] Waited 58.6 ms on lock 0x7b540017eb54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:54.832972 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134797568}
I20260709 12:25:54.823132  7788 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.833477  7743 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:54.857590  7864 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:54.888026  4749 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:54.888167  7864 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:54.889533  7864 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:54.857415  7849 consensus_queue.cc:237] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:54.869336  7778 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:54.870754  7926 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:25:54.832320  7782 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:54.873955  7734 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:54.876926  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:54.878969  7930 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:54.879774  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:54.882210  7928 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:25:54.846760  7771 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:54.855823  7918 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:25:54.858800  7929 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:54.864871  7788 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:54.919168  7927 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:25:54.920367  4680 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:54.855973  7887 consensus_queue.cc:237] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:54.956643  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 152540us
I20260709 12:25:54.877254  7796 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:55.013150  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.981173  7913 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:55.070914  5346 raft_consensus.cc:2393] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:54.949083  7788 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:55.096290  4750 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:55.098067  4753 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:55.100957  7898 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:55.116685  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2069) took 1326 ms. Trace:
I20260709 12:25:55.069717  7835 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:55.118230  7835 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:55.120968  7835 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:55.122121  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2544) took 1120 ms. Trace:
I20260709 12:25:55.122535  4752 rpcz_store.cc:276] 0709 12:25:54.001865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.457778 (+455913us) service_pool.cc:224] Handling call
0709 12:25:54.484546 (+ 26768us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:54.526966 (+ 42420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.527019 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:54.534924 (+  7905us) log.cc:844] Serialized 52 byte log entry
0709 12:25:54.638563 (+103639us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:54.638626 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.638730 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.121895 (+483165us) raft_consensus.cc:1685] finished
0709 12:25:55.121913 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.122062 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4406,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":67039,"threads_started":1,"wal-append.queue_time_us":179573}
W20260709 12:25:55.126787  7894 consensus_meta.cc:287] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.635s	user 0.010s	sys 0.003s
I20260709 12:25:55.127414  7894 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:55.132668  7704 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:55.133435  7782 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:55.117293  4603 rpcz_store.cc:276] 0709 12:25:53.789951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.740274 (+950323us) service_pool.cc:224] Handling call
0709 12:25:54.754546 (+ 14272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.758843 (+  4297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.758870 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:54.764420 (+  5550us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.861647 (+ 97227us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:54.861707 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.861807 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.116349 (+254542us) raft_consensus.cc:1685] finished
0709 12:25:55.116377 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.116595 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":435,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":4587,"thread_start_us":79184,"threads_started":1,"wal-append.queue_time_us":160864}
W20260709 12:25:55.150168  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 295987us
I20260709 12:25:55.151600  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:55.067893  7923 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:55.096591  7875 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:55.162180  4602 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:25:55.181317  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.194885  4603 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:55.077822  7913 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:55.176162  7854 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:55.198953  7808 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:55.219058  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.222846  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.227591  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.228278  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.172868  7923 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:55.239270  7894 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:55.239961  7875 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:55.242238  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:55.242952  4679 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:25:55.256630  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:55.267177  4679 raft_consensus.cc:2393] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 6: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:55.268200  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:55.271958  4679 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:55.283067  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2067) took 1522 ms. Trace:
I20260709 12:25:55.244987  7796 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:55.288087  7542 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:55.290257  4750 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:55.304760  4603 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:55.296314  7863 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:55.294016  7743 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:55.316164  7938 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
W20260709 12:25:55.332840  7937 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 12:25:55.336244  7917 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.668s	user 0.001s	sys 0.001s
I20260709 12:25:55.322258  5341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 458) took 2086 ms. Trace:
I20260709 12:25:55.337344  5341 rpcz_store.cc:276] 0709 12:25:53.236251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.239940 (+  3689us) service_pool.cc:224] Handling call
0709 12:25:53.277404 (+ 37464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.783150 (+505746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.314339 (+1531189us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b540020dad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:55.320284 (+  5945us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057542528}
I20260709 12:25:55.344731  4680 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:55.347831  7913 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:55.350646  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2056) took 1286 ms. Trace:
I20260709 12:25:55.351106  4749 rpcz_store.cc:276] 0709 12:25:54.064633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.889052 (+824419us) service_pool.cc:224] Handling call
0709 12:25:54.889406 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:54.889964 (+   558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.889995 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:54.890627 (+   632us) log.cc:844] Serialized 52 byte log entry
0709 12:25:55.288307 (+397680us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:55.308254 (+ 19947us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.308547 (+   293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.308643 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.350368 (+ 41725us) raft_consensus.cc:1685] finished
0709 12:25:55.350389 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.350576 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":60537,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":1787,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":3281,"thread_start_us":355139,"threads_started":2,"wal-append.queue_time_us":430146}
W20260709 12:25:55.354633  7862 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.581s	user 0.003s	sys 0.009s
I20260709 12:25:55.356539  7862 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:55.355131  7766 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.358282  7766 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:55.358868  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 508) took 1355 ms. Trace:
I20260709 12:25:55.359416  4756 rpcz_store.cc:276] 0709 12:25:54.003896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.462718 (+458822us) service_pool.cc:224] Handling call
0709 12:25:54.628956 (+166238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.641472 (+ 12516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.358592 (+717120us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b5400183650. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:55.358787 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381354240}
I20260709 12:25:55.321127  7788 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:55.357429  7741 raft_consensus.cc:3055] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:55.390206  7741 raft_consensus.cc:740] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:55.391463  7741 consensus_queue.cc:260] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:55.392817  7741 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:55.366014  7537 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:55.393885  7537 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:55.394807  7537 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:55.381541  7820 consensus_queue.cc:1059] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:25:55.395000  7849 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:55.406574  7849 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:55.407823  7743 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:55.408370  7805 consensus_meta.cc:287] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.005s
I20260709 12:25:55.437639  7766 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:55.441692  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1656) took 1199 ms. Trace:
I20260709 12:25:55.442178  4602 rpcz_store.cc:276] 0709 12:25:54.243497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.163440 (+919943us) service_pool.cc:224] Handling call
0709 12:25:55.163887 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:55.164425 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:55.164466 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:55.165169 (+   703us) log.cc:844] Serialized 26 byte log entry
0709 12:25:55.256152 (+ 90983us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:55.256203 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.256291 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.441374 (+185083us) raft_consensus.cc:1685] finished
0709 12:25:55.441401 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.441616 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13720,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":7505,"thread_start_us":84105,"threads_started":1,"wal-append.queue_time_us":222771}
W20260709 12:25:55.444466  7864 consensus_meta.cc:287] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.000s
I20260709 12:25:55.445392  7923 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:55.396798  7820 raft_consensus.cc:3055] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:55.449954  7820 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:55.435962  7937 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:25:55.451020  7820 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:55.451907  7946 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.452472  7820 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:55.452430  7946 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:55.408957  7805 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:55.397068  7787 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:55.522686  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2534) took 1564 ms. Trace:
I20260709 12:25:55.523180  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:55.452736  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:55.525404  4677 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:55.493328  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.559275  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:55.523157  4751 rpcz_store.cc:276] 0709 12:25:53.959772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.971880 (+ 12108us) service_pool.cc:224] Handling call
0709 12:25:53.993734 (+ 21854us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:54.016916 (+ 23182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.016962 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:54.041919 (+ 24957us) log.cc:844] Serialized 52 byte log entry
0709 12:25:55.102341 (+1060422us) spinlock_profiling.cc:243] Waited 41.5 ms on lock 0x7b1800147e20. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:25:55.250683 (+148342us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:55.287655 (+ 36972us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.287861 (+   206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.287953 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.459202 (+171249us) raft_consensus.cc:1685] finished
0709 12:25:55.522462 (+ 63260us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.522626 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10041,"prepare.queue_time_us":97783,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":67168,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":9963,"spinlock_wait_cycles":95491968,"thread_start_us":176212,"threads_started":2,"wal-append.queue_time_us":1246077}
W20260709 12:25:55.695138  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.841814  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 282597us
I20260709 12:25:55.842628  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:55.827617  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2564) took 3149 ms. Trace:
W20260709 12:25:55.622231  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.843842  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:55.843429  4678 rpcz_store.cc:276] 0709 12:25:52.678490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.235074 (+556584us) service_pool.cc:224] Handling call
0709 12:25:53.371836 (+136762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.405480 (+ 33644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.827316 (+2421836us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54001bbf50. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:55.827496 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234695040}
I20260709 12:25:55.707150  4680 raft_consensus.cc:3055] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:55.844993  4680 raft_consensus.cc:740] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:55.846699  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.848096  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.617774  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2586) took 1057 ms. Trace:
I20260709 12:25:55.560606  7787 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:55.524848  7840 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.849841  4675 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:55.850687  4677 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
W20260709 12:25:55.660296  7955 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:25:55.584861  7913 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:55.599709  5341 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:55.887956  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 486) took 1142 ms. Trace:
W20260709 12:25:55.762480  7921 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.089s	user 0.007s	sys 0.001s
I20260709 12:25:55.586781  7923 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:55.586781  7771 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:25:55.831598  7925 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.255s	user 0.010s	sys 0.000s
I20260709 12:25:55.849593  4679 rpcz_store.cc:276] 0709 12:25:54.573672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.280612 (+706940us) service_pool.cc:224] Handling call
0709 12:25:55.280916 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.281164 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:55.385365 (+104201us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.614991 (+229626us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:55.615054 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:55.615102 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.617431 (+  2329us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.617641 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":165853,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":81,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":25266,"thread_start_us":311021,"threads_started":2}
I20260709 12:25:55.893231  4675 rpcz_store.cc:276] 0709 12:25:54.745419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.591943 (+846524us) service_pool.cc:224] Handling call
0709 12:25:55.592377 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.887684 (+295307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.887894 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:55.557905  7808 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:25:55.902855  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.772991  7766 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.892947  7840 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:55.903684  7766 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:55.904609  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.905067  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.832253  7537 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:55.812126  7946 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:55.890378  4680 consensus_queue.cc:260] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:55.792410  7805 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:55.849649  7846 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:55.857949  7864 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:55.858215  7894 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:25:55.899619  7734 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.018s	sys 0.056s
W20260709 12:25:55.905504  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.710215  7939 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.481s	user 0.001s	sys 0.007s
I20260709 12:25:55.897121  7923 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:55.918783  4680 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:55.925956  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:55.928479  5349 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:55.917699  7771 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:55.895071  7913 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:55.907277  7862 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:55.905483  7840 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:55.923245  7734 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.018s	sys 0.056s
I20260709 12:25:55.930804  7923 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:55.907068  7808 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:55.929894  7805 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:55.945132  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.928376  7766 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:55.961048  5349 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:55.965808  4644 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:55.950212  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 478) took 2030 ms. Trace:
I20260709 12:25:55.950281  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 479) took 2029 ms. Trace:
I20260709 12:25:55.922405  7846 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:55.969965  4674 rpcz_store.cc:276] 0709 12:25:53.911247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.304277 (+393030us) service_pool.cc:224] Handling call
0709 12:25:54.396494 (+ 92217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.398204 (+  1710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.398235 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:54.426762 (+ 28527us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.587344 (+160582us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:54.587383 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.587446 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.761809 (+1174363us) spinlock_profiling.cc:243] Waited 7.66 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:55.934966 (+173157us) raft_consensus.cc:1685] finished
0709 12:25:55.934990 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.940492 (+  5502us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":583,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":17635840,"thread_start_us":146451,"threads_started":1,"wal-append.queue_time_us":201058}
I20260709 12:25:55.922739  7864 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:55.978704  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.980984  4600 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:25:55.981319  4600 pending_rounds.cc:85] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 5
I20260709 12:25:55.981528  4600 pending_rounds.cc:107] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:25:55.981773  4600 raft_consensus.cc:2889] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:55.941351  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:55.951496  7862 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:55.984236  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 140469us
I20260709 12:25:55.987313  7968 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:55.950187  7771 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:25:55.986235  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.960711  7840 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.988018  7968 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:55.994009  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:56.000908  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.001679  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:56.013996  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.014686  4600 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:56.014147  7972 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.018110  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:56.017544  4568 leader_election.cc:400] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7.
I20260709 12:25:55.967588  4676 rpcz_store.cc:276] 0709 12:25:53.910960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.295896 (+384936us) service_pool.cc:224] Handling call
0709 12:25:54.396494 (+100598us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:54.427385 (+ 30891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.427416 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:54.448746 (+ 21330us) log.cc:844] Serialized 52 byte log entry
0709 12:25:54.576465 (+127719us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:54.582066 (+  5601us) log.cc:844] Serialized 14 byte log entry
0709 12:25:54.583460 (+  1394us) log.cc:844] Serialized 14 byte log entry
0709 12:25:54.583667 (+   207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.583738 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.868907 (+285169us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b1800016cc0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:25:55.830859 (+961952us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:55.940917 (+110058us) raft_consensus.cc:1685] finished
0709 12:25:55.940941 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.941090 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":742,"prepare.queue_time_us":624,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":7800,"raft.run_cpu_time_us":408,"raft.run_wall_time_us":15860,"spinlock_wait_cycles":24090624,"thread_start_us":45866,"threads_started":1,"wal-append.queue_time_us":91163}
I20260709 12:25:56.020567  4568 leader_election.cc:403] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:25:56.023082  4600 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:56.024227  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.025058  4600 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:25:56.030990  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.033983  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.047686  4597 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:56.034837  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:55.960606  7805 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:55.994148  7862 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:56.041363  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.041631  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:56.036401  7925 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:25:56.002807  7771 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.050813  7974 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:56.051653  4596 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:56.058687  4680 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:25:56.059988  7975 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7.
W20260709 12:25:56.062525  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.063302  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:56.064616  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:56.075773  4679 raft_consensus.cc:2393] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:56.046759  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:56.068946  4680 pending_rounds.cc:85] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 4
I20260709 12:25:56.075574  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:56.018965  7840 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:56.077008  4676 raft_consensus.cc:2368] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6.
I20260709 12:25:56.065347  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:56.092113  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:56.097826  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:56.091970  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 90403.2us
I20260709 12:25:56.062780  7766 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:56.099643  4680 pending_rounds.cc:107] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:25:56.100598  4679 raft_consensus.cc:2368] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6.
W20260709 12:25:56.101008  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 78890.5us
I20260709 12:25:56.100841  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:56.102018  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:56.102550  4568 leader_election.cc:400] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9.
I20260709 12:25:56.102598  4677 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:56.102735  4680 raft_consensus.cc:2889] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:56.061266  7894 raft_consensus.cc:697] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 7 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:56.037233  7972 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:56.104352  7734 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:25:56.105741  7966 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:25:56.108219  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2593) took 1251 ms. Trace:
I20260709 12:25:56.112226  4680 rpcz_store.cc:276] 0709 12:25:54.856890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.625089 (+768199us) service_pool.cc:224] Handling call
0709 12:25:55.668544 (+ 43455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.103561 (+435017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.108168 (+  4607us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:56.113289  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:56.111756  4571 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:56.114501  4597 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:25:56.117770  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:56.109014  4568 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:56.118439  4677 raft_consensus.cc:2376] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8. Re-sending same reply.
I20260709 12:25:56.118412  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.098860  7974 raft_consensus.cc:697] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:56.122464  4679 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:25:56.120203  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.124924  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:56.064380  7970 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:25:56.123658  4679 pending_rounds.cc:85] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 5
I20260709 12:25:56.130167  4679 pending_rounds.cc:107] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:25:56.130455  4679 raft_consensus.cc:2889] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:56.133213  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2590) took 1499 ms. Trace:
I20260709 12:25:56.128197  7981 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:56.098083  7865 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:25:56.135500  7771 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:56.136106  7771 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:25:56.136832  7741 consensus_meta.cc:287] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.744s	user 0.007s	sys 0.004s
W20260709 12:25:56.137699  7965 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:25:56.137866  7846 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:56.141026  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.142539  7894 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:56.143322  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:56.146178  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:56.114228  7734 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:56.149025  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.152808  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:56.150079  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:56.150780  7982 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:56.151363  7865 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:56.151546  7970 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:56.156719  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.132710  7913 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:56.163712  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:56.166319  7868 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 2.670s	user 0.023s	sys 0.016s
I20260709 12:25:56.166904  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:56.173666  4596 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:56.157591  7970 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:56.170364  7862 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:56.169298  7982 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 12:25:56.123875  7968 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:56.192123  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.152256  7850 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:56.195750  4605 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:56.183054  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:56.152235  7972 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:56.185837  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.201468  4597 raft_consensus.cc:2410] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:56.202301  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:56.203294  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:56.203814  4597 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:25:56.204607  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.186137  7805 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:56.154515  7808 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:56.200258  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.205530  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.206693  4597 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:56.206902  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.207841  4598 raft_consensus.cc:2393] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:56.208951  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.210150  4598 raft_consensus.cc:2393] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:56.157341  7865 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.159986  7864 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:56.211093  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:56.170787  7976 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:56.217914  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:56.218223  7976 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.218616  7976 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:56.218904  7734 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:56.220752  7774 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:56.175943  7913 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.222358  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:56.224530  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:56.225854  4677 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 6: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:25:56.227088  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.227931  5346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 465) took 1579 ms. Trace:
I20260709 12:25:56.158336  7846 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 12:25:56.229640  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 130202us
I20260709 12:25:56.228300  5346 rpcz_store.cc:276] 0709 12:25:54.642537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.642649 (+   112us) service_pool.cc:224] Handling call
0709 12:25:56.221984 (+1579335us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b5400782dd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:56.222111 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":982169984}
I20260709 12:25:56.217360  4597 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:56.200309  7850 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:56.238076  4720 leader_election.cc:336] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.604s (SENT)
I20260709 12:25:56.239355  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2038) took 2990 ms. Trace:
I20260709 12:25:56.214002  7865 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:56.238854  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:56.224700  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:56.238898  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:56.242105  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:56.242611  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:25:56.243463  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 114557us
I20260709 12:25:56.243698  4597 raft_consensus.cc:2410] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:25:56.244626  4598 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:56.240458  4754 rpcz_store.cc:276] 0709 12:25:53.248732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.248864 (+   132us) service_pool.cc:224] Handling call
0709 12:25:53.249292 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.272414 (+ 23122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.272449 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.283150 (+ 10701us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.384445 (+101295us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:53.384499 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.384891 (+   392us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:56.165731 (+2780840us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:25:56.229726 (+ 63995us) raft_consensus.cc:1685] finished
0709 12:25:56.229747 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.239249 (+  9502us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10566,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":13509,"spinlock_wait_cycles":11953920,"thread_start_us":45671,"threads_started":1,"wal-append.queue_time_us":146049}
I20260709 12:25:56.250690  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:56.247151  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 84000.1us
I20260709 12:25:56.249987  5349 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:25:56.258078  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:56.262346  5343 raft_consensus.cc:2393] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 4: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:56.262805  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:56.263586  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:56.250763  7846 consensus_queue.cc:1243] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:56.250490  7885 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:56.266508  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:56.268339  4677 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:56.243948  7766 raft_consensus.cc:697] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:25:56.261919  7820 consensus_meta.cc:287] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.809s	user 0.016s	sys 0.001s
I20260709 12:25:56.270011  5341 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:56.264724  5342 raft_consensus.cc:2368] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6.
W20260709 12:25:56.273118  7912 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 1.604s	user 0.000s	sys 0.008s
I20260709 12:25:56.287884  4644 leader_election.cc:400] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6.
I20260709 12:25:56.288223  4644 leader_election.cc:403] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 12:25:56.274247  4673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1654) took 2520 ms (client timeout 2342 ms). Trace:
I20260709 12:25:56.275594  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 510) took 2271 ms. Trace:
W20260709 12:25:56.267272  4720 leader_election.cc:336] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.612s (SENT)
I20260709 12:25:56.229337  7913 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.240010  7978 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
W20260709 12:25:56.310335  4720 leader_election.cc:336] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.693s (SENT)
I20260709 12:25:56.292101  4753 rpcz_store.cc:276] 0709 12:25:54.004560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.740878 (+736318us) service_pool.cc:224] Handling call
0709 12:25:55.097892 (+357014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.098719 (+   827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.275403 (+1176684us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b54001f2550. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:56.275539 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819346176}
I20260709 12:25:56.288831  7968 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 5. Current term is 6.
I20260709 12:25:56.313714  4677 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:56.314859  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:56.290621  4673 rpcz_store.cc:269] 0709 12:25:53.753889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.788740 (+ 34851us) service_pool.cc:224] Handling call
0709 12:25:56.274001 (+2485261us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b5400086454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:56.274149 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132487936}
I20260709 12:25:56.284019  4596 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:56.310814  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:56.255574  7983 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:25:56.312088  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.266674  7840 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:56.226992  7974 consensus_queue.cc:237] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:56.319337  4720 leader_election.cc:336] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.711s (SENT)
I20260709 12:25:56.321903  4720 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:56.257673  7982 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:25:56.321170  7734 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:56.324697  7734 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:56.327092  7734 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:56.256426  7970 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.330317  7970 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:56.331234  7787 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:56.332201  4597 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:56.322728  7983 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.317929  7922 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:56.334508  7983 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:56.336256  7913 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:56.336019  7923 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.338991  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:56.343163  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.351547  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 102667us
W20260709 12:25:56.352311  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.333506  7787 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:56.267540  7850 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.357971  4673 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:56.359203  7766 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.347043  7922 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:56.370172  7985 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:25:56.345310  7978 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:56.447831  7850 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:56.448654  7097 raft_consensus.cc:3055] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:56.449126  7097 raft_consensus.cc:740] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:56.449642  7971 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.449947  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:56.452558  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:56.715823  4674 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:56.716027  4680 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:56.624006  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.492648  7982 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.226s
W20260709 12:25:56.468632  7984 log.cc:927] Time spent T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 12:25:56.637941  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.450443  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:56.721241  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:56.510561  7923 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:56.482034  7777 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.242s
I20260709 12:25:56.723528  7777 raft_consensus.cc:3055] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:56.724112  7777 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:56.483669  7976 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.725066  7777 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:56.521992  7896 log.cc:927] Time spent T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 2.456s	user 0.000s	sys 0.007s
I20260709 12:25:56.482601  7922 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.541647  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.452003  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:56.450273  7971 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.452036  7865 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.450762  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:56.855327  7987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.010s	sys 0.000s
I20260709 12:25:56.728763  7922 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.463896  7097 consensus_queue.cc:260] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:56.734232  7983 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.938308  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.938724  7097 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:56.714318  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.726617  7976 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:56.874719  4597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.043s	sys 0.058s
I20260709 12:25:56.717628  7766 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:56.493825  7978 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.920091  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:56.722177  4679 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:56.745028  7717 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:56.923864  4680 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:25:56.943457  7777 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:56.944324  7968 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:56.948104  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:56.948691  7992 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.446s	user 0.000s	sys 0.001s
I20260709 12:25:56.509483  7863 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:56.949796  7863 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:56.950909  7850 consensus_meta.cc:287] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.000s
W20260709 12:25:56.943747  7970 consensus_meta.cc:287] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.613s	user 0.004s	sys 0.007s
I20260709 12:25:56.952088  7971 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:56.952608  7971 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:56.953037  7971 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.955489  7971 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:56.957309  7976 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.957739  7978 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:56.958387  7976 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.960121  7978 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:56.963662  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.962513  7976 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:56.966876  7922 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:56.970045  4680 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:25:56.973137  7865 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:56.946497  7983 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:56.975725  7863 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.929051  4674 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:56.931949  4673 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:56.468765  4676 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:56.933547  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 477) took 3864 ms. Trace:
W20260709 12:25:56.937266  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 617917us
W20260709 12:25:56.942780  4597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.044s	sys 0.058s
I20260709 12:25:56.948881  7968 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:55.283895  4601 rpcz_store.cc:276] 0709 12:25:53.760954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.740084 (+979130us) service_pool.cc:224] Handling call
0709 12:25:54.753713 (+ 13629us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.753972 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:54.899711 (+145739us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.282412 (+382701us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:55.282546 (+   134us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:55.282610 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.282692 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.282976 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":59006,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":14439,"raft.queue_time_us":5369,"thread_start_us":444033,"threads_started":2}
I20260709 12:25:57.028671  4672 rpcz_store.cc:276] 0709 12:25:53.069618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.558567 (+488949us) service_pool.cc:224] Handling call
0709 12:25:53.574768 (+ 16201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.907217 (+332449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.933210 (+3025993us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54002dee54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:56.933471 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762760960}
I20260709 12:25:57.031517  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.031950  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:56.977515  7983 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:56.977646  7766 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:56.990760  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1657) took 2747 ms. Trace:
I20260709 12:25:56.988279  7849 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:56.998595  4755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2531) took 3512 ms (client timeout 2005 ms). Trace:
W20260709 12:25:57.041458  4755 rpcz_store.cc:269] 0709 12:25:53.486025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.508698 (+ 22673us) service_pool.cc:224] Handling call
0709 12:25:56.998335 (+3489637us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b54000730d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:56.998512 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1094654336}
I20260709 12:25:56.999619  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2072) took 3206 ms. Trace:
I20260709 12:25:57.043649  7968 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:57.042598  4604 rpcz_store.cc:276] 0709 12:25:53.793422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.804256 (+1010834us) service_pool.cc:224] Handling call
0709 12:25:54.804971 (+   715us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.862418 (+ 57447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.862479 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:54.892823 (+ 30344us) log.cc:844] Serialized 26 byte log entry
0709 12:25:55.168389 (+275566us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:55.168448 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.999249 (+1830801us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b5400034654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:56.999294 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:56.999343 (+    49us) raft_consensus.cc:1685] finished
0709 12:25:56.999354 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.999562 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8882,"raft.queue_time_us":9036,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":6336,"spinlock_wait_cycles":110976512,"thread_start_us":249806,"threads_started":1,"wal-append.queue_time_us":331360,"wal-append.run_cpu_time_us":2167,"wal-append.run_wall_time_us":1100074}
I20260709 12:25:57.044534  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.002256  7886 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.045858  7886 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:57.047092  7972 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:57.047462  7972 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:57.047956  7886 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:56.133745  4681 rpcz_store.cc:276] 0709 12:25:54.634316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.410856 (+776540us) service_pool.cc:224] Handling call
0709 12:25:55.411191 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.411395 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:55.902298 (+490903us) log.cc:844] Serialized 14 byte log entry
0709 12:25:56.132801 (+230503us) log.cc:844] Serialized 14 byte log entry
0709 12:25:56.132922 (+   121us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:56.132949 (+    27us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:56.132984 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.133026 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.133163 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":42680,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":55479,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":4656,"spinlock_wait_cycles":39503104,"thread_start_us":719092,"threads_started":3,"wal-append.queue_time_us":217488}
I20260709 12:25:56.981459  4679 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:25:57.048300  7972 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:56.925009  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.050230  7785 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:57.029315  4720 leader_election.cc:336] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.887s (SENT)
I20260709 12:25:57.052809  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:57.053253  4747 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:25:57.053746  4747 pending_rounds.cc:85] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 4
I20260709 12:25:57.054023  4747 pending_rounds.cc:107] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:25:57.054297  4747 raft_consensus.cc:2889] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:57.055938  4754 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:57.056201  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:57.057003  4748 raft_consensus.cc:2410] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:57.057559  4756 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:57.058615  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.059206  4749 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:57.060070  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.060792  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:57.036284  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.039999  4603 rpcz_store.cc:276] 0709 12:25:54.243881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.196145 (+952264us) service_pool.cc:224] Handling call
0709 12:25:55.211095 (+ 14950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.305428 (+ 94333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.990492 (+1685064us) spinlock_profiling.cc:243] Waited 93.3 ms on lock 0x7b54003d0254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:56.990697 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214689024}
I20260709 12:25:57.045329  7766 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:57.060947  4752 raft_consensus.cc:2410] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:57.057005  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.062565  4754 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:57.063086  4751 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:57.064751  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.065840  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:57.066759  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.049305  7972 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:57.069192  4750 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:57.070091  4751 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:57.070353  4604 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:25:57.027107  7885 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.008s	sys 0.031s
I20260709 12:25:57.061358  7998 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:57.071121  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.072126  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.071610  4749 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:57.078123  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:57.074561  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.079423  4748 raft_consensus.cc:2393] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:57.079543  4598 raft_consensus.cc:2410] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:57.079428  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.080432  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.080585  4752 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:57.081290  4749 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 3.
I20260709 12:25:57.071610  4756 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:57.077081  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.071573  7887 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.088562  4751 raft_consensus.cc:2393] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:57.073993  7849 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.090938  4747 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:57.077732  7885 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.008s	sys 0.031s
I20260709 12:25:57.076611  7886 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:57.109596  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.088583  7832 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:57.091426  7891 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 3.187s	user 0.009s	sys 0.027s
I20260709 12:25:57.110445  4751 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:57.106222  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.114390  4756 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:57.111160  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 487) took 1094 ms. Trace:
W20260709 12:25:57.121074  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.199363  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.199142  4598 rpcz_store.cc:276] 0709 12:25:56.016789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.096372 (+1079583us) service_pool.cc:224] Handling call
0709 12:25:57.110620 (+ 14248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.110797 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.110830 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.110880 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:57.110920 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.110990 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.111106 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:57.112287  7886 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:57.095177  7968 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:57.152077  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.154284  4747 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
W20260709 12:25:57.187445  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.197073  4751 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:57.144511  4752 raft_consensus.cc:3055] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:57.204223  4752 raft_consensus.cc:740] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:57.202924  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:57.210733  4750 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:57.102679  7849 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:57.202857  4749 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:57.214035  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.100405  7887 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:57.250427  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 327555us
I20260709 12:25:57.251252  4598 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:57.254707  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 489) took 1237 ms. Trace:
I20260709 12:25:57.258991  4598 rpcz_store.cc:276] 0709 12:25:56.017331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.243440 (+1226109us) service_pool.cc:224] Handling call
0709 12:25:57.243963 (+   523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.254443 (+ 10480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.254650 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:57.266119  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:57.267901  7993 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
W20260709 12:25:57.255793  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 313849us
I20260709 12:25:57.271267  7887 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:57.271569  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.272476  7995 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 12:25:57.273716  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1667) took 1259 ms. Trace:
I20260709 12:25:57.274053  4597 rpcz_store.cc:276] 0709 12:25:56.014630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.333204 (+318574us) service_pool.cc:224] Handling call
0709 12:25:56.333585 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.333943 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.333973 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.334458 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.030238 (+695780us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:57.030300 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.030398 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.273494 (+243096us) raft_consensus.cc:1685] finished
0709 12:25:57.273512 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.273656 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1269,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":695302,"threads_started":1,"wal-append.queue_time_us":729544}
I20260709 12:25:57.276083  4597 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:57.276620  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 494) took 1248 ms. Trace:
I20260709 12:25:57.276889  4597 rpcz_store.cc:276] 0709 12:25:56.028576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.275433 (+1246857us) service_pool.cc:224] Handling call
0709 12:25:57.275752 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.276405 (+   653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.276571 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:57.107990  7807 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:57.280560  4645 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:57.146986  7832 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:57.253189  4752 consensus_queue.cc:260] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:57.293592  4752 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:57.302979  7820 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:57.307520  7886 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:57.320662  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.324188  7863 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:57.325610  7807 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:57.324234  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:57.328949  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:57.329782  4749 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:57.333369  4751 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:57.338011  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.339635  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:57.341549  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:57.342597  4751 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:57.219653  7968 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:57.310956  8000 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:25:57.349949  4645 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:57.360105  4752 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:57.377784  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.380318  4750 raft_consensus.cc:2410] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:25:57.382073  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.328442  7776 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:57.331797  7849 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:57.407624  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 353689us
W20260709 12:25:57.408380  4720 leader_election.cc:336] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 2.035s (SENT)
I20260709 12:25:57.408808  4752 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:57.408773  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:57.414004  4749 raft_consensus.cc:2410] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:25:57.350999  7840 consensus_queue.cc:1059] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:25:57.386246  4750 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:25:57.375808  7820 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:25:57.422534  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.425150  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.427510  4752 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 7: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:57.429117  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.387670  7717 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:25:57.436233  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.451298  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.452509  4752 raft_consensus.cc:2393] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:57.437547  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:57.418253  7776 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:25:57.426553  7981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.275s	user 0.022s	sys 0.025s
I20260709 12:25:57.442234  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:57.453843  4599 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:57.444293  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.428579  7771 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:25:57.446880  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.455031  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.456404  4750 raft_consensus.cc:2393] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:25:57.446660  4644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.006s	sys 0.007s
W20260709 12:25:57.457875  4644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.006s	sys 0.007s
I20260709 12:25:57.461656  5348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 509) took 1213 ms. Trace:
I20260709 12:25:57.462205  5348 rpcz_store.cc:276] 0709 12:25:56.247942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.253826 (+  5884us) service_pool.cc:224] Handling call
0709 12:25:57.437215 (+1183389us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b5800003c98. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:25:57.461582 (+ 24367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26834176}
W20260709 12:25:57.461948  7981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.311s	user 0.022s	sys 0.026s
I20260709 12:25:57.475109  7981 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:57.475790  7968 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:57.476138  7981 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:57.477325  7779 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.477928  7779 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:57.482615  4720 connection.cc:256] RPC call timeout handler was delayed by -1.1190302529998917s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1617s for a response
W20260709 12:25:57.483199  4720 connection.cc:256] RPC call timeout handler was delayed by -1.1100302529998771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 12:25:57.483485  4720 connection.cc:256] RPC call timeout handler was delayed by -1.1100302529998771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 12:25:57.484495  4720 leader_election.cc:336] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 2.162s (SENT)
I20260709 12:25:57.485021  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:57.479240  5347 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:57.501679  7542 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:57.502446  4599 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
W20260709 12:25:57.503365  8006 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:25:57.461126  7849 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
W20260709 12:25:57.518184  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.522413  4751 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:56.141019  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:57.529241  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 44852.7us
I20260709 12:25:57.488291  7776 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:57.530563  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.488781  7771 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:57.532222  4720 leader_election.cc:336] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 2.223s (SENT)
I20260709 12:25:57.533195  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:57.538295  7776 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:25:57.542249  4604 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:25:57.542793  4604 pending_rounds.cc:85] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 3
I20260709 12:25:57.543047  4604 pending_rounds.cc:107] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:25:57.543318  4604 raft_consensus.cc:2889] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:57.547088  8010 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:57.549234  7776 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:57.551254  7771 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:57.553972  7771 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:57.555027  7945 raft_consensus.cc:3055] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:57.556058  7945 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:57.557049  7945 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:57.564103  7945 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:57.568116  7832 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:57.571313  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.573470  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:57.574151  4602 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:57.576754  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2595) took 2685 ms. Trace:
I20260709 12:25:57.577378  4678 rpcz_store.cc:276] 0709 12:25:54.891677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.852342 (+960665us) service_pool.cc:224] Handling call
0709 12:25:55.852731 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.982332 (+129601us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:55.982361 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:55.982401 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:55.982438 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.576379 (+1593941us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b540038e5d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:57.576449 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.576663 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":297587456}
W20260709 12:25:57.590263  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 58125.3us
I20260709 12:25:57.598213  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:57.607671  4602 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:25:57.608855  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:57.632470  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.648228  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:57.601760  7913 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.615402  7771 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:25:57.687695  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 145150us
I20260709 12:25:57.654316  7840 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:57.609603  4602 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:25:57.715852  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.716259  4571 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:57.717133  7972 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:57.715847  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.717653  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.707863  4750 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:57.718538  4681 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:25:57.708071  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.709718  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.710363  7913 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:57.713950  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.723634  4679 raft_consensus.cc:2393] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:57.647758  7998 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
W20260709 12:25:57.719491  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 121395us
I20260709 12:25:57.735457  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:57.737972  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:57.714154  7776 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.740869  7776 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:57.742630  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.743885  7776 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:57.743875  4678 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:25:57.719641  4599 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:25:57.729095  4674 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:57.757732  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.729714  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.719867  4750 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:57.756385  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.714216  7542 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:57.762974  4602 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:25:57.764190  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 316836us
W20260709 12:25:57.765079  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47645.1us
I20260709 12:25:57.768186  4677 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:25:57.770807  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:57.776886  4750 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:57.778807  4599 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:25:57.784751  4596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 490) took 1767 ms (client timeout 1848 ms). Trace:
W20260709 12:25:57.785235  4596 rpcz_store.cc:269] 0709 12:25:56.026558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.153092 (+126534us) service_pool.cc:224] Handling call
0709 12:25:57.784536 (+1631444us) spinlock_profiling.cc:243] Waited 9.15 ms on lock 0x7b5400033254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:57.784679 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21037312}
I20260709 12:25:57.787036  4601 raft_consensus.cc:1240] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:57.784276  4571 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:57.792604  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.793558  4602 raft_consensus.cc:2410] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:57.793673  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:57.796396  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.798166  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:57.762120  8003 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.801543  4596 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:25:57.723307  7840 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:57.810765  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:57.815760  4681 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:57.788587  7887 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:57.765254  7849 consensus_queue.cc:1059] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:25:57.814517  4571 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:57.841069  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:57.818226  8012 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.819026  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.822100  4673 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:57.847364  4681 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:57.825691  7864 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:57.826670  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.806483  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:57.838635  7913 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:57.843317  4754 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:57.845185  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.823813  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.854708  4596 raft_consensus.cc:2410] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:57.802261  7734 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:57.815418  7832 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.806352  8002 raft_consensus.cc:3055] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:57.851931  4571 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:57.866078  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:57.872498  5312 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:57.873024  4674 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:57.865765  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.874348  4681 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:57.850610  8012 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:57.842720  7887 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.871251  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:57.856604  7796 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:57.881888  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:57.883477  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.851377  7886 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.857097  7734 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:57.885064  4596 raft_consensus.cc:2376] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6. Re-sending same reply.
I20260709 12:25:57.811148  8003 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:57.885274  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.889911  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.866828  7972 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:57.867982  7864 raft_consensus.cc:3055] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:57.902999  4675 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:57.910076  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.910782  4675 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:57.911062  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.814435  8018 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 12:25:57.872380  8002 raft_consensus.cc:740] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:57.886880  7856 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:57.915066  4681 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:57.911679  4673 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:25:57.915513  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.906266  4674 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:57.886317  7787 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:57.887012  7887 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:57.898584  7734 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:57.916420  4675 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:57.925102  4675 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:57.916445  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:57.925796  4673 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:25:57.907227  7972 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.917308  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:25:57.929543  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.932274  5312 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:57.912828  7981 raft_consensus.cc:2764] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:25:57.933827  8037 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:57.888860  7849 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:25:57.907886  7796 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:57.909199  7864 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:57.886724  7785 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:25:57.941991  4673 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:57.884091  7832 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:57.923475  7787 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.918687  7856 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.947430  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:57.947521  4673 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:25:57.948603  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.940573  7972 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:57.906833  7886 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:57.948684  4674 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:57.947510  7743 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:57.960780  7846 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:57.960870  7743 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.961395  7846 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:57.961611  7743 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:57.962471  7846 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:57.963937  7846 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:57.965962  7796 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:57.969105  4672 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:57.951586  7856 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:57.970374  7840 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:57.973340  7886 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:57.977766  7864 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:57.978272  4596 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:57.979023  7864 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:57.981227  4755 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:57.982679  4605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 496) took 1953 ms (client timeout 1627 ms). Trace:
W20260709 12:25:57.983067  4605 rpcz_store.cc:269] 0709 12:25:56.029229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.062978 (+ 33749us) service_pool.cc:224] Handling call
0709 12:25:57.982516 (+1919538us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b5400343854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:57.982612 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277402880}
I20260709 12:25:57.988363  4754 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:57.991230  4754 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:57.992980  4754 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:57.955772  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 495) took 1926 ms. Trace:
I20260709 12:25:57.998610  4673 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:57.999642  4676 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:57.997889  4597 rpcz_store.cc:276] 0709 12:25:56.028872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.277514 (+1248642us) service_pool.cc:224] Handling call
0709 12:25:57.284045 (+  6531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.385024 (+100979us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:57.606227 (+221203us) log.cc:844] Serialized 14 byte log entry
0709 12:25:57.955314 (+349087us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.955389 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:57.955451 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.955556 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.955698 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":161047,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"thread_start_us":411989,"threads_started":2}
W20260709 12:25:58.005648  5313 connection.cc:256] RPC call timeout handler was delayed by -1.3377360890003729s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:25:57.951926  7787 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.019240  4755 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:58.003379  7840 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:25:58.022023  4597 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.6-12.6]
W20260709 12:25:57.983755  7807 consensus_meta.cc:287] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.005s
I20260709 12:25:58.024261  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:57.957116  7981 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:25:58.025760  5341 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:58.040218  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:58.033607  4569 connection.cc:256] RPC call timeout handler was delayed by -1.6845554339997761s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1663s for a response
I20260709 12:25:58.023660  8003 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:58.037766  5343 raft_consensus.cc:1240] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:58.038096  5345 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:58.037997  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:25:58.041385  4569 connection.cc:256] RPC call timeout handler was delayed by -1.3858172589998503s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
I20260709 12:25:58.020000  7886 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.043079  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:58.042156  4603 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:58.051851  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:58.050359  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:58.055115  4597 raft_consensus.cc:2376] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9. Re-sending same reply.
W20260709 12:25:58.061218  8027 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:25:58.051744  7886 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:58.089403  8021 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:25:58.089004  7856 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.050115  7986 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.090272  7986 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.091851  7856 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:58.007403  8002 consensus_queue.cc:260] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.094033  8002 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.095100  8012 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:58.096693  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 491) took 2069 ms. Trace:
I20260709 12:25:58.047802  7945 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.097651  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:58.053736  8035 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.055s
W20260709 12:25:58.102947  4569 connection.cc:256] RPC call timeout handler was delayed by -1.1803211239998745s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
I20260709 12:25:58.103615  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:58.104379  7945 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.104549  7846 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:58.103710  8035 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.038218  7840 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.105325  7846 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.105518  8035 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.105902  7846 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:58.107206  7840 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:58.107721  8035 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:58.104612  4569 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:58.038429  7887 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.117945  8035 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:58.118101  7887 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:58.118731  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:58.124393  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.129345  7886 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:58.129760  8038 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:58.130229  8038 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:58.130611  8038 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:58.129177  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:58.135135  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:58.097078  4598 rpcz_store.cc:276] 0709 12:25:56.027046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.261462 (+1234416us) service_pool.cc:224] Handling call
0709 12:25:57.261877 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.262105 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:57.599864 (+337759us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.048331 (+448467us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.096355 (+ 48024us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.096406 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:58.096457 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.096527 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.096638 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57741,"prepare.queue_time_us":226712,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":1959,"raft.queue_time_us":128358,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":12976,"thread_start_us":781997,"threads_started":3,"wal-append.queue_time_us":356633}
I20260709 12:25:58.136044  5342 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:58.136310  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:58.137200  4598 raft_consensus.cc:2410] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:58.137982  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:58.138916  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:58.139650  5346 raft_consensus.cc:2393] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 7: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:58.140055  7972 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.141273  5342 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:58.138916  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:58.134976  7734 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.148402  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:58.045301  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 111002us
I20260709 12:25:58.154431  4569 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:58.161738  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:58.162691  5345 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:58.153020  8036 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:25:58.164157  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:58.164507  5346 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:58.165756  7846 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.166651  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:58.142972  7972 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:58.168061  4598 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:25:58.168581  7846 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:58.169855  7840 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:58.169323  4598 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:58.174264  4597 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:58.178915  7840 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:58.181329  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:58.182046  8036 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:58.189646  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:58.204043  4597 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:58.217695  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:58.225060  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:58.226370  4569 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:58.228873  4600 raft_consensus.cc:2376] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7. Re-sending same reply.
I20260709 12:25:58.234814  4645 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:58.236526  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:58.240784  4645 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:58.238634  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:58.246812  4645 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:58.254026  4675 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:58.232533  8003 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.255913  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:58.232758  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:58.233664  7734 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:58.243373  7983 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:58.248386  7776 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:58.248185  8039 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:58.242164  8042 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:58.265375  7849 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:58.276721  5314 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:58.272826  7785 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:25:58.251628  7779 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:58.265194  7972 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.248482  8026 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:58.248435  8019 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:58.288760  7796 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:58.307708  7796 raft_consensus.cc:697] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:58.308693  7796 consensus_queue.cc:237] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.308836  7743 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.310973  7743 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:58.311805  5348 raft_consensus.cc:2393] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:58.312253  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:25:58.292757  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.293756  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:58.314361  5346 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
I20260709 12:25:58.273439  8012 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:25:58.294420  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:58.251382  8038 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.326965  8038 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:58.301802  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:58.274453  8039 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:25:58.296376  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49688.9us
I20260709 12:25:58.282428  7983 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:58.267910  8003 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.278942  8011 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:25:58.323872  4568 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:58.324090  4569 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:58.322065  8012 raft_consensus.cc:3055] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:58.328682  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2089) took 1389 ms. Trace:
I20260709 12:25:58.339584  4603 rpcz_store.cc:276] 0709 12:25:56.939090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.068290 (+129200us) service_pool.cc:224] Handling call
0709 12:25:58.328515 (+1260225us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b5400352ad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:58.328623 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2217060608}
I20260709 12:25:58.329658  4599 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:58.301055  7807 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:58.332345  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.335248  4645 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:58.284520  7787 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.335410  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.344134  4752 raft_consensus.cc:2410] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:58.345917  4568 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:58.306905  7832 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:58.332604  8026 raft_consensus.cc:697] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:58.355755  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.300079  7972 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.334946  8039 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:58.346112  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:58.333972  7849 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:58.335979  7945 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:58.342543  7807 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.362210  4645 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:58.368076  4569 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:58.341398  8012 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:58.350652  8038 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:58.379748  7983 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.380102  7807 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.371682  7975 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:58.359686  7886 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:58.380635  7782 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:58.381007  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.354519  8040 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:58.353871  7913 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:58.388365  8049 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:58.388458  8036 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:25:58.389137  5347 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 501) took 4810 ms (client timeout 1733 ms). Trace:
W20260709 12:25:58.409970  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.393705  7975 raft_consensus.cc:697] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:58.398808  7782 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.391669  7983 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.389775  7887 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:58.406098  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.422441  4747 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:58.387866  8038 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.370929  7849 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:58.408263  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:58.410043  5347 rpcz_store.cc:269] 0709 12:25:53.578252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.578411 (+   159us) service_pool.cc:224] Handling call
0709 12:25:58.388931 (+4810520us) spinlock_profiling.cc:243] Waited 3.89 s on lock 0x7b54002954d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:58.389042 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8954806656}
I20260709 12:25:58.422986  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2084) took 1020 ms. Trace:
I20260709 12:25:58.425459  4747 rpcz_store.cc:276] 0709 12:25:57.402245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.589223 (+186978us) service_pool.cc:224] Handling call
0709 12:25:58.405962 (+816739us) spinlock_profiling.cc:243] Waited 59.9 ms on lock 0x7b58000d2698. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:25:58.422928 (+ 16966us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137837824}
I20260709 12:25:58.405859  8026 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.366024  8042 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:25:58.430342  7782 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:58.431597  8003 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:58.431393  7975 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.432571  7864 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:58.434109  7787 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:58.407953  7981 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:25:58.434293  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:58.435180  5348 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:25:58.426584  4645 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:58.369764  8002 consensus_queue.cc:1059] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:25:58.436470  8002 raft_consensus.cc:3055] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:58.436914  8002 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:25:58.422073  8047 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:25:58.437960  8002 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.439594  8002 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:58.440156  8038 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.442369  4644 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:58.443087  7886 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:58.408363  8040 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.439687  7807 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.470410  8052 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:58.466486  8049 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:58.476435  8049 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.476822  8049 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:58.408069  8018 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 12:25:58.444897  7849 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.453379  7913 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:58.453935  7886 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:58.483247  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 92967.4us
I20260709 12:25:58.495607  4569 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:58.414044  8019 consensus_queue.cc:1059] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:25:58.439863  7972 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:58.465811  8040 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:58.501842  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:58.498482  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:58.503667  4678 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:58.504262  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:58.502547  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:58.504944  4676 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.507869  4680 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:58.510010  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:58.510681  4681 raft_consensus.cc:2410] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:25:58.513414  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:58.513991  4674 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.515793  8038 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.499696  7981 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:58.517714  8053 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:58.517791  7981 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:58.518179  8053 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:58.518354  7981 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.519042  8053 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.520279  8053 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:58.471212  8012 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.499744  7787 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:25:58.522651  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:58.498020  7886 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.524904  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:58.526171  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:58.526283  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:58.526854  4603 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
W20260709 12:25:58.528218  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 174701us
I20260709 12:25:58.529078  4568 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:58.530565  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:58.532963  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.497624  7807 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:58.537464  5312 leader_election.cc:336] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Timed out: RequestConsensusVote RPC to 127.3.89.193:45077 timed out after 1.762s (SENT)
I20260709 12:25:58.537720  4598 raft_consensus.cc:2368] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10.
I20260709 12:25:58.537977  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:58.536963  4674 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:25:58.499053  7913 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:58.539279  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:58.539520  5312 leader_election.cc:336] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Timed out: RequestConsensusVote RPC to 127.3.89.193:45077 timed out after 1.785s (SENT)
I20260709 12:25:58.541026  8051 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:58.544792  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2085) took 2528 ms. Trace:
I20260709 12:25:58.549873  4604 rpcz_store.cc:276] 0709 12:25:56.015994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.089161 (+1073167us) service_pool.cc:224] Handling call
0709 12:25:57.089443 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.544016 (+454573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.544581 (+1000565us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b54002337d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:25:58.544737 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1040200576}
I20260709 12:25:58.544785  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:58.547648  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2090) took 1143 ms. Trace:
I20260709 12:25:58.551649  4755 rpcz_store.cc:276] 0709 12:25:57.404079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.981988 (+577909us) service_pool.cc:224] Handling call
0709 12:25:57.989629 (+  7641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.031255 (+ 41626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.547449 (+516194us) spinlock_profiling.cc:243] Waited 7.33 ms on lock 0x7b54001f2550. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:58.547578 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16860416}
I20260709 12:25:58.552603  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:58.547645  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:58.553231  8018 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:58.553691  7832 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:58.553771  8018 raft_consensus.cc:697] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:58.554821  8018 consensus_queue.cc:237] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.558399  5312 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:58.559456  7832 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:58.548416  4674 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:58.548265  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:58.548274  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:58.563242  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:58.563836  5347 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:58.564666  8051 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.528153  8012 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:58.567432  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.541026  7945 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:58.577826  8038 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:58.592656  8039 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.595567  8038 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:58.597306  8049 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.597805  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:58.598353  5344 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.602468  7913 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.606166  7983 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:58.612188  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 51190.1us
W20260709 12:25:58.615480  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.619316  4569 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:58.625780  4678 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:58.566123  7886 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:58.629702  8038 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:25:58.572311  5347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:25:58.646675  8049 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:58.545418  7796 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:25:58.649866  7779 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:58.630313  8011 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:25:58.565122  8051 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:58.630033  7807 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:58.665683  4602 raft_consensus.cc:2393] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:25:58.665759  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 239309us
I20260709 12:25:58.656867  4676 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:58.673785  4645 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:58.690670  5344 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:25:58.692214  4568 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:58.693703  8038 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:58.694362  8038 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:58.695501  8038 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.650210  7835 consensus_queue.cc:1059] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:25:58.677065  7807 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.701857  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:58.702023  7840 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:58.720863  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:58.724448  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1670) took 2709 ms. Trace:
I20260709 12:25:58.730762  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:58.734730  4569 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:25:58.741560  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.734294  4601 rpcz_store.cc:276] 0709 12:25:56.015401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.074068 (+1058667us) service_pool.cc:224] Handling call
0709 12:25:57.786797 (+712729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.787614 (+   817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.724241 (+936627us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b5400033250. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:58.724371 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1601976576}
I20260709 12:25:58.747548  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.748724  4753 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:58.749366  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2074) took 1412 ms. Trace:
I20260709 12:25:58.749727  4753 rpcz_store.cc:276] 0709 12:25:57.336910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.615432 (+278522us) service_pool.cc:224] Handling call
0709 12:25:58.748242 (+1132810us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b58000d2698. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff348ffc0 00007efff347a929 00007efff347a3d2 00007effec9f1bd4
0709 12:25:58.748417 (+   175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.749135 (+   718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.749314 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77760256}
W20260709 12:25:58.751827  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 50036.1us
I20260709 12:25:58.748622  4751 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:58.753530  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2089) took 1349 ms. Trace:
I20260709 12:25:58.754030  4751 rpcz_store.cc:276] 0709 12:25:57.403789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.523455 (+119666us) service_pool.cc:224] Handling call
0709 12:25:58.747171 (+1223716us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b58000d2698. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:25:58.753449 (+  6278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":345921792}
I20260709 12:25:58.755252  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2563) took 1665 ms. Trace:
I20260709 12:25:58.704982  7983 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:58.753647  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:58.730193  7807 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.768383  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2085) took 1365 ms. Trace:
I20260709 12:25:58.755672  4748 rpcz_store.cc:276] 0709 12:25:57.090311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.101151 (+ 10840us) service_pool.cc:224] Handling call
0709 12:25:57.110053 (+  8902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.110268 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:57.374789 (+264521us) log.cc:844] Serialized 14 byte log entry
0709 12:25:57.715251 (+340462us) log.cc:844] Serialized 14 byte log entry
0709 12:25:57.720404 (+  5153us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.747294 (+1026890us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b1800344200. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007effeb3cba40 00007effeb3c64e4 00007effeb3c6455 00007effee4ae060 00007effee4c0716 00007effee4d1287 00007effee4d1208
0709 12:25:58.747458 (+   164us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.747533 (+    75us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:58.747588 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.747659 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.755185 (+  7526us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1060434,"prepare.queue_time_us":390963,"prepare.run_cpu_time_us":139,"prepare.run_wall_time_us":2743,"raft.queue_time_us":1080897,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":10698,"spinlock_wait_cycles":8314624,"thread_start_us":451807,"threads_started":2,"wal-append.queue_time_us":446779}
W20260709 12:25:58.768419  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.751044  7846 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:58.746829  7886 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:58.723462  8037 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:58.810247  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:25:58.813652  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 60130.1us
I20260709 12:25:58.758780  7998 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:58.758812  7818 raft_consensus.cc:3055] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:58.758900  7771 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:58.822214  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.769511  7820 raft_consensus.cc:3055] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:58.826113  5312 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:58.828091  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:58.829591  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:58.830202  4673 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:58.846503  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.721508  8040 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.766978  8062 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:58.798580  7863 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:58.853384  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:58.819185  8049 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:58.859472  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:58.898696  4753 raft_consensus.cc:2393] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:58.859361  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.899345  4751 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:25:58.900624  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.860507  4747 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:58.903138  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.903870  4747 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:58.904837  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.860562  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.906080  4749 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:25:58.861176  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:58.907361  4752 raft_consensus.cc:2376] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6. Re-sending same reply.
I20260709 12:25:58.908452  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.861852  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.910187  4754 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:25:58.911566  4754 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:58.862749  4647 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:58.864506  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.916123  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.822001  7818 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:58.846341  8053 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:25:58.877902  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:58.822338  7542 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:58.884490  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 50438.8us
I20260709 12:25:58.884953  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2567) took 1769 ms. Trace:
I20260709 12:25:58.832564  7886 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:58.832903  7820 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:25:58.822549  7846 raft_consensus.cc:697] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:58.885598  4673 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:25:58.892270  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.894739  4755 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:58.848771  7779 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:58.848421  7981 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.918198  4751 raft_consensus.cc:2410] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:58.815109  8037 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:58.848611  8011 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:25:58.918999  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.928460  4677 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:58.955359  4748 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:58.849255  7771 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.956218  4679 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.957183  4747 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
W20260709 12:25:58.959077  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.917268  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.997471  4753 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:58.960404  4673 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:58.943565  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:58.999344  4749 raft_consensus.cc:2393] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 12: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:25:59.001261  8040 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.001924  8040 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.002393  8040 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.004426  8040 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.005990  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:58.962272  8060 log.cc:927] Time spent T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:25:58.961740  7818 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.931463  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:25:58.963280  8061 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:25:58.966353  8059 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:25:58.966655  8072 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.969951  8070 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:58.974241  8039 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:25:58.974937  4600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1674) took 2958 ms (client timeout 1507 ms). Trace:
I20260709 12:25:58.975069  7885 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.980901  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:59.067468  4601 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
W20260709 12:25:59.067534  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 123152us
I20260709 12:25:59.068820  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:58.981739  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:58.982484  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:59.070218  4596 raft_consensus.cc:2376] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10. Re-sending same reply.
I20260709 12:25:59.070502  4604 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 12: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:25:58.984334  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:59.071645  4597 raft_consensus.cc:2410] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:58.881249  7998 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.991235  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:59.014178  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:25:59.015944  8058 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.464s	user 0.002s	sys 0.000s
I20260709 12:25:59.016851  7771 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:59.021495  7981 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.022912  8037 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.025391  7846 consensus_queue.cc:237] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:58.896574  8049 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.055644  4754 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:59.087637  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.088124  4754 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:59.059415  4679 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:25:59.062943  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:59.034614  4675 raft_consensus.cc:2955] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 2 to 4, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:25:59.065877  4600 rpcz_store.cc:269] 0709 12:25:56.016489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.036446 (+ 19957us) service_pool.cc:224] Handling call
0709 12:25:58.974766 (+2938320us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b540004f454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:58.974871 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4950121344}
I20260709 12:25:59.069746  8074 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:58.914716  7863 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.072757  7998 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.065232  7818 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:58.988771  7542 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:59.073856  7849 consensus_meta.cc:287] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.577s	user 0.011s	sys 0.000s
I20260709 12:25:59.090893  8049 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.106918  7849 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.092837  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:59.066988  8072 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.104385  4754 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 4.
I20260709 12:25:59.111802  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.112361  4754 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:59.115597  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.116134  4755 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:59.095335  7779 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:59.117044  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 1.38157e+06us
I20260709 12:25:59.118708  7779 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.119005  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:59.120191  8072 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:59.120813  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:59.075675  7885 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:25:59.111738  8012 consensus_meta.cc:287] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 12:25:59.122833  4754 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:25:59.124043  4600 pending_rounds.cc:85] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 4
I20260709 12:25:59.123965  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.109112  8062 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:59.109138  7849 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.125409  4600 pending_rounds.cc:107] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:25:59.128057  4600 raft_consensus.cc:2889] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:59.127153  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.126276  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:59.126669  5347 raft_consensus.cc:606] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:59.130637  5347 raft_consensus.cc:614] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:59.130697  4754 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:59.131721  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:59.120806  8072 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:59.132453  4754 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:25:59.135177  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.123160  7779 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.124782  7771 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:59.127497  8070 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:59.138727  8081 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:25:59.136907  7998 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:59.157979  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:59.147650  8072 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:59.096109  8074 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.162716  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 500) took 3132 ms. Trace:
I20260709 12:25:59.163084  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:59.163106  4599 rpcz_store.cc:276] 0709 12:25:56.030426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.789355 (+1758929us) service_pool.cc:224] Handling call
0709 12:25:58.329281 (+539926us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.341062 (+ 11781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.162528 (+821466us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b5400352ad0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:25:59.162658 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1240630016}
I20260709 12:25:59.092315  7886 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.164672  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:59.165351  4754 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:25:59.166538  4755 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:59.166754  7863 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:59.167062  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:59.167511  8074 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:59.167609  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.168239  7779 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:59.168385  4755 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.168732  7779 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:59.169162  7779 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.175567  4755 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:59.176811  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.176347  7779 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.178541  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:59.179170  4747 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:59.179560  7779 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.179968  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:25:59.181151  8051 consensus_meta.cc:287] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.503s	user 0.003s	sys 0.004s
I20260709 12:25:59.160100  7998 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:59.181998  7998 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:59.182973  7998 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.185249  4747 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:25:59.185911  8081 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:59.186579  8081 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:59.187007  7885 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:59.157285  8049 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.187701  8081 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.188570  4676 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:25:59.189651  8049 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:59.192030  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:59.192831  8070 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:59.193332  8070 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:59.194257  8070 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.196441  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:59.197093  4753 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:59.197186  5347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:25:59.197862  5347 raft_consensus.cc:606] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:59.198050  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:59.198695  4598 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:25:59.198890  7863 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:59.199287  7863 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:59.199642  7863 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:59.200968  5347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:25:59.201685  5347 raft_consensus.cc:606] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:25:59.204383  5348 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:25:59.204968  5348 raft_consensus.cc:606] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:25:59.205291  5348 raft_consensus.cc:614] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:59.206525  8083 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:59.206768  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:59.207260  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:59.207441  4753 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:25:59.207899  4748 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:25:59.208323  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.208727  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.208807  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.209405  4748 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:25:59.180805  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:59.216777  5348 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:25:59.221473  4755 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:25:59.221588  5348 raft_consensus.cc:606] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:25:58.957731  4756 rpcz_store.cc:276] 0709 12:25:57.115938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.193534 (+ 77596us) service_pool.cc:224] Handling call
0709 12:25:57.212577 (+ 19043us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.212840 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:57.514465 (+301625us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.883242 (+1368777us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.884606 (+  1364us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.884651 (+    45us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:58.884703 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.884763 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.884890 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":858009,"prepare.queue_time_us":332843,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":931106,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":29688,"thread_start_us":772055,"threads_started":3,"wal-append.queue_time_us":421845}
I20260709 12:25:59.221694  4571 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:59.210230  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.222957  4749 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.223553  4571 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:59.222074  5348 raft_consensus.cc:614] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:59.211045  5314 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:59.209307  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:59.226254  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.232823  4747 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:25:59.234356  4603 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:59.237917  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.249140  4571 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.249344  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:59.240278  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:59.240798  5312 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:59.177852  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:59.250377  4598 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:25:59.242491  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:59.250667  4754 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:25:59.250672  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:59.251436  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.251881  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:59.253173  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:59.253772  5348 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:25:59.251447  4598 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:59.254716  4719 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:59.265399  4749 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:25:59.267686  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:59.268033  4602 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:25:59.268806  5347 raft_consensus.cc:2410] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:25:59.269527  5314 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:59.269999  5312 leader_election.cc:400] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10.
I20260709 12:25:59.270351  5312 leader_election.cc:403] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:25:59.267926  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.271093  4571 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:25:59.275405  5343 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:59.211984  8083 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:59.276561  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.277145  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:59.217672  8051 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.280231  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:59.278169  7846 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:59.281286  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.281875  4755 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.282079  8051 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:59.285702  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:59.286751  5341 raft_consensus.cc:2393] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 7: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:25:59.270140  7771 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:59.287799  7771 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.288172  7771 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:59.272064  7849 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:59.292409  7849 raft_consensus.cc:697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:59.293450  7849 consensus_queue.cc:237] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:58.770061  4750 rpcz_store.cc:276] 0709 12:25:57.402586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.781228 (+378642us) service_pool.cc:224] Handling call
0709 12:25:57.781625 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.781882 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:57.790825 (+  8943us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.768010 (+977185us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.768058 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:58.768107 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.768167 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.768310 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":830910,"prepare.queue_time_us":10495,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":832656,"thread_start_us":145735,"threads_started":1,"wal-append.queue_time_us":158336}
I20260709 12:25:59.295866  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.248406  7885 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:59.246122  7787 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:59.281409  7846 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.302721  4756 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.269999  8038 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:59.214799  7807 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.277526  7832 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:59.307300  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:59.278002  7818 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:59.296715  8039 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:59.304999  7885 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.307096  7787 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.308424  8083 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:25:59.321803  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 67228.7us
I20260709 12:25:59.317219  4755 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:25:59.307684  7835 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:59.307935  7846 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.308821  8070 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:59.332204  4601 raft_consensus.cc:2368] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10.
I20260709 12:25:59.318429  7832 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:25:59.355517  4748 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
W20260709 12:25:59.335678  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.323948  7796 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10.
I20260709 12:25:59.321456  8077 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:59.364009  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 94578.1us
I20260709 12:25:59.352137  7835 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.371658  7835 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.372678  7771 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.362178  8083 raft_consensus.cc:697] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:59.374344  8083 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.376322  7542 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:59.377154  7945 raft_consensus.cc:3055] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:25:59.381438  4498 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 163) took 16083 ms (client timeout 15000 ms). Trace:
I20260709 12:25:59.360239  8035 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.321591  7818 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:59.331902  7885 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:59.390202  4500 catalog_manager.cc:5697] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 3 to 4, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
W20260709 12:25:59.392072  8064 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.477s	user 0.002s	sys 0.000s
I20260709 12:25:59.392000  4500 catalog_manager.cc:5697] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 3 to 5, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 5 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:59.365760  8077 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.397929  8077 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.399158  4500 catalog_manager.cc:5697] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 1 to 4, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 4 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:25:59.400676  4500 catalog_manager.cc:5697] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 6 to 8, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 8 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:25:59.402189  7863 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.402879  7846 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.307904  8051 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:59.365669  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.408264  4748 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:59.359933  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.325297  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 73925us
I20260709 12:25:59.358253  7787 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.437459  7820 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.438462  8049 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.438638  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:59.439026  8049 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.439564  7983 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:25:59.440090  7983 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.359305  8070 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.441776  8070 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:59.441888  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:59.442582  4676 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:59.442068  4500 catalog_manager.cc:5697] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 4 to 6, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 6 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:59.444043  4500 catalog_manager.cc:5697] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 3 to 5, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 5 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
W20260709 12:25:59.446801  8087 log.cc:927] Time spent T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:25:59.370914  7807 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:59.387499  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:59.466583  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:59.388011  4755 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:25:59.362601  7832 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:59.331609  4750 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:59.415417  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:59.406093  7818 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:59.484242  4679 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 12:25:59.429173  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 566531us
I20260709 12:25:59.430219  7776 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:59.485911  7776 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:59.486477  7776 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:59.433808  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 455803us
W20260709 12:25:59.491699  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.492192  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.433934  8052 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:59.435433  7771 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:25:59.436718  7863 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:25:59.448576  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.449172  5314 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:59.449843  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:59.461156  4756 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:25:59.445480  4500 catalog_manager.cc:5697] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:25:59.497160  4678 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:59.465368  4676 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:25:59.441283  8049 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.479400  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:59.480759  4674 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:25:59.482796  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 67440.4us
I20260709 12:25:59.427160  7986 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:59.498607  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:59.498193  4500 catalog_manager.cc:5697] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 4 to 6, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 6 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:25:59.510172  4600 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.511025  4500 catalog_manager.cc:5697] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:59.514279  4675 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:59.383536  7945 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:59.509799  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:59.520287  4645 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.510164  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:25:59.524106  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 75015.9us
W20260709 12:25:59.384389  4498 rpcz_store.cc:269] 0709 12:25:43.407474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.435724 (+ 28250us) service_pool.cc:224] Handling call
0709 12:25:59.381303 (+15945579us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:52.006221 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:52.243296 (+237075us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:52.268381 (+ 25085us) write_op.cc:435] Acquired schema lock
0709 12:25:52.288534 (+ 20153us) tablet.cc:662] Decoding operations
0709 12:25:52.290285 (+  1751us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:52.290376 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:52.290405 (+    29us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:25:52.379083 (+ 88678us) tablet.cc:701] Row locks acquired
0709 12:25:52.399890 (+ 20807us) write_op.cc:260] PREPARE: finished
0709 12:25:52.400057 (+   167us) write_op.cc:270] Start()
0709 12:25:52.400230 (+   173us) write_op.cc:276] Timestamp: P: 1783599952399988 usec, L: 0
0709 12:25:52.400254 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:25:52.481295 (+ 81041us) log.cc:844] Serialized 6472 byte log entry
0709 12:25:52.942495 (+461200us) op_driver.cc:464] REPLICATION: finished
0709 12:25:53.085757 (+143262us) write_op.cc:301] APPLY: starting
0709 12:25:53.088368 (+  2611us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:53.088729 (+   361us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:53.088871 (+   142us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:53.090770 (+  1899us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:53.091113 (+   343us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 12:25:53.091160 (+    47us) write_op.cc:312] APPLY: finished
0709 12:25:53.091585 (+   425us) log.cc:844] Serialized 107 byte log entry
0709 12:25:53.159135 (+ 67550us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:53.182657 (+ 23522us) write_op.cc:454] Released schema lock
0709 12:25:53.193046 (+ 10389us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:53.273770 (+ 80724us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:53.276901 (+  3131us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1258423,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":139462,"apply.run_cpu_time_us":4753,"apply.run_wall_time_us":151333,"num_ops":15,"prepare.queue_time_us":231589,"prepare.run_cpu_time_us":43909,"prepare.run_wall_time_us":596308,"raft.queue_time_us":162889,"raft.run_cpu_time_us":16943,"raft.run_wall_time_us":190460,"replication_time_us":501694,"thread_start_us":249843,"threads_started":4,"wal-append.queue_time_us":79235,"wal-append.run_cpu_time_us":28198,"wal-append.run_wall_time_us":1543226}]]}
I20260709 12:25:59.524736  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.524756  5314 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:59.524809  4672 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:59.525936  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:59.526261  4678 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:25:59.529207  4600 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:25:59.500435  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:59.483911  8051 raft_consensus.cc:697] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:25:59.502727  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.529821  4748 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:59.499737  4596 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:59.540411  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.530452  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:59.532286  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:25:59.541332  4748 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:59.541277  4603 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:25:59.542404  4675 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:59.515300  7820 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:59.534096  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 264209us
I20260709 12:25:59.537431  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:25:59.549140  4674 raft_consensus.cc:2393] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 7: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:25:59.549338  4750 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:59.537515  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:59.405979  8035 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.538476  4604 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:25:59.501155  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:25:59.550606  4602 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:25:59.532335  8049 raft_consensus.cc:2764] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:25:59.483263  7542 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:59.534772  8091 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:59.533357  8089 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:59.533787  7771 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:59.570822  4675 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:25:59.551656  4678 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:59.552613  8053 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 12:25:59.554735  8037 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:59.556110  8068 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.465s	user 0.000s	sys 0.002s
I20260709 12:25:59.557423  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:59.535876  7796 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:59.534437  8052 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:59.564419  4604 raft_consensus.cc:1240] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:59.570402  4750 raft_consensus.cc:1240] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:59.539515  8081 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:59.552762  8019 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:25:59.548372  8012 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:59.579901  4647 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:25:59.590646  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 70813.8us
I20260709 12:25:59.590317  4678 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:25:59.592259  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.567878  7983 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.593380  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.595904  4750 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 12:25:59.599889  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:25:59.575523  8091 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:59.600800  4647 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.599514  8002 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:59.607597  4747 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:25:59.608650  7983 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:59.600243  8012 raft_consensus.cc:697] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:59.581082  7796 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.538542  8077 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.625490  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:59.609341  7818 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.631434  8062 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:59.631855  7787 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.635742  7846 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.636341  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:25:59.592644  8049 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:59.638764  4751 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:59.584666  7971 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:59.653739  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:25:59.669355  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 43986.3us
I20260709 12:25:59.673050  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:59.673192  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:25:59.674081  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:59.677305  4678 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:25:59.650278  7983 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:59.574522  7945 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.609314  7776 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.600219  7913 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:25:59.708133  7776 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.598176  7782 raft_consensus.cc:2764] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:25:59.690312  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.691135  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.692276  5343 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.669265  7796 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:59.719058  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:59.599866  8003 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:25:59.596977  7885 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.647337  8002 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:25:59.725589  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 71993.2us
I20260709 12:25:59.696408  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:59.696475  4602 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:59.697803  4605 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:25:59.702142  8096 log.cc:927] Time spent T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:25:59.702482  8101 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:25:59.677362  8052 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:59.703629  7818 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.728793  8052 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:59.677915  8062 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:59.704707  8092 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:25:59.729285  8052 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:59.705644  8098 log.cc:927] Time spent T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:25:59.729537  8062 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:59.706653  8088 log.cc:927] Time spent T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.386s	user 0.004s	sys 0.000s
I20260709 12:25:59.729274  7818 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:25:59.708633  8035 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:59.659601  8011 consensus_queue.cc:1059] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:25:59.655208  7832 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.694244  7846 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.696844  7983 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.721032  5348 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:59.597909  8051 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.723941  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:25:59.675216  8097 consensus_queue.cc:1059] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
W20260709 12:25:59.741536  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 43961.2us
I20260709 12:25:59.714394  7945 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.747058  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.733820  8002 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:59.765239  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:25:59.766183  4750 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:59.748905  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 203715us
W20260709 12:25:59.749557  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 119430us
I20260709 12:25:59.768150  4569 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:25:59.767539  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:59.749738  4571 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.749701  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:25:59.720885  7779 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:59.696097  8053 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:25:59.752286  4748 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:25:59.755012  4647 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.724783  7782 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:59.724627  7971 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:59.721856  7981 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.763144  8019 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:59.759312  7832 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.732594  7776 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:25:59.722854  8091 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.762940  4749 raft_consensus.cc:2368] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7.
I20260709 12:25:59.745988  8114 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:25:59.771049  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:25:59.774585  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.803296  4750 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:25:59.777426  4647 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:25:59.713165  8106 consensus_queue.cc:1175] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3a5aeedd11fe49938079fe86cb01232c" responder_term: 7 status { last_received { term: 6 index: 7 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:59.776137  7771 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:59.786063  8107 raft_consensus.cc:3055] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:59.784202  8011 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:25:59.784281  7779 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:25:59.783445  8019 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:59.753814  7846 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.791927  8077 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.792092  5343 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:25:59.782078  7913 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:59.784907  7981 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.805306  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:59.854980  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49806.1us
I20260709 12:25:59.856710  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:25:59.725839  8012 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.805821  8110 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:59.822069  5314 leader_election.cc:400] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7.
I20260709 12:25:59.867390  5314 leader_election.cc:403] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:25:59.817849  7776 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:59.790298  7787 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:25:59.845232  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 77173.2us
W20260709 12:25:59.848642  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.851965  5345 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:59.858490  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:25:59.863781  8090 raft_consensus.cc:2764] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:59.864135  7885 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:25:59.843823  7771 raft_consensus.cc:697] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:59.864650  7972 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:59.843256  5347 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:25:59.868862  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 106047us
I20260709 12:25:59.817881  8107 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:25:59.869664  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 51126.5us
I20260709 12:25:59.785317  7835 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:25:59.876044  5348 raft_consensus.cc:2955] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Committing config change with OpId 7.7: config changed from index 3 to 7, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:25:59.877939  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.864212  8081 raft_consensus.cc:2764] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:59.852099  8019 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.918013  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:25:59.861970  8035 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:59.923671  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 45859.8us
W20260709 12:25:59.935343  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 61865.2us
I20260709 12:25:59.935580  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:25:59.936121  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:25:59.870570  7776 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:59.872884  7832 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7.
I20260709 12:25:59.936916  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:25:59.914482  7971 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.936722  4597 raft_consensus.cc:2368] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7.
I20260709 12:25:59.947894  4645 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:25:59.869427  7846 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:25:59.952770  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:25:59.949680  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 200027us
I20260709 12:25:59.869632  7968 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:25:59.879215  8110 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:25:59.938313  7542 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:59.930909  8003 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:59.958624  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 100238us
I20260709 12:25:59.944429  7776 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.945452  7972 raft_consensus.cc:697] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:59.955178  8053 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:59.956979  7782 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:59.986276  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:25:59.986735  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 67897us
I20260709 12:25:59.944998  8051 consensus_queue.cc:1059] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:25:59.947171  8091 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:25:59.985607  5347 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:59.999742  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 82797.9us
I20260709 12:25:59.954845  8012 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:25:59.966958  8109 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:26:00.033140  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.034277  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48084.7us
I20260709 12:25:59.964404  7537 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:25:59.964959  7968 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:25:59.950922  7779 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:00.058668  8116 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:00.042876  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 509) took 1055 ms. Trace:
I20260709 12:26:00.043035  4599 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:59.981236  8035 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:25:59.958849  7971 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:25:59.998872  8070 consensus_meta.cc:287] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.554s	user 0.013s	sys 0.001s
I20260709 12:26:00.045140  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:00.045528  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 92868.7us
W20260709 12:26:00.042009  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 59586.5us
I20260709 12:26:00.138605  4569 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:26:00.052989  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 66607.9us
I20260709 12:25:59.999642  8113 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.146565  4597 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:00.148154  4602 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:00.150019  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:00.151929  8129 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.152392  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:00.091568  8121 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:26:00.152695  8129 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.111390  8110 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:00.089808  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 127715us
W20260709 12:26:00.154045  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 108989us
I20260709 12:26:00.066788  4601 rpcz_store.cc:276] 0709 12:25:58.987266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.079929 (+ 92663us) service_pool.cc:224] Handling call
0709 12:26:00.000669 (+920740us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b5400352fd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:00.042793 (+ 42124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":494004608}
I20260709 12:26:00.120090  7796 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:00.118072  8090 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.155546  8090 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.045006  8122 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.156417  7796 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:00.156628  8091 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:26:00.122339  8126 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:26:00.094542  7846 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:26:00.052537  8077 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:26:00.145228  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:00.118468  8081 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:00.153355  4603 raft_consensus.cc:2393] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:00.156203  5312 leader_election.cc:400] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7.
I20260709 12:26:00.168269  4600 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:00.157595  8131 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.158039  8090 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:00.091941  7981 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:00.062212  8003 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:00.060385  7820 consensus_queue.cc:1059] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:26:00.112136  7832 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:00.150339  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:00.164014  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:00.033536  7771 consensus_queue.cc:237] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.165081  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:00.088986  8107 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:00.171924  8129 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:00.206945  8097 consensus_queue.cc:1059] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:26:00.209647  8097 consensus_queue.cc:1243] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:00.209812  8081 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.178253  8132 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:00.137833  7972 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:00.214198  8132 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.216464  8132 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:00.172046  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:00.206720  5312 leader_election.cc:400] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10.
I20260709 12:26:00.174130  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:00.219398  4596 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 4.
I20260709 12:26:00.222033  4678 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:26:00.165942  4569 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:00.223433  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 57585.2us
I20260709 12:26:00.224382  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:00.225023  4756 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:26:00.226539  4569 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:00.228157  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:00.228835  4605 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:00.234758  4604 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:00.241127  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:00.038178  7885 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:00.159876  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:00.118283  8109 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:00.254891  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:00.243587  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:00.261051  4756 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:00.262604  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.263025  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:00.264745  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.265473  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:00.265719  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:00.266510  4747 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:00.268770  4756 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:00.270599  4647 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:00.273757  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 113978us
I20260709 12:26:00.274587  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:00.275754  4754 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:00.272346  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:00.277812  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:00.199005  8108 raft_consensus.cc:3055] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:00.278438  4601 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.212757  8081 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:00.172326  8120 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:00.190162  7835 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:00.161623  8070 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:00.285974  8018 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:00.190660  8035 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.287117  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:00.286993  8119 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:00.286685  7849 raft_consensus.cc:2955] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Committing config change with OpId 7.7: config changed from index 3 to 7, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:00.231843  7846 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:00.178510  7971 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:00.176061  7968 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:00.244693  8038 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:00.292698  8133 consensus_queue.cc:1059] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:00.166672  8113 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.315456  4747 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
W20260709 12:26:00.330904  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.351229  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2582) took 1435 ms. Trace:
I20260709 12:26:00.355439  4754 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:00.156672  8122 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:00.203439  8002 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:00.291610  7981 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:00.398705  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 531) took 1387 ms. Trace:
I20260709 12:26:00.257151  8114 consensus_queue.cc:1059] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 12:26:00.245645  8132 consensus_queue.cc:1059] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 12:26:00.291113  8108 raft_consensus.cc:740] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:00.272648  7782 raft_consensus.cc:697] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:00.406448  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:00.259471  8077 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.340829  7820 raft_consensus.cc:3055] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:00.267995  7832 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:00.349004  8120 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:00.309975  8003 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.104s
I20260709 12:26:00.173195  8131 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:00.415866  4596 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:26:00.516260  4602 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:26:00.516197  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 289746us
I20260709 12:26:00.507503  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:00.516907  4751 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:00.515754  4755 rpcz_store.cc:276] 0709 12:25:58.916199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.353023 (+436824us) service_pool.cc:224] Handling call
0709 12:25:59.353462 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.478186 (+124724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.350855 (+872669us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b5400309154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:00.351149 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76772736}
I20260709 12:26:00.531294  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:00.532401  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:00.517290  4748 rpcz_store.cc:276] 0709 12:25:59.011646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.566080 (+554434us) service_pool.cc:224] Handling call
0709 12:25:59.582573 (+ 16493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.774421 (+191848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.398432 (+624011us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b540030a054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:00.398645 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389555456}
I20260709 12:26:00.515216  4601 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:00.526149  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:00.527571  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:00.535398  4747 raft_consensus.cc:2410] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:00.535889  4754 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:26:00.537645  4754 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:26:00.540755  5312 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:00.413643  8062 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.683s	user 0.008s	sys 0.004s
I20260709 12:26:00.542701  4747 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:00.548887  4601 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:26:00.549989  4747 raft_consensus.cc:3055] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:00.550482  4747 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:00.406133  8018 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:00.552290  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:00.542932  4604 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:00.551460  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:00.551442  4747 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.558161  4747 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:00.558499  4748 raft_consensus.cc:1240] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:00.554878  4672 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:00.462067  8038 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:00.516749  7820 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:00.556509  4678 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: [6.8-9.9]
I20260709 12:26:00.462818  7968 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:00.561597  4748 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:00.577231  7968 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:00.577957  4748 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:00.578147  7968 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.580080  4748 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:00.581832  4748 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:00.582142  4605 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
W20260709 12:26:00.562474  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 423900us
I20260709 12:26:00.563683  4749 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:00.582744  4747 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:26:00.583091  4747 pending_rounds.cc:85] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 4
I20260709 12:26:00.547463  7846 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.550496  8037 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.528719  7975 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:00.584784  4747 pending_rounds.cc:107] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:26:00.611879  4747 raft_consensus.cc:2889] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:00.527863  8077 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.569876  7779 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:00.611958  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:00.604965  5312 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:00.611459  4748 raft_consensus.cc:2955] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Committing config change with OpId 7.7: config changed from index 3 to 7, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:26:00.568187  8052 consensus_meta.cc:287] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.838s	user 0.013s	sys 0.000s
I20260709 12:26:00.615442  4605 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:00.618643  4747 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:00.609217  4751 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
W20260709 12:26:00.652791  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 47940us
I20260709 12:26:00.604477  7734 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.556903  8062 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:00.615852  7796 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.665722  4747 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:26:00.671396  4644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.021s	sys 0.058s
I20260709 12:26:00.616056  8117 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:26:00.657478  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:00.710943  4644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.021s	sys 0.059s
W20260709 12:26:00.711464  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 557170us
I20260709 12:26:00.712951  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:00.613255  7846 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.803984  4675 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:26:00.718633  7832 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:00.617142  8037 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.660199  8106 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:26:00.814047  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 79597us
I20260709 12:26:00.664067  7779 raft_consensus.cc:697] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:00.545754  8109 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:00.806056  7983 consensus_meta.cc:287] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.043s	user 0.010s	sys 0.005s
I20260709 12:26:00.706211  7734 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:00.795388  8093 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.715819  8003 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.125s
I20260709 12:26:00.854907  5314 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:00.864300  4602 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:00.637506  7975 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.656314  8130 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.544s
I20260709 12:26:00.719324  7796 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:00.673786  7820 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:00.656725  8052 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.710706  8018 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:00.662267  8120 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.816937  8038 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.673274  7782 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
W20260709 12:26:00.946058  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 91307.7us
I20260709 12:26:01.061949  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:00.882999  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:01.111429  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.073840  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:01.112859  4750 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 12: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:00.881366  4601 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:26:01.030678  5343 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 531) took 1358 ms (client timeout 1502 ms). Trace:
I20260709 12:26:01.114733  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.107491  8119 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.115453  8077 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.115257  8003 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:01.117789  8119 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:00.879783  8093 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.118289  8003 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.061296  8113 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.120450  8119 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:01.114876  5343 rpcz_store.cc:269] 0709 12:25:59.672569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.672715 (+   146us) service_pool.cc:224] Handling call
0709 12:26:01.030428 (+1357713us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b54006fca54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:01.030594 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11089664}
I20260709 12:26:00.862680  8106 raft_consensus.cc:697] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:01.108939  7981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.003s	sys 0.021s
I20260709 12:26:00.918960  8002 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:01.143293  8146 log.cc:927] Time spent T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:26:00.861130  7983 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:01.120945  8138 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 12:26:01.068607  8124 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.120457  8003 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.038223  8049 raft_consensus.cc:3055] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:00.955093  8011 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:01.040978  7945 raft_consensus.cc:3055] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:01.017616  8105 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:01.058105  8122 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:01.124488  8135 log.cc:927] Time spent T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.286s	user 0.000s	sys 0.000s
I20260709 12:26:00.925369  8012 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:01.120435  8093 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:01.145807  7981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.004s	sys 0.021s
I20260709 12:26:00.946455  7968 consensus_queue.cc:1059] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.210s
I20260709 12:26:01.122071  8139 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:01.144695  8131 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:01.157188  7771 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:01.395967  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 509) took 1175 ms. Trace:
I20260709 12:26:01.396690  7771 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.397506  7771 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:01.397815  5311 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.003s	sys 0.008s
W20260709 12:26:01.398348  5311 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.003s	sys 0.008s
W20260709 12:26:01.231195  5345 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.021s	sys 0.013s
W20260709 12:26:01.400490  5345 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.021s	sys 0.013s
I20260709 12:26:01.401891  5345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53808 (request call id 490) took 1106 ms. Trace:
I20260709 12:26:01.402032  7542 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.402316  5345 rpcz_store.cc:276] 0709 12:26:00.295134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.295342 (+   208us) service_pool.cc:224] Handling call
0709 12:26:00.295761 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.296038 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:00.703086 (+407048us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.401059 (+697973us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.401178 (+   119us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:01.401330 (+   152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.401447 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.401716 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59250,"prepare.queue_time_us":980025,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":68,"raft.queue_time_us":133938,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":19727,"thread_start_us":1092801,"threads_started":2}
I20260709 12:26:01.403167  7542 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:01.390980  4644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.002s	sys 0.017s
W20260709 12:26:01.405838  4644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.002s	sys 0.018s
W20260709 12:26:01.391681  4645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.023s	sys 0.037s
W20260709 12:26:01.144168  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 261236us
I20260709 12:26:01.393972  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2112) took 1169 ms. Trace:
I20260709 12:26:01.410281  4755 rpcz_store.cc:276] 0709 12:26:00.224603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.533231 (+308628us) service_pool.cc:224] Handling call
0709 12:26:00.533553 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.533975 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.534002 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.547998 (+ 13996us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.041471 (+493473us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:01.041510 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.042937 (+  1427us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.393256 (+350319us) raft_consensus.cc:1685] finished
0709 12:26:01.393294 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.393725 (+   431us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3396,"raft.queue_time_us":82,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":7219,"thread_start_us":282925,"threads_started":1,"wal-append.queue_time_us":733063}
I20260709 12:26:01.411777  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:01.412431  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.413110  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.413774  4681 raft_consensus.cc:3055] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:01.414212  4681 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:01.414354  7734 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:01.185488  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.225805  5348 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.061s	user 0.021s	sys 0.046s
I20260709 12:26:01.248514  7846 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:01.424096  5348 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.259s	user 0.024s	sys 0.046s
I20260709 12:26:01.390460  4605 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:26:01.425611  5348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 537) took 1551 ms. Trace:
I20260709 12:26:01.426081  5348 rpcz_store.cc:276] 0709 12:25:59.874286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.874796 (+   510us) service_pool.cc:224] Handling call
0709 12:25:59.875729 (+   933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.875931 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:00.164457 (+288526us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.424971 (+1260514us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.425097 (+   126us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:01.425204 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.425326 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.425539 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1259795,"threads_started":1,"wal-append.queue_time_us":1149101}
W20260709 12:26:01.409677  4645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.023s	sys 0.037s
W20260709 12:26:01.429268  7849 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.851s	user 0.006s	sys 0.008s
W20260709 12:26:01.429664  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 817635us
I20260709 12:26:01.430236  5341 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 12:26:01.168557  7820 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:01.443656  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.444703  4753 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:01.217257  7945 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:01.255909  7968 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.237032  7981 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:01.415103  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.493214  8155 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:01.493966  4750 raft_consensus.cc:2410] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:01.421365  7863 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:01.422390  7975 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.422616  7983 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.408041  7796 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.394652  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 542) took 1359 ms. Trace:
I20260709 12:26:01.036749  7779 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.438023  7846 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:01.440542  7776 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.401s	user 0.006s	sys 0.005s
I20260709 12:26:01.442327  4676 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:01.308430  7913 consensus_meta.cc:287] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.439s	user 0.006s	sys 0.008s
W20260709 12:26:01.337543  8127 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.044s	user 0.010s	sys 0.016s
I20260709 12:26:01.233309  8011 raft_consensus.cc:697] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:01.494747  8157 consensus_queue.cc:1059] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:26:01.177752  8002 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.523959  8002 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.524010  7945 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:01.525346  8053 consensus_meta.cc:287] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.526s	user 0.006s	sys 0.008s
I20260709 12:26:01.526043  8053 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:01.527171  8002 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:01.527433  4750 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:01.223416  8037 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.214174  8049 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:01.209022  8062 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:01.214944  8124 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.233016  8105 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.252192  8131 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:01.246603  8122 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.556404  8105 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.255295  8139 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:01.431665  5341 pending_rounds.cc:85] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 7
I20260709 12:26:01.441339  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:01.462560  4646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.000s	sys 0.007s
I20260709 12:26:01.445806  8093 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:01.469465  8083 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:01.470192  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 57986.7us
W20260709 12:26:01.471271  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.474341  8173 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:01.475005  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63695.3us
I20260709 12:26:01.478412  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.478626  7968 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.479594  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:01.396651  4674 rpcz_store.cc:276] 0709 12:26:00.219872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.220037 (+   165us) service_pool.cc:224] Handling call
0709 12:26:00.220361 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.228698 (+  8337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.228728 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.273261 (+ 44533us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.674180 (+400919us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:00.674220 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.674300 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.395104 (+720804us) raft_consensus.cc:1685] finished
0709 12:26:01.395145 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.395592 (+   447us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7975,"raft.queue_time_us":47567,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":18416,"thread_start_us":308526,"threads_started":1,"wal-append.queue_time_us":669977}
I20260709 12:26:01.284404  8106 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.500253  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.507843  7818 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:26:01.508317  8052 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:01.508723  8127 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.238s	user 0.010s	sys 0.016s
I20260709 12:26:01.507964  7913 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:01.507879  7846 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.509048  7863 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.509783  4679 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:01.513886  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2116) took 1288 ms. Trace:
I20260709 12:26:01.513660  8153 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:01.494433  7981 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.497135  4752 rpcz_store.cc:276] 0709 12:26:00.035457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.035618 (+   161us) service_pool.cc:224] Handling call
0709 12:26:00.036055 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.036497 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.036539 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.037083 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.308983 (+271900us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:00.309038 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.309135 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.394138 (+1085003us) raft_consensus.cc:1685] finished
0709 12:26:01.394170 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.394467 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":896,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":142,"thread_start_us":216909,"threads_started":1,"wal-append.queue_time_us":606164}
I20260709 12:26:01.500358  4681 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.519325  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 507) took 1841 ms. Trace:
I20260709 12:26:01.504733  8140 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.508391  4645 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:01.514643  8160 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:01.508594  8171 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:01.539283  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 510) took 2527 ms. Trace:
I20260709 12:26:01.545670  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:01.522257  7776 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.547534  4596 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:01.549409  4571 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.594s (SENT) [suppressed 29 similar messages]
I20260709 12:26:01.550153  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:01.563457  4751 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:26:01.563781  8019 consensus_meta.cc:287] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.615s	user 0.012s	sys 0.005s
I20260709 12:26:01.566185  8122 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.617077  4599 rpcz_store.cc:276] 0709 12:25:58.988391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.164519 (+176128us) service_pool.cc:224] Handling call
0709 12:26:00.042775 (+878256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.128248 (+ 85473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.514733 (+1386485us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b5400352fd0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:01.515000 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2019173248}
I20260709 12:26:01.537482  7945 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:01.574596  4646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.000s	sys 0.008s
I20260709 12:26:01.552861  8049 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.564462  8130 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:01.588093  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.572908  5341 pending_rounds.cc:107] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 12:26:01.589509  8093 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:01.591524  8169 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:01.593895  8105 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.595079  8173 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.612608  4747 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:26:01.605723  4754 rpcz_store.cc:276] 0709 12:26:00.225532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.538818 (+313286us) service_pool.cc:224] Handling call
0709 12:26:00.539175 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.554309 (+ 15134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.554357 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.557935 (+  3578us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.101846 (+543911us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:01.101905 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.102003 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.513223 (+411220us) raft_consensus.cc:1685] finished
0709 12:26:01.513261 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.513650 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":282,"raft.queue_time_us":11264,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":3727,"spinlock_wait_cycles":114560,"thread_start_us":486066,"threads_started":1,"wal-append.queue_time_us":745774}
I20260709 12:26:01.246083  8012 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.605990  7542 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:01.620664  4753 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:01.624470  4681 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:01.609995  4672 rpcz_store.cc:276] 0709 12:25:59.677381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.678196 (+   815us) service_pool.cc:224] Handling call
0709 12:25:59.678582 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.561273 (+882691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.518835 (+957562us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b54001d1ad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:01.519136 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2014870912}
I20260709 12:26:01.626400  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:01.635456  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 127239us
I20260709 12:26:01.635828  8052 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:01.636301  8052 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.636827  4645 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:01.637310  8171 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.637995  8173 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.640460  8049 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:01.640933  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.641294  4678 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:01.642230  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.642556  8180 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.643340  4752 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:26:01.643466  8180 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.644809  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.635156  8105 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.648326  4752 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:01.649538  8093 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.651082  7846 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.652597  4755 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:01.652778  4750 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:01.656746  7983 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:01.657217  7983 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:01.657730  7983 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:01.657881  8153 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.658576  8153 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.660956  8153 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.662487  7863 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:01.663406  7863 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:01.664700  7863 raft_consensus.cc:3055] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:01.664935  7782 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.665800  7782 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.668773  8157 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:01.670258  8157 raft_consensus.cc:3055] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:01.670691  8157 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:01.670542  8011 consensus_queue.cc:237] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.671666  8117 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.671989  8157 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.672369  8117 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.626005  8130 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:01.627560  5341 raft_consensus.cc:2889] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:01.676409  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.652031  8093 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:01.682101  8166 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:26:01.682330  8124 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.668286  7782 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.682385  8131 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.683728  4680 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:01.606935  8038 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 12:26:01.688470  7776 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:01.689179  7968 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:01.690696  4754 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:01.706648  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.656574  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.696588  4673 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:01.636804  8052 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:01.679275  8140 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.654700  7542 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 12:26:01.699720  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63184us
I20260709 12:26:01.665323  7863 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:01.679153  8157 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:01.640874  8171 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.710927  4751 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:01.615756  8083 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:26:01.675644  8122 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:01.619889  8160 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.631469  8019 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.665122  8003 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.744031  4755 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:01.750512  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.752415  8012 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.675060  8130 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.754580  8117 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.758320  5314 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:01.758682  8131 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.734323  8180 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:01.762043  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 133444us
I20260709 12:26:01.757265  8038 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:01.767105  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 217520us
W20260709 12:26:01.771152  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 75599.7us
I20260709 12:26:01.775180  4753 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:26:01.777732  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.690768  8139 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.785140  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2621) took 1477 ms. Trace:
I20260709 12:26:01.798380  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:01.770874  7968 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:01.804256  8132 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.805003  8132 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.805838  8145 raft_consensus.cc:3055] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:01.806481  8145 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:01.808337  4599 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:01.809984  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:01.810235  8130 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:01.810847  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:01.812680  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:01.813413  8140 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.814247  8003 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:01.816769  8182 log.cc:927] Time spent T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:26:01.818377  4674 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:01.818436  8083 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:26:01.819526  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.820029  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.820133  8083 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.810232  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:01.820766  8083 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.821878  8187 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.822441  4679 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:26:01.822824  8137 log.cc:927] Time spent T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.985s	user 0.001s	sys 0.002s
I20260709 12:26:01.823490  4673 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:01.816315  8093 consensus_queue.cc:1059] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:01.826156  4676 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:01.773017  8169 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:26:01.769922  8108 consensus_queue.cc:260] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.812723  4597 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:01.796912  4748 rpcz_store.cc:276] 0709 12:26:00.300818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.582700 (+281882us) service_pool.cc:224] Handling call
0709 12:26:00.611095 (+ 28395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.611322 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:01.044713 (+433391us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.778131 (+733418us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.778243 (+   112us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:01.778333 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.778425 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.778608 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5488,"raft.queue_time_us":26221,"raft.run_cpu_time_us":477,"raft.run_wall_time_us":47615,"thread_start_us":714129,"threads_started":2,"wal-append.queue_time_us":414345}
I20260709 12:26:01.779918  8180 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:26:01.811280  8117 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:01.835203  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.838320  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.843557  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.844363  4679 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:01.785789  7782 consensus_queue.cc:1059] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:26:01.778236  8038 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.859674  8195 raft_consensus.cc:3055] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:01.860286  8195 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:01.845402  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:01.822602  8187 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.861618  4673 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:01.862003  7782 raft_consensus.cc:3055] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:01.862537  7782 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:01.863688  7782 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.865089  7782 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:01.866020  4754 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 12: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:01.840092  8117 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.845084  7983 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.878322  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.882644  4753 raft_consensus.cc:2410] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:01.886376  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2622) took 1292 ms. Trace:
I20260709 12:26:01.886855  4749 rpcz_store.cc:276] 0709 12:26:00.561881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.583433 (+ 21552us) service_pool.cc:224] Handling call
0709 12:26:00.634594 (+ 51161us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.654285 (+ 19691us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.654340 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.665621 (+ 11281us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.469979 (+804358us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:01.470595 (+   616us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.602282 (+131687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.619052 (+ 16770us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.846273 (+227221us) raft_consensus.cc:1685] finished
0709 12:26:01.846290 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.854697 (+  8407us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56633,"prepare.queue_time_us":296,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":68,"raft.queue_time_us":90184,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":4320,"spinlock_wait_cycles":18642560,"thread_start_us":489837,"threads_started":2,"wal-append.queue_time_us":818842}
I20260709 12:26:01.811424  7863 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.889787  7863 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:01.892380  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.893355  4674 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 7: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:01.894690  7971 raft_consensus.cc:3055] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:01.883957  8140 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.895183  7971 raft_consensus.cc:740] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:01.895936  8140 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.897423  7971 consensus_queue.cc:260] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.898566  8140 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:01.895125  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.879735  8117 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:01.845084  7771 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.855865  7968 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.882297  8093 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.907434  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:01.907776  8093 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.908388  8038 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:01.909538  5347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 538) took 1818 ms. Trace:
I20260709 12:26:01.910231  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.910322  8019 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.911266  4679 raft_consensus.cc:2410] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:01.911376  8019 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.912163  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.912101  8019 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:01.909930  5347 rpcz_store.cc:276] 0709 12:26:00.090776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.090932 (+   156us) service_pool.cc:224] Handling call
0709 12:26:00.091340 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.091704 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.091734 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.092188 (+   454us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.246511 (+154323us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:00.246555 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.246656 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.796295 (+1549639us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:01.909111 (+112816us) raft_consensus.cc:1685] finished
0709 12:26:01.909130 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.909369 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13322,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":7972,"spinlock_wait_cycles":43514240,"thread_start_us":124973,"threads_started":1,"wal-append.queue_time_us":593982}
I20260709 12:26:01.874465  8169 consensus_queue.cc:1243] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:01.904404  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.918610  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:01.899556  7971 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:01.924065  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.852047  8131 consensus_queue.cc:1059] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:26:01.925030  4672 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:01.930899  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.906642  8105 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:01.940507  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.940372  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:01.919341  7787 raft_consensus.cc:3055] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:01.924985  7771 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:01.936121  4754 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:01.904405  8132 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:01.941977  4673 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:01.919864  8140 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.948354  4681 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:26:01.955118  4681 pending_rounds.cc:85] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 7
I20260709 12:26:01.955451  4681 pending_rounds.cc:107] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:26:01.955937  4681 raft_consensus.cc:2889] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:01.951149  8160 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:01.951431  4679 raft_consensus.cc:2410] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:01.952589  4749 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:26:01.945955  7787 raft_consensus.cc:740] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:01.925112  8083 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:01.954145  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:01.933014  8003 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:01.961561  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.963739  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.964880  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.964533  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.966266  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:01.972004  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.973495  8169 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:26:01.974399  8133 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:26:01.974815  8070 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:01.978294  8011 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:01.978880  7968 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:01.974272  5312 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:01.968130  4674 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:01.968425  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.983986  8160 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.984673  8160 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.985469  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:01.988147  8003 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:01.988966  8003 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:01.992244  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.993633  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:01.982379  4748 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:26:01.994729  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:01.995445  4604 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:01.995668  4673 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:01.995472  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.996627  4598 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:01.997089  4748 raft_consensus.cc:2410] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:01.997005  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:01.996749  4597 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:26:01.997808  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:01.998025  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:01.998246  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:01.999660  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:02.001313  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.000819  4673 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:01.999850  8187 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:02.007651  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.014186  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:02.015020  4749 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:02.018685  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:02.024693  8192 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:26:02.029194  8107 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:02.024785  8189 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:26:02.075703  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:02.079545  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:02.030519  4753 raft_consensus.cc:2376] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8. Re-sending same reply.
I20260709 12:26:01.967202  8140 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:02.085654  4754 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:02.087178  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:02.091856  4749 raft_consensus.cc:2393] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:02.101696  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:02.102428  4599 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:02.104584  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 130402us
I20260709 12:26:02.107899  4604 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:26:02.110106  4681 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:26:02.111974  8155 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:02.112411  8155 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.112767  8155 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:02.114564  8145 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:02.016474  8052 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:02.122040  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:02.094745  8093 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:02.136722  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.921253  8195 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:02.130259  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:02.141876  4599 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
W20260709 12:26:02.142997  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.143095  4672 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:02.110175  8019 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:02.148003  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:02.153895  4674 raft_consensus.cc:2368] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9.
I20260709 12:26:02.187722  8051 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.183889  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:02.200702  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.192217  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.193266  4749 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
W20260709 12:26:02.206365  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.139999  7983 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:02.157557  8012 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:02.228281  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:02.233771  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:02.234542  4748 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:02.241973  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:02.188529  8051 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:02.189811  8157 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:02.225725  8122 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.227919  8012 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:02.262552  8160 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:02.263949  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:02.265085  4602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1701) took 1713 ms (client timeout 1899 ms). Trace:
I20260709 12:26:02.262203  8051 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:02.272023  8150 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.979s	user 0.001s	sys 0.000s
I20260709 12:26:02.275007  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.275369  4749 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:26:02.276178  8140 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:02.279413  4748 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:02.266371  8122 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:02.315543  4748 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:02.316512  4748 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:02.317625  4748 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:02.317795  8122 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:02.283188  8117 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:02.318898  5347 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 12:26:02.285626  8199 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:26:02.319561  4753 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:26:02.321861  8117 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:02.324373  8155 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:02.309693  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:02.302785  4602 rpcz_store.cc:269] 0709 12:26:00.551895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.552004 (+   109us) service_pool.cc:224] Handling call
0709 12:26:02.264622 (+1712618us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54002e59d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:02.264828 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":705701120}
I20260709 12:26:02.311532  4672 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
W20260709 12:26:02.314914  8125 log.cc:927] Time spent T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 2.270s	user 0.003s	sys 0.002s
I20260709 12:26:02.332507  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:02.302197  8012 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:02.400588  8193 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.439s	user 0.000s	sys 0.001s
I20260709 12:26:02.338346  4748 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:02.796219  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.794811  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2123) took 1320 ms. Trace:
W20260709 12:26:02.713384  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.333909  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.792490  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:02.407171  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:02.663865  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.819548  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.624720  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.408463  4605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1704) took 1751 ms (client timeout 1565 ms). Trace:
I20260709 12:26:02.793792  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1706) took 1950 ms. Trace:
W20260709 12:26:02.334306  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.333187  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:02.824116  4674 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:02.817962  4747 rpcz_store.cc:276] 0709 12:26:01.474902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.486720 (+ 11818us) service_pool.cc:224] Handling call
0709 12:26:02.794588 (+1307868us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b54001f1654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:02.794726 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64041856}
W20260709 12:26:02.820775  4605 rpcz_store.cc:269] 0709 12:26:00.657020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.657172 (+   152us) service_pool.cc:224] Handling call
0709 12:26:02.408199 (+1751027us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b5400034654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:02.408376 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1561065216}
W20260709 12:26:02.783703  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.827980  4681 raft_consensus.cc:2393] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:02.826956  4676 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
W20260709 12:26:02.424832  8130 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.614s	user 0.013s	sys 0.000s
I20260709 12:26:02.410033  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:02.825986  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.432032  7782 consensus_meta.cc:287] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.567s	user 0.008s	sys 0.004s
W20260709 12:26:02.824683  5313 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.014s	sys 0.040s
I20260709 12:26:02.402552  8036 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.821169  4596 rpcz_store.cc:276] 0709 12:26:00.843106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.843381 (+   275us) service_pool.cc:224] Handling call
0709 12:26:00.876776 (+ 33395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.630292 (+753516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.793356 (+1163064us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b540013e2d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:02.793694 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1541159168}
W20260709 12:26:02.840356  5313 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.014s	sys 0.040s
I20260709 12:26:02.287833  8052 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:02.596699  7971 consensus_meta.cc:287] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.665s	user 0.008s	sys 0.005s
W20260709 12:26:02.843750  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 509928us
I20260709 12:26:02.399722  5341 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:26:02.844625  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.333652  8081 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.845621  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:02.848666  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.849742  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.405719  7986 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:26:02.852046  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.852464  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.850835  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:02.845382  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:02.854338  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:02.854118  8036 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:02.793457  7981 consensus_queue.cc:1059] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:26:02.856060  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:02.386116  8157 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:02.856951  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.857065  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.848146  5341 pending_rounds.cc:85] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 7
I20260709 12:26:02.412127  8038 consensus_queue.cc:1059] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.439s
I20260709 12:26:02.860713  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.864394  4680 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:02.864984  4681 raft_consensus.cc:2410] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:02.866343  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:02.406086  7975 consensus_queue.cc:1059] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.451s
I20260709 12:26:02.866998  4672 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:26:02.851506  8155 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:02.869505  4605 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
W20260709 12:26:02.875653  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.876185  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.858358  8130 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.877259  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.878057  4673 raft_consensus.cc:2410] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:26:02.866470  5341 pending_rounds.cc:107] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:26:02.880402  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:02.867789  7782 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:02.881485  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.884346  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:02.884197  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:02.888345  5349 raft_consensus.cc:2410] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:26:02.881417  5341 raft_consensus.cc:2889] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:02.887359  4681 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:26:02.883975  8130 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:02.879128  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:02.888401  4596 raft_consensus.cc:2393] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:02.874186  7820 raft_consensus.cc:3055] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:02.869277  7986 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:02.874264  8208 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:02.897487  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:02.891041  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:02.883605  7782 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.900362  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:02.894542  4676 raft_consensus.cc:2368] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11.
W20260709 12:26:02.901846  4720 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.046s	sys 0.080s
I20260709 12:26:02.903754  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:02.904218  4720 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.046s	sys 0.080s
W20260709 12:26:02.904557  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 825337us
I20260709 12:26:02.890458  7796 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.906508  7796 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:02.903075  8036 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:02.911257  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:02.905715  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.914594  8195 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:02.869889  8081 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:02.920810  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.918598  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:02.919420  4673 raft_consensus.cc:2393] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:02.921753  4681 raft_consensus.cc:2410] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 12:26:02.921815  4754 consensus_meta.cc:287] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.791s	user 0.006s	sys 0.005s
I20260709 12:26:02.922196  4754 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:02.922259  4674 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:02.923599  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:02.924149  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1704) took 1011 ms. Trace:
I20260709 12:26:02.924466  4676 rpcz_store.cc:276] 0709 12:26:01.912948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.839356 (+926408us) service_pool.cc:224] Handling call
0709 12:26:02.924081 (+ 84725us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.894131  8155 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.899951  7820 raft_consensus.cc:740] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:02.894296  8157 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:02.925712  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:02.933291  4674 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:26:02.936667  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 518) took 1045 ms. Trace:
I20260709 12:26:02.937814  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:02.937595  4674 rpcz_store.cc:276] 0709 12:26:01.890965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.927599 (+1036634us) service_pool.cc:224] Handling call
0709 12:26:02.933000 (+  5401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.936394 (+  3394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.936610 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.935814  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:02.938699  5342 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:02.939669  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:02.939687  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:02.940905  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:02.936767  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:02.938097  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:02.935881  8035 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.738s	user 0.026s	sys 0.010s
I20260709 12:26:02.942631  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1697) took 1031 ms. Trace:
I20260709 12:26:02.944177  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:02.949891  4750 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:02.949990  5349 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 4. Current term is 5. Ops: []
W20260709 12:26:02.948944  4755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2124) took 1473 ms (client timeout 1876 ms). Trace:
W20260709 12:26:02.951874  4755 rpcz_store.cc:269] 0709 12:26:01.475357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.479199 (+  3842us) service_pool.cc:224] Handling call
0709 12:26:02.948772 (+1469573us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b5400184054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:02.948883 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114629376}
W20260709 12:26:02.949214  8202 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.663s	user 0.001s	sys 0.001s
I20260709 12:26:02.905715  7782 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:02.952656  4681 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:02.953485  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 516) took 1063 ms. Trace:
I20260709 12:26:02.935835  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:02.954629  4605 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:02.945258  5346 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:02.921025  8169 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:02.957063  5346 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:26:02.957479  5346 pending_rounds.cc:85] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 6
I20260709 12:26:02.957939  5346 pending_rounds.cc:107] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:26:02.958207  5346 raft_consensus.cc:2889] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:02.958335  4673 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:26:02.959156  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1700) took 1047 ms. Trace:
I20260709 12:26:02.957444  7986 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:02.951543  4679 rpcz_store.cc:276] 0709 12:26:01.910935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.911869 (+   934us) service_pool.cc:224] Handling call
0709 12:26:02.942429 (+1030560us) spinlock_profiling.cc:243] Waited 38.3 ms on lock 0x7b540017f054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:02.942571 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88118016}
I20260709 12:26:02.961786  5346 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEARNER]: Advancing to term 10
I20260709 12:26:02.953874  4681 rpcz_store.cc:276] 0709 12:26:01.890291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.926119 (+1035828us) service_pool.cc:224] Handling call
0709 12:26:02.952161 (+ 26042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.953287 (+  1126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.953436 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.961742  4673 rpcz_store.cc:276] 0709 12:26:01.911826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.950194 (+1038368us) service_pool.cc:224] Handling call
0709 12:26:02.958051 (+  7857us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.958830 (+   779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.959090 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.965757  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:02.966600  7971 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:02.967581  4681 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:26:02.968263  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2675) took 1016 ms. Trace:
I20260709 12:26:02.967084  4647 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:02.964548  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:02.968878  4681 rpcz_store.cc:276] 0709 12:26:01.951703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.966168 (+1014465us) service_pool.cc:224] Handling call
0709 12:26:02.967333 (+  1165us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.968006 (+   673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.968189 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.969353  4676 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:02.969931  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 78974.2us
I20260709 12:26:02.968207  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2662) took 1347 ms. Trace:
I20260709 12:26:02.971473  4678 rpcz_store.cc:276] 0709 12:26:01.625271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.642003 (+ 16732us) service_pool.cc:224] Handling call
0709 12:26:01.642370 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.765512 (+123142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.765558 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.773201 (+  7643us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:01.773266 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.967984 (+1194718us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54000b22d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:02.968030 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.968150 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271269120}
I20260709 12:26:02.972302  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1701) took 1060 ms. Trace:
I20260709 12:26:02.972651  4676 rpcz_store.cc:276] 0709 12:26:01.912159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.951033 (+1038874us) service_pool.cc:224] Handling call
0709 12:26:02.969198 (+ 18165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.972046 (+  2848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.972245 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:02.955224  8035 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.836s	user 0.026s	sys 0.011s
I20260709 12:26:02.941538  7796 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:02.975466  4679 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:26:02.975193  7971 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:02.980340  8062 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.943105  8081 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:02.980558  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:02.982301  4673 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:26:02.981521  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1703) took 1068 ms. Trace:
I20260709 12:26:02.984495  4679 rpcz_store.cc:276] 0709 12:26:01.912678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.963953 (+1051275us) service_pool.cc:224] Handling call
0709 12:26:02.971693 (+  7740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.981177 (+  9484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.981455 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.987022  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1707) took 1042 ms. Trace:
I20260709 12:26:02.987349  4673 rpcz_store.cc:276] 0709 12:26:01.944363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.964575 (+1020212us) service_pool.cc:224] Handling call
0709 12:26:02.979911 (+ 15336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.986826 (+  6915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.986980 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.989830  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:02.945434  8155 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:02.978392  8052 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:02.932658  7820 consensus_queue.cc:260] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:02.994978  8115 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:02.985396  8062 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:02.998895  5342 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:03.037756  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2634) took 1513 ms. Trace:
I20260709 12:26:03.002111  7796 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:03.003597  8105 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:03.004592  7840 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:03.005239  8108 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:03.039582  4679 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:03.038261  4750 rpcz_store.cc:276] 0709 12:26:01.524563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.678801 (+154238us) service_pool.cc:224] Handling call
0709 12:26:02.949704 (+1270903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.955862 (+  6158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.037271 (+ 81409us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400184050. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:03.037679 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2882936064}
I20260709 12:26:03.051242  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2138) took 1218 ms. Trace:
I20260709 12:26:03.051756  8155 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:03.054268  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:03.056236  5347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 546) took 1092 ms. Trace:
I20260709 12:26:03.062815  5344 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:03.065847  5347 rpcz_store.cc:276] 0709 12:26:01.964036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.964191 (+   155us) service_pool.cc:224] Handling call
0709 12:26:02.007064 (+ 42873us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.319307 (+312243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.055882 (+736575us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b5400640354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:03.056152 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716542336}
I20260709 12:26:03.062973  4681 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:03.066644  5346 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:26:03.067293  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2677) took 1115 ms. Trace:
I20260709 12:26:03.067538  4681 rpcz_store.cc:276] 0709 12:26:01.952349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.982422 (+1030073us) service_pool.cc:224] Handling call
0709 12:26:03.047436 (+ 65014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.067113 (+ 19677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.067251 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.067855  5348 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:26:03.062351  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 505) took 3385 ms. Trace:
I20260709 12:26:03.064457  4601 rpcz_store.cc:276] 0709 12:26:01.833103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.835057 (+  1954us) service_pool.cc:224] Handling call
0709 12:26:02.408372 (+573315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.408650 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.408691 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.408744 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:02.408786 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.408855 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.051065 (+642210us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b5400034650. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:03.051176 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1302562048}
I20260709 12:26:03.038821  8052 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.068886  4675 rpcz_store.cc:276] 0709 12:25:59.676798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.677373 (+   575us) service_pool.cc:224] Handling call
0709 12:25:59.677711 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.813307 (+1135596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.059050 (+2245743us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540017eb54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:03.062166 (+  3116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2567868288}
I20260709 12:26:03.082587  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:03.084641  4604 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:03.046829  7975 consensus_queue.cc:1243] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:26:03.084195  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:03.089499  5349 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:26:03.090322  8081 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.102329  8081 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.104509  8081 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:03.105497  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:03.106120  4601 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:03.063542  7840 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:03.079056  7820 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.109495  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:03.118803  4568 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:03.119989  8212 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:03.120275  7981 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:03.081804  8105 consensus_queue.cc:1243] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:03.124382  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:03.087344  8052 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.125613  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:03.133116  4604 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
W20260709 12:26:03.124444  4720 leader_election.cc:336] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.965s (SENT)
I20260709 12:26:03.126955  4644 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:03.137108  4647 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:03.120990  8213 consensus_queue.cc:1059] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:26:03.136245  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:03.140690  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 202073us
W20260709 12:26:03.143270  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.146965  4679 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:03.149034  5348 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:03.178431  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:03.138897  8210 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:03.183084  5312 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:03.184859  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:03.187640  4604 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:26:03.188131  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1719) took 1096 ms. Trace:
W20260709 12:26:03.189905  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 58062.7us
I20260709 12:26:03.191053  8221 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:03.192584  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:03.133764  8012 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:03.195237  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 100816us
I20260709 12:26:03.188411  4604 rpcz_store.cc:276] 0709 12:26:02.091422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.148898 (+1057476us) service_pool.cc:224] Handling call
0709 12:26:03.188075 (+ 39177us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:03.196642  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47990.5us
I20260709 12:26:03.198501  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:03.199210  4679 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:03.157881  8130 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:03.204092  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:03.205320  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 287848us
I20260709 12:26:03.186515  8210 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.213465  4680 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:03.213491  4601 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:03.196200  8038 raft_consensus.cc:3055] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:03.216666  8038 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:03.217885  8038 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:03.219512  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.219496  8038 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.226734  8195 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.227312  8195 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:03.228313  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:03.200264  7796 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:03.231678  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 104808us
I20260709 12:26:03.235850  4644 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:03.140466  7840 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.237016  8212 consensus_queue.cc:1059] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:26:03.242774  7840 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.243186  7981 consensus_queue.cc:1059] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:03.244319  7981 raft_consensus.cc:3055] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:03.244644  7981 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:03.245838  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.245463  7981 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.246840  7981 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:03.246912  8012 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:03.202320  8221 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:03.202275  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:03.250814  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.241130  4602 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:03.200927  7975 consensus_queue.cc:1059] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:26:03.264597  8169 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:03.259090  4720 leader_election.cc:336] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 2.082s (SENT)
W20260709 12:26:03.272554  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.231048  8210 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.229470  8195 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:03.260466  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:03.262066  8212 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:03.262230  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:03.288517  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:03.262986  4569 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:03.263645  8211 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:03.265400  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2664) took 1575 ms. Trace:
W20260709 12:26:03.299459  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.265349  7975 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:03.238569  7796 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:03.266093  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 73757.1us
I20260709 12:26:03.302064  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:03.269721  8222 consensus_queue.cc:1175] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49277a7653524787a715232ab908bbc6" responder_term: 8 status { last_received { term: 4 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: [7.4-7.4]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:03.258987  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1715) took 1248 ms. Trace:
I20260709 12:26:03.275952  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:03.242139  4680 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:03.311298  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 525) took 1179 ms. Trace:
I20260709 12:26:03.311403  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:03.277274  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:03.259338  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2144) took 1185 ms. Trace:
I20260709 12:26:03.312351  4754 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:03.312310  4601 rpcz_store.cc:276] 0709 12:26:02.073968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.227829 (+1153861us) service_pool.cc:224] Handling call
0709 12:26:03.259281 (+ 31452us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.311623  4680 rpcz_store.cc:276] 0709 12:26:02.132063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.215347 (+1083284us) service_pool.cc:224] Handling call
0709 12:26:03.215747 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.311030 (+ 95283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.311245 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:03.310597  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 47744.9us
I20260709 12:26:03.280956  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:03.282249  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:03.262923  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:03.289203  4748 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 12:26:03.291070  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 55304.1us
I20260709 12:26:03.346480  4644 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:03.296051  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:03.300625  5346 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:03.354959  8223 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:03.355931  8223 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.356452  8223 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.356950  8232 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:26:03.358668  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 56692.2us
I20260709 12:26:03.359570  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:03.359980  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:03.360704  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 524) took 1147 ms. Trace:
I20260709 12:26:03.360988  4601 rpcz_store.cc:276] 0709 12:26:02.213098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.313481 (+1100383us) service_pool.cc:224] Handling call
0709 12:26:03.360648 (+ 47167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.361660  4647 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:03.362663  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:03.363936  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:03.366756  4604 raft_consensus.cc:2393] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:03.310127  4600 rpcz_store.cc:276] 0709 12:26:02.010982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.012182 (+  1200us) service_pool.cc:224] Handling call
0709 12:26:02.017169 (+  4987us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.121674 (+104505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.130206 (+  8532us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:02.139496 (+  9290us) log.cc:844] Serialized 26 byte log entry
0709 12:26:02.888807 (+749311us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b180032e3c0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3972a6 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9
0709 12:26:03.052509 (+163702us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:03.052568 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.052651 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.248575 (+195924us) raft_consensus.cc:1685] finished
0709 12:26:03.248601 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.258922 (+ 10321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26380,"raft.queue_time_us":62618,"spinlock_wait_cycles":41995904,"thread_start_us":163455,"threads_started":1,"wal-append.queue_time_us":983252}
I20260709 12:26:03.299592  4677 rpcz_store.cc:276] 0709 12:26:01.690200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.907967 (+1217767us) service_pool.cc:224] Handling call
0709 12:26:02.908312 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.921305 (+ 12993us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:02.921337 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:02.925535 (+  4198us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.200925 (+275390us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:03.200961 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.201046 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.265182 (+ 64136us) raft_consensus.cc:1685] finished
0709 12:26:03.265200 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.265344 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4276,"raft.queue_time_us":12747,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":8246,"thread_start_us":274928,"threads_started":1,"wal-append.queue_time_us":337771}
I20260709 12:26:03.368898  4680 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:03.369829  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1711) took 1228 ms. Trace:
I20260709 12:26:03.370337  4680 rpcz_store.cc:276] 0709 12:26:02.141127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.342660 (+1201533us) service_pool.cc:224] Handling call
0709 12:26:03.368644 (+ 25984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.369492 (+   848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.369758 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.303696  5342 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:03.283840  7863 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:03.309985  7796 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:03.372491  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2151) took 1090 ms. Trace:
W20260709 12:26:03.376677  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 95794.3us
I20260709 12:26:03.376750  4604 rpcz_store.cc:276] 0709 12:26:02.276449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.363691 (+1087242us) service_pool.cc:224] Handling call
0709 12:26:03.367113 (+  3422us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.374661  4647 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:03.371980  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2155) took 1094 ms. Trace:
I20260709 12:26:03.378384  4602 rpcz_store.cc:276] 0709 12:26:02.280377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.262651 (+982274us) service_pool.cc:224] Handling call
0709 12:26:03.371915 (+109264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.358788  8062 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:03.358341  8212 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:03.380988  4680 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:26:03.382383  4647 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:03.383934  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2685) took 1135 ms. Trace:
I20260709 12:26:03.384783  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:03.385490  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 527) took 1140 ms. Trace:
I20260709 12:26:03.385896  4604 rpcz_store.cc:276] 0709 12:26:02.243474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.377637 (+1134163us) service_pool.cc:224] Handling call
0709 12:26:03.377906 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.381430 (+  3524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.381472 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:03.381527 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:03.381586 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.381649 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.381752 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.388460  4604 raft_consensus.cc:1240] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 3. Current term is 4. Ops: []
W20260709 12:26:03.390549  8207 log.cc:927] Time spent T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.509s	user 0.000s	sys 0.001s
I20260709 12:26:03.392262  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1716) took 1377 ms. Trace:
W20260709 12:26:03.393385  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.394042  4604 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:03.371906  8195 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:03.392619  4598 rpcz_store.cc:276] 0709 12:26:02.014376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.018728 (+  4352us) service_pool.cc:224] Handling call
0709 12:26:02.076554 (+ 57826us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.118642 (+ 42088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.118688 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:02.183509 (+ 64821us) log.cc:844] Serialized 26 byte log entry
0709 12:26:02.573258 (+389749us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:02.573325 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.573452 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.389919 (+816467us) spinlock_profiling.cc:243] Waited 6.02 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:03.391951 (+  2032us) raft_consensus.cc:1685] finished
0709 12:26:03.391968 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.392198 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22651,"raft.queue_time_us":41023,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":12965,"spinlock_wait_cycles":13854464,"thread_start_us":327768,"threads_started":1,"wal-append.queue_time_us":658844}
I20260709 12:26:03.395754  4604 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:26:03.290750  8224 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 12:26:03.384295  4680 rpcz_store.cc:276] 0709 12:26:02.248172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.371329 (+1123157us) service_pool.cc:224] Handling call
0709 12:26:03.380680 (+  9351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.383701 (+  3021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.383882 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.397897  4604 raft_consensus.cc:3055] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:03.397951  4598 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:26:03.402679  4604 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:03.398427  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:03.398927  7986 raft_consensus.cc:3055] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:03.399571  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:03.403775  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:03.411962  4680 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:26:03.396744  8212 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.414896  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2145) took 1305 ms. Trace:
I20260709 12:26:03.415323  4602 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:26:03.415599  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1713) took 1147 ms. Trace:
I20260709 12:26:03.417419  4680 rpcz_store.cc:276] 0709 12:26:02.268640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.402670 (+1134030us) service_pool.cc:224] Handling call
0709 12:26:03.411569 (+  8899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.412428 (+   859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.415540 (+  3112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.415907  4569 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:03.416352  4601 rpcz_store.cc:276] 0709 12:26:02.109618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.367346 (+1257728us) service_pool.cc:224] Handling call
0709 12:26:03.414832 (+ 47486us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.378443  7863 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.433790  5345 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:26:03.435130  5345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 539) took 2010 ms. Trace:
I20260709 12:26:03.435467  5345 rpcz_store.cc:276] 0709 12:26:01.424988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.454863 (+ 29875us) service_pool.cc:224] Handling call
0709 12:26:01.480149 (+ 25286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.434218 (+1954069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.434871 (+   653us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b54007d1254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:03.435067 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4491832064}
I20260709 12:26:03.435895  8224 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:03.436697  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:03.436520  8224 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.437436  8011 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.438289  8011 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:03.439322  4604 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.441144  4604 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:03.442129  8218 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:03.442597  8218 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.442849  4569 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:03.442611  8223 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:03.421115  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2156) took 1136 ms. Trace:
I20260709 12:26:03.444150  8223 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:03.444036  4602 rpcz_store.cc:276] 0709 12:26:02.280700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.379134 (+1098434us) service_pool.cc:224] Handling call
0709 12:26:03.384911 (+  5777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.416964 (+ 32053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.417197 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.444684  8223 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:03.444847  8018 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:03.445166  8223 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:03.405931  7986 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:03.445820  7863 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:03.446269  7986 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.447789  7986 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:03.448033  8204 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.775s	user 0.001s	sys 0.000s
I20260709 12:26:03.451561  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1712) took 1455 ms. Trace:
I20260709 12:26:03.422251  8195 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:03.418844  8212 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:03.456323  5345 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:03.443291  8211 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:03.452708  8210 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:03.445307  8018 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:03.451941  4597 rpcz_store.cc:276] 0709 12:26:01.995754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.998035 (+  2281us) service_pool.cc:224] Handling call
0709 12:26:01.998373 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:01.998781 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.998808 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:01.999421 (+   613us) log.cc:844] Serialized 52 byte log entry
0709 12:26:02.244806 (+245385us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:02.244864 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.244966 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.447361 (+1202395us) spinlock_profiling.cc:243] Waited 56.4 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:03.451127 (+  3766us) raft_consensus.cc:1685] finished
0709 12:26:03.451148 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.451377 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1461,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"spinlock_wait_cycles":129713152,"thread_start_us":244880,"threads_started":1,"wal-append.queue_time_us":444875}
I20260709 12:26:03.405838  8219 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:03.443013  8218 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:03.426823  8052 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.460287  4680 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:26:03.474488  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1717) took 1202 ms. Trace:
I20260709 12:26:03.474916  4680 rpcz_store.cc:276] 0709 12:26:02.272217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.418300 (+1146083us) service_pool.cc:224] Handling call
0709 12:26:03.459869 (+ 41569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.474232 (+ 14363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.474428 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:03.475996  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 84084.7us
I20260709 12:26:03.476289  4680 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:03.477108  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1718) took 1204 ms. Trace:
W20260709 12:26:03.478502  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 79058.4us
I20260709 12:26:03.477483  4680 rpcz_store.cc:276] 0709 12:26:02.272585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.475819 (+1203234us) service_pool.cc:224] Handling call
0709 12:26:03.476114 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.476833 (+   719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.477050 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.461516  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2676) took 1509 ms. Trace:
W20260709 12:26:03.472725  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 126330us
W20260709 12:26:03.487813  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.480552  4644 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:03.469452  8062 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:03.474968  8018 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:03.472183  8081 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.414208  7796 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:03.497128  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:03.497273  4598 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:03.442795  8213 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:26:03.501951  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 521) took 1610 ms. Trace:
I20260709 12:26:03.480082  4676 rpcz_store.cc:276] 0709 12:26:01.952034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.973260 (+1021226us) service_pool.cc:224] Handling call
0709 12:26:02.982966 (+  9706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.993198 (+ 10232us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:03.144413 (+151215us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.461190 (+316777us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:03.461238 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:03.461285 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.461358 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.461460 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2193,"prepare.queue_time_us":160780,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":76141,"spinlock_wait_cycles":22661888,"thread_start_us":330475,"threads_started":3,"wal-append.queue_time_us":102671}
I20260709 12:26:03.524443  4674 rpcz_store.cc:276] 0709 12:26:01.891851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.941995 (+1050144us) service_pool.cc:224] Handling call
0709 12:26:02.942313 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.942559 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:03.158292 (+215733us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.501460 (+343168us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:03.501529 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:03.501627 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.501722 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.501878 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8940,"prepare.queue_time_us":211177,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":82928,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":518171,"threads_started":3,"wal-append.queue_time_us":185337}
I20260709 12:26:03.499691  8195 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.530648  4644 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:03.523190  8108 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.520171  8231 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:03.550566  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 107777us
W20260709 12:26:03.551339  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 59305.3us
I20260709 12:26:03.529911  7840 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:03.533749  7820 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:03.531548  7863 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:03.556632  4604 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:26:03.562371  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 164602us
I20260709 12:26:03.564085  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:03.554728  8108 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:03.523180  8081 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.519809  8219 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:03.555924  7820 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:03.562072  7782 consensus_meta.cc:287] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.590s	user 0.005s	sys 0.006s
W20260709 12:26:03.593604  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 60481.6us
I20260709 12:26:03.559194  8229 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:03.594204  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:03.597028  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:03.599888  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.595754  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:03.551831  8195 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:03.612507  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:03.591341  7820 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.586771  7840 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:03.585530  7863 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:03.635754  4752 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:03.599257  7835 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.162s
I20260709 12:26:03.642903  8245 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:03.643083  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:03.643738  4754 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:26:03.642607  5343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 540) took 2160 ms. Trace:
I20260709 12:26:03.643643  8245 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:03.644552  8129 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.644320  5343 rpcz_store.cc:276] 0709 12:26:01.425596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.427566 (+  1970us) service_pool.cc:224] Handling call
0709 12:26:01.428326 (+   760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.674017 (+245691us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:01.674047 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.674114 (+    67us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:01.674160 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.585735 (+1911575us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b540055e6d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:03.585788 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.585928 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563942656}
I20260709 12:26:03.645249  8129 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.646106  7796 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:03.646325  4753 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:03.647302  8129 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:03.648504  8243 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:26:03.648947  4569 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:03.653831  8247 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:03.654084  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 523) took 1531 ms. Trace:
W20260709 12:26:03.654281  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.654413  8247 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:03.584997  8236 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:26:03.636848  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.653833  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 524) took 1522 ms. Trace:
I20260709 12:26:03.678279  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:03.679126  4752 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.598275  8011 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:03.685640  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:03.686864  4753 raft_consensus.cc:2393] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 5: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:03.656072  8213 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:03.654489  4675 rpcz_store.cc:276] 0709 12:26:02.122419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.108743 (+986324us) service_pool.cc:224] Handling call
0709 12:26:03.129851 (+ 21108us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.193988 (+ 64137us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.194034 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.234591 (+ 40557us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.404521 (+169930us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:03.404566 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.404655 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.653894 (+249239us) raft_consensus.cc:1685] finished
0709 12:26:03.653915 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.654023 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55247,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":132,"thread_start_us":159157,"threads_started":2,"wal-append.queue_time_us":202155}
I20260709 12:26:03.675657  4679 rpcz_store.cc:276] 0709 12:26:02.131731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.211634 (+1079903us) service_pool.cc:224] Handling call
0709 12:26:03.237858 (+ 26224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.403808 (+165950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.403840 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.419228 (+ 15388us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.555379 (+136151us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:03.555448 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.555590 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.653642 (+ 98052us) raft_consensus.cc:1685] finished
0709 12:26:03.653660 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.653777 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18942,"raft.queue_time_us":101536,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":88,"thread_start_us":196572,"threads_started":2,"wal-append.queue_time_us":128230}
W20260709 12:26:03.664709  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.655793  7840 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:03.699596  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.669981  8236 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:03.683921  8211 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:03.717135  4753 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:26:03.690220  8235 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:03.702139  7840 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.673635  7981 consensus_queue.cc:1059] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:26:03.671074  7863 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:03.724049  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 127101us
I20260709 12:26:03.596781  7782 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.729938  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:03.702018  8052 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:03.700438  8213 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:03.717542  4753 pending_rounds.cc:85] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 8
I20260709 12:26:03.725181  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:03.736080  4753 pending_rounds.cc:107] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:26:03.736447  4753 raft_consensus.cc:2889] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:03.683171  8239 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:26:03.727051  4752 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:03.719530  8211 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.745464  8214 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:03.700137  8038 consensus_queue.cc:1059] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:26:03.719313  8236 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:03.723520  8210 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:03.690827  8212 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.753420  4677 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:03.754593  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:03.750193  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:03.738260  8213 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.772083  8256 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:03.772868  8246 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:26:03.755924  8211 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.783293  4754 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:03.747674  8235 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:03.826817  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:03.676636  8224 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:03.838662  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 111020us
W20260709 12:26:03.846390  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 96261.8us
I20260709 12:26:03.846786  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:03.847734  4645 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:03.703791  8245 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:03.775245  8250 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.855886  4675 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:03.857945  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:03.858812  4752 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:03.871219  4645 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:03.893987  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:03.874522  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:03.906728  4676 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:03.926057  4675 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:03.850150  8108 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:03.909718  5342 consensus_meta.cc:287] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.513s	user 0.004s	sys 0.007s
W20260709 12:26:03.932490  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 61357.2us
I20260709 12:26:03.936312  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:03.932703  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:03.937913  4752 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:03.849812  8254 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.934136  4754 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:03.852984  8235 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:03.892537  8224 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:03.892205  8081 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:03.941887  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 380833us
W20260709 12:26:03.946744  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 106600us
I20260709 12:26:03.947028  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:03.933506  8106 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.964s	user 0.023s	sys 0.019s
I20260709 12:26:03.950301  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:03.956280  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:03.958005  4754 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:26:03.941525  5342 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:26:03.959561  4645 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:03.960279  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:03.961282  4749 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
W20260709 12:26:03.962997  8241 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 12:26:03.965121  8248 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 12:26:03.946415  8224 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:03.965962  8224 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:03.967087  8240 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:26:03.968892  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1717) took 1951 ms. Trace:
I20260709 12:26:03.948340  8232 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:26:03.969859  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:03.970352  8232 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.970701  4753 raft_consensus.cc:2410] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:03.970816  8232 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:03.958788  8239 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:03.972631  8246 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:03.972929  8232 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:03.973776  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:03.974339  4754 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.976980  5314 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:03.978575  5314 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:03.988067  8011 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:03.988574  8011 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:03.988984  8011 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.994555  8108 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:03.994139  7835 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:03.907357  8250 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:03.995131  8108 raft_consensus.cc:697] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:03.996097  8108 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:03.994396  5314 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:03.979046  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 554) took 2245 ms. Trace:
I20260709 12:26:03.969345  4600 rpcz_store.cc:276] 0709 12:26:02.017474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.373821 (+1356347us) service_pool.cc:224] Handling call
0709 12:26:03.374134 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.401731 (+ 27597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.401766 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.413711 (+ 11945us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.528818 (+115107us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:03.546508 (+ 17690us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.546716 (+   208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.546787 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.968658 (+421871us) raft_consensus.cc:1685] finished
0709 12:26:03.968676 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.968811 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17513,"prepare.run_cpu_time_us":15184,"prepare.run_wall_time_us":22371,"raft.queue_time_us":6018,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":3587,"thread_start_us":105716,"threads_started":1,"wal-append.queue_time_us":290958}
W20260709 12:26:03.952808  8106 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.228s	user 0.023s	sys 0.020s
I20260709 12:26:04.002525  4754 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:03.949846  8195 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:03.968925  7986 consensus_meta.cc:287] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.000s
I20260709 12:26:03.943202  8257 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:04.007174  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:04.019894  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:03.997457  4751 rpcz_store.cc:276] 0709 12:26:01.733958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.428248 (+1694290us) service_pool.cc:224] Handling call
0709 12:26:03.428617 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.429004 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.429033 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.429507 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.530424 (+100917us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:03.530467 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.530544 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.965631 (+435087us) raft_consensus.cc:1685] finished
0709 12:26:03.978869 (+ 13238us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.978994 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18787,"raft.queue_time_us":30845,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":4978,"thread_start_us":100518,"threads_started":1,"wal-append.queue_time_us":260279}
I20260709 12:26:04.032912  7863 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:04.017865  8260 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:04.034443  8018 consensus_meta.cc:287] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.003s
W20260709 12:26:04.035076  8251 log.cc:927] Time spent T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:26:04.034961  8018 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.021184  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:03.994396  7796 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:26:04.032236  4753 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 4.
I20260709 12:26:04.037904  4754 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:26:03.997970  8052 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:04.019709  7945 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:04.028198  8245 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:04.019021  8250 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:04.037328  8018 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:04.024039  8256 raft_consensus.cc:697] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:04.042563  7835 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:03.947957  8254 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.052812  4719 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:04.061858  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 111641us
I20260709 12:26:04.062773  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:04.065831  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:04.047232  7782 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:04.048482  8257 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:04.101131  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 122424us
I20260709 12:26:04.106278  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:04.106901  4673 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:04.059710  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 100223us
I20260709 12:26:04.030678  8247 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:04.057111  7945 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.094168  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:04.106113  8232 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:04.126904  4748 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:04.132870  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:04.138602  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:04.141434  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 47342.8us
I20260709 12:26:04.141952  7782 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:04.142398  4720 leader_election.cc:400] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11.
I20260709 12:26:04.142684  4720 leader_election.cc:403] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:04.134970  7945 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.144250  8081 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:04.144747  8081 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:26:04.145153  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.145658  8081 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.149315  7863 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11.
I20260709 12:26:04.151144  5341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 542) took 2360 ms. Trace:
I20260709 12:26:04.151575  5341 rpcz_store.cc:276] 0709 12:26:01.790294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.796638 (+  6344us) service_pool.cc:224] Handling call
0709 12:26:01.820868 (+ 24230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.890319 (+1069451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.150865 (+1260546us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54006fbdd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:04.151077 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1274651008}
W20260709 12:26:04.156952  8141 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 3.211s	user 0.004s	sys 0.000s
I20260709 12:26:04.128265  8195 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:04.160458  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1712) took 1934 ms. Trace:
I20260709 12:26:04.160879  4677 rpcz_store.cc:276] 0709 12:26:02.226548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.369225 (+1142677us) service_pool.cc:224] Handling call
0709 12:26:03.381850 (+ 12625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.782820 (+400970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.160198 (+377378us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b54001bc454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:04.160391 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":853911808}
W20260709 12:26:04.163317  7886 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:26:04.175009  4754 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:04.059032  8106 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:26:04.180774  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:04.098986  7835 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:04.183085  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:04.183877  4604 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.122311  8052 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.125283  8249 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:04.198678  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:04.197762  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:04.200261  4677 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.203060  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:04.225123  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2164) took 1235 ms. Trace:
I20260709 12:26:04.219265  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:04.243373  4598 rpcz_store.cc:276] 0709 12:26:02.989629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.403176 (+413547us) service_pool.cc:224] Handling call
0709 12:26:03.414786 (+ 11610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.521495 (+106709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.224816 (+703321us) spinlock_profiling.cc:243] Waited 77.3 ms on lock 0x7b5400352ad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:04.225036 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177826688}
I20260709 12:26:04.238106  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:04.221824  5344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 488) took 1365 ms. Trace:
I20260709 12:26:04.227821  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 529) took 1124 ms. Trace:
I20260709 12:26:04.246070  4680 rpcz_store.cc:276] 0709 12:26:03.103449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.498905 (+395456us) service_pool.cc:224] Handling call
0709 12:26:03.499341 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.531716 (+ 32375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.531754 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.547099 (+ 15345us) log.cc:844] Serialized 26 byte log entry
0709 12:26:04.227022 (+679923us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b18002940a0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effee05e4bc 00007effeb3ac48b 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1
0709 12:26:04.227238 (+   216us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:04.227288 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.227396 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.227465 (+    69us) raft_consensus.cc:1685] finished
0709 12:26:04.227501 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.227747 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68047,"raft.queue_time_us":91281,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":38520,"spinlock_wait_cycles":10097536,"thread_start_us":26072,"threads_started":1,"wal-append.queue_time_us":141396}
I20260709 12:26:04.244879  5344 rpcz_store.cc:276] 0709 12:26:02.861731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.866703 (+  4972us) service_pool.cc:224] Handling call
0709 12:26:04.221560 (+1354857us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54005c0154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:04.221722 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334253952}
I20260709 12:26:04.060412  7787 consensus_queue.cc:260] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:04.252473  4673 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
W20260709 12:26:04.252453  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 219770us
I20260709 12:26:04.302760  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:04.288895  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.287282  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.309664  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.311657  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 193867us
I20260709 12:26:04.313946  4604 raft_consensus.cc:2410] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:26:04.249938  8268 log.cc:927] Time spent T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:26:04.049114  8235 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:04.231642  8223 consensus_meta.cc:287] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.786s	user 0.010s	sys 0.003s
W20260709 12:26:04.233454  8231 consensus_meta.cc:287] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.684s	user 0.002s	sys 0.010s
I20260709 12:26:04.318158  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:04.218837  8227 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.932s	user 0.002s	sys 0.001s
I20260709 12:26:04.321249  5314 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:04.251869  7820 consensus_meta.cc:287] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.579s	user 0.007s	sys 0.004s
W20260709 12:26:04.333070  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 1.06778e+06us
I20260709 12:26:04.197624  7971 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:04.325371  4597 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:04.325810  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:04.339538  4754 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:04.340154  4603 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:04.327051  7787 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:04.329679  7835 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:04.214298  8254 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:04.204818  8052 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.233716  8249 raft_consensus.cc:697] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:04.176092  8265 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.130s
I20260709 12:26:04.302745  8129 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:04.229833  8238 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:04.233986  8245 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:04.322219  8226 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.036s	user 0.002s	sys 0.011s
I20260709 12:26:04.236697  8247 consensus_queue.cc:1059] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 12:26:04.352459  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:04.237509  7863 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.106s
I20260709 12:26:04.306031  8195 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:26:04.236928  8260 consensus_queue.cc:1059] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 12:26:04.339214  8231 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:04.372767  5314 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:04.364075  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:04.350956  8213 consensus_meta.cc:287] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.508s	user 0.009s	sys 0.004s
I20260709 12:26:04.215278  7840 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:04.236275  8011 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:04.363564  4569 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:04.367669  8245 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.366305  8258 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:26:04.367352  8210 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
W20260709 12:26:04.375267  8236 consensus_meta.cc:287] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.602s	user 0.008s	sys 0.002s
I20260709 12:26:04.384688  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:04.373370  4571 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:04.347541  8105 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.026s
W20260709 12:26:04.370550  8226 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.081s	user 0.003s	sys 0.012s
I20260709 12:26:04.368475  4601 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:04.338294  8223 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.343137  7820 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.348248  7971 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.364852  8273 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:04.386059  8081 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:04.317152  8256 consensus_queue.cc:237] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.412957  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:04.413959  5344 raft_consensus.cc:2368] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11.
I20260709 12:26:04.370311  8259 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:26:04.395159  7886 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:04.421675  4681 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:04.371716  8269 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:26:04.417112  4571 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:04.407181  4644 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:04.407626  8273 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.420382  7863 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:04.422904  4678 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:04.427912  4644 leader_election.cc:400] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11.
I20260709 12:26:04.428246  4644 leader_election.cc:403] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:26:04.410292  8038 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:04.398072  8224 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:04.392155  8213 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:04.354429  8257 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.429728  7863 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:04.398903  8236 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:04.463136  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 46136us
I20260709 12:26:04.429519  8265 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:04.465698  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:04.429538  8273 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:04.482343  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:26:04.497179  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 69403.2us
I20260709 12:26:04.468127  7886 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:04.605329  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 228262us
W20260709 12:26:04.631979  8275 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 12:26:04.630735  8280 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:04.499843  8210 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:04.684478  5346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 553) took 1611 ms. Trace:
I20260709 12:26:04.480337  8250 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:04.464603  8277 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.699541  4680 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:04.708132  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:04.712481  4679 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:04.716245  7820 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:04.716800  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 234530us
I20260709 12:26:04.717329  7971 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:04.717619  8276 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.275s	user 0.002s	sys 0.005s
I20260709 12:26:04.720214  8223 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:04.721576  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.722182  8236 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:04.718129  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:04.718324  8277 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.728348  8277 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:04.707772  5346 rpcz_store.cc:276] 0709 12:26:03.072855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.085619 (+ 12764us) service_pool.cc:224] Handling call
0709 12:26:03.085981 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.350635 (+264654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.634476 (+1283841us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54004c6b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:04.684400 (+ 49924us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493025152}
I20260709 12:26:04.739248  4677 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
W20260709 12:26:04.740940  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 422858us
I20260709 12:26:04.741721  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1725) took 1026 ms. Trace:
I20260709 12:26:04.741545  4569 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:04.742157  4679 rpcz_store.cc:276] 0709 12:26:03.715049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.682711 (+967662us) service_pool.cc:224] Handling call
0709 12:26:04.740825 (+ 58114us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:04.701049  7945 consensus_meta.cc:287] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.014s
I20260709 12:26:04.763455  4720 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:04.446422  8249 consensus_queue.cc:237] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:04.696362  8279 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:04.695241  8105 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:26:04.703410  8280 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:04.702347  8265 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:04.749825  8035 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:04.791730  8266 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.783334  7796 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:04.792340  8266 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.792599  7796 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:04.793671  8279 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:04.794643  8279 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:04.796887  7886 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.746943  8259 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:04.603130  7863 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:04.750717  8272 log.cc:927] Time spent T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.319s	user 0.005s	sys 0.002s
I20260709 12:26:04.790614  8052 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:04.774794  8288 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:04.799151  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.818307  4679 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 12:26:04.845192  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 81874.1us
I20260709 12:26:04.854825  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:04.766883  8247 consensus_queue.cc:1059] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 12:26:04.757894  7945 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:04.817034  8260 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:04.824559  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:04.817307  8259 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:04.844868  8213 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:04.857672  8211 consensus_meta.cc:287] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.973s	user 0.010s	sys 0.002s
I20260709 12:26:04.875746  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:04.878566  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 54105.7us
I20260709 12:26:04.822791  8038 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:26:04.779685  8284 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 12:26:04.817945  8035 raft_consensus.cc:697] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:04.858712  8108 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:04.820134  7796 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:04.887377  4571 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:04.879823  8259 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:04.985598  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 109913us
I20260709 12:26:04.853691  8265 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.885149  8288 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:04.999724  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 112469us
I20260709 12:26:04.895444  8269 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:04.985124  8195 raft_consensus.cc:3055] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:05.010573  4674 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:05.009649  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:04.987943  8011 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:04.988871  8260 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:05.006145  8213 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:05.014973  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:04.884832  8298 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.128s
I20260709 12:26:04.889866  8211 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:04.987321  8108 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.016162  7981 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:05.008605  8195 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:05.034806  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:05.048305  4755 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:05.045424  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:05.049531  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:05.054900  8003 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:05.054761  8038 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:26:05.026093  8293 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:26:05.056419  8105 consensus_queue.cc:1059] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:05.057806  8269 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:05.058233  8269 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.058291  7945 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:05.058699  8269 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:05.059847  8286 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
W20260709 12:26:05.062023  8302 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:26:05.062124  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.008590  8280 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.063439  8280 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:05.065742  7981 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:05.065932  8035 consensus_queue.cc:237] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.066769  7981 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:05.072248  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.076303  8265 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:05.029433  8305 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:26:05.077986  5347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 557) took 1867 ms. Trace:
I20260709 12:26:05.085659  4599 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:05.033896  8288 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:05.086686  8288 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:05.087078  8288 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.095085  8238 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:05.095600  8238 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:05.100692  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:05.101015  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:05.101675  4750 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:05.102579  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:05.103102  4750 raft_consensus.cc:2376] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7. Re-sending same reply.
I20260709 12:26:05.076884  8260 raft_consensus.cc:2764] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:26:05.104326  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:05.104220  8195 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.105479  8195 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:05.115258  8019 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.119127  8019 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.107040  8211 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:05.112007  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.116739  8265 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:26:05.122478  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1702) took 4565 ms. Trace:
I20260709 12:26:05.122583  8257 raft_consensus.cc:3055] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:05.123152  8257 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:05.123047  4603 rpcz_store.cc:276] 0709 12:26:00.557448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.557591 (+   143us) service_pool.cc:224] Handling call
0709 12:26:00.581429 (+ 23838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.340754 (+3759325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.122141 (+781387us) spinlock_profiling.cc:243] Waited 3.76 s on lock 0x7b5400301954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:05.122377 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8643362688}
I20260709 12:26:05.124202  8257 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.078382  5347 rpcz_store.cc:276] 0709 12:26:03.210147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.210315 (+   168us) service_pool.cc:224] Handling call
0709 12:26:03.248520 (+ 38205us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.256909 (+  8389us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:04.230836 (+973927us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b180008e4e0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c6455 00007efff1eae3c7 00007effee4c09a7 00007effee4d00bf 00007effee4d0038
0709 12:26:04.390691 (+159855us) log.cc:844] Serialized 14 byte log entry
0709 12:26:04.585604 (+194913us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.711408 (+125804us) log.cc:844] Serialized 26 byte log entry
0709 12:26:04.712521 (+  1113us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:04.712673 (+   152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.712871 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.077689 (+364818us) raft_consensus.cc:1685] finished
0709 12:26:05.077791 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.077921 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9407,"prepare.queue_time_us":1291306,"prepare.run_cpu_time_us":6578,"prepare.run_wall_time_us":88527,"raft.queue_time_us":260300,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":31865,"spinlock_wait_cycles":11356416,"thread_start_us":321181,"threads_started":2,"wal-append.queue_time_us":378712}
I20260709 12:26:05.126349  8211 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.126857  8211 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.125506  8257 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.116140  4645 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:05.130143  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.131563  8265 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:05.131563  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.132153  8265 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.132546  4679 raft_consensus.cc:2410] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:05.134122  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.134776  4679 raft_consensus.cc:2376] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10. Re-sending same reply.
I20260709 12:26:05.139882  8229 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:05.140545  8229 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:05.132507  8265 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.151752  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 571) took 1177 ms. Trace:
I20260709 12:26:05.152196  4748 rpcz_store.cc:276] 0709 12:26:03.974823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.065521 (+ 90698us) service_pool.cc:224] Handling call
0709 12:26:05.151595 (+1086074us) spinlock_profiling.cc:243] Waited 60.2 ms on lock 0x7b540030a054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:05.151694 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138429184}
I20260709 12:26:05.154173  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1726) took 1438 ms. Trace:
I20260709 12:26:05.154567  4678 rpcz_store.cc:276] 0709 12:26:03.715311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.715440 (+   129us) service_pool.cc:224] Handling call
0709 12:26:05.153957 (+1438517us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54000b18d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:05.154093 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1589439232}
I20260709 12:26:05.134796  8115 raft_consensus.cc:3055] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:05.156011  8115 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:05.153430  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:05.158350  4748 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:26:05.159124  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2695) took 1178 ms. Trace:
I20260709 12:26:05.159633  4680 rpcz_store.cc:276] 0709 12:26:03.980608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.258622 (+278014us) service_pool.cc:224] Handling call
0709 12:26:04.259022 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.707934 (+448912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.113439 (+405505us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54000b1dd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:05.159045 (+ 45606us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":526605056}
I20260709 12:26:05.158695  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:05.132584  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.105147  4750 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:05.177248  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:05.179960  8228 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.180694  8228 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:05.184180  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.203629  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.205382  8260 consensus_queue.cc:1059] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:26:05.217641  4601 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:05.217775  8260 consensus_queue.cc:1243] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:05.219172  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.203018  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.225888  4605 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:26:05.226981  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.227958  4605 raft_consensus.cc:2393] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:05.229808  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:05.207755  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.239289  4597 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:05.240085  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:05.240422  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:05.201960  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.222632  8038 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.223227  8120 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.202513  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:05.171568  5312 connection.cc:256] RPC call timeout handler was delayed by -1.3897367350000422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
W20260709 12:26:05.255537  5312 connection.cc:256] RPC call timeout handler was delayed by -1.1213220879999426s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1683s for a response
I20260709 12:26:05.256240  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:05.197113  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:05.258006  4750 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:05.225222  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.243106  4605 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.157121  8294 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.182152  7945 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:05.290198  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:05.237128  8062 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.314538  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:05.316228  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.318377  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.288653  8019 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:05.263394  8245 consensus_meta.cc:287] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.865s	user 0.016s	sys 0.001s
I20260709 12:26:05.325889  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:05.325830  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.238346  7840 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:05.340418  4755 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:26:05.156900  8115 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.288173  8279 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:05.287706  7945 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.310917  8213 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:05.365049  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.369867  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:05.370635  4755 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:05.371050  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:05.372525  8290 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:05.253826  8038 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.287446  8011 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:26:05.377400  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.380095  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2173) took 1171 ms. Trace:
I20260709 12:26:05.402376  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.347774  7945 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:05.322095  8062 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.345621  8279 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:05.379176  8218 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 1.877s	user 0.006s	sys 0.014s
I20260709 12:26:05.286950  8120 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.407157  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:05.368950  8282 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:05.391387  4602 rpcz_store.cc:276] 0709 12:26:04.208812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.209014 (+   202us) service_pool.cc:224] Handling call
0709 12:26:04.209440 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.314863 (+105423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.314894 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.326436 (+ 11542us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.149215 (+822779us) spinlock_profiling.cc:243] Waited 8.59 ms on lock 0x7b18003753a0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:26:05.372310 (+223095us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:05.372355 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.373171 (+   816us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.375328 (+  2157us) raft_consensus.cc:1685] finished
0709 12:26:05.375344 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.380021 (+  4677us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52750,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":10212,"spinlock_wait_cycles":19766144,"thread_start_us":222911,"threads_started":1,"wal-append.queue_time_us":1045845}
I20260709 12:26:05.424791  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:05.414106  4596 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:05.421301  4601 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:05.379784  8115 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:05.293064  8294 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.337674  8245 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:05.405969  8250 consensus_meta.cc:287] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.688s	user 0.011s	sys 0.002s
I20260709 12:26:05.421797  4755 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:26:05.436790  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.437598  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:05.411904  5342 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.034s	sys 0.038s
I20260709 12:26:05.447213  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.407829  8211 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:05.421082  8279 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.454133  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:05.455307  4755 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:05.456027  4602 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:05.461516  4602 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:05.462498  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.424096  8282 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:26:05.453238  5342 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.035s	sys 0.039s
I20260709 12:26:05.478122  4596 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:26:05.482034  4602 raft_consensus.cc:3055] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:05.454674  8250 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11.
W20260709 12:26:05.475930  8273 consensus_meta.cc:287] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.977s	user 0.008s	sys 0.006s
I20260709 12:26:05.424309  8218 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.489367  4601 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:05.479405  7787 consensus_meta.cc:287] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 1.127s	user 0.009s	sys 0.006s
I20260709 12:26:05.510516  4596 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:05.464237  7840 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.496984  4597 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:05.500887  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:05.483230  4602 raft_consensus.cc:740] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:05.473465  8282 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.469856  8298 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:05.521500  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.506600  8062 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:05.495414  4755 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
W20260709 12:26:05.515584  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 162615us
I20260709 12:26:05.476814  8108 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:05.453186  8269 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.520601  4600 raft_consensus.cc:2410] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:05.524210  4750 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:05.488087  8228 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:05.475033  7981 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:26:05.529757  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:05.544701  4601 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:26:05.545498  5314 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:05.552194  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:05.553301  4750 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:05.553540  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.550882  8108 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:05.567442  8108 raft_consensus.cc:697] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:05.558517  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:05.569774  4755 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:05.568377  8245 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:05.571022  8108 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.484884  8259 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.574122  8120 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:05.576201  8038 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:05.576444  8212 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:05.557201  8228 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.579121  4604 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:05.579500  4645 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:05.581270  8003 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.556352  8106 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:26:05.581813  8003 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.583070  8081 raft_consensus.cc:3055] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:05.583501  8081 raft_consensus.cc:740] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:05.577603  7981 raft_consensus.cc:3055] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:05.584955  4750 raft_consensus.cc:2410] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:05.557410  4754 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:05.585237  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.530753  8293 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.578245  8228 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.511101  8273 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.591609  4679 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
W20260709 12:26:05.606907  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.553076  8284 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:05.512395  8210 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:26:05.609117  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:05.592710  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:05.614905  8019 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.616555  4755 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:05.525702  7840 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:05.593883  8288 consensus_meta.cc:287] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.506s	user 0.005s	sys 0.010s
I20260709 12:26:05.529995  8298 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.591957  8232 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:05.623852  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:05.623402  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:05.641739  8228 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:05.643656  5342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 505) took 1302 ms. Trace:
I20260709 12:26:05.644080  7840 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:05.644392  4605 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:05.646087  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:05.590811  7981 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:05.650782  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:05.650172  8288 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:05.662845  8310 log.cc:927] Time spent T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:26:05.663609  7981 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.664353  5341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 504) took 1323 ms. Trace:
I20260709 12:26:05.611374  8284 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.664650  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:05.665222  8284 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:05.665797  4750 raft_consensus.cc:2410] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:26:05.666857  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:05.668969  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:05.670295  4680 raft_consensus.cc:2410] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:05.667500  4750 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:05.680634  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:05.644730  8019 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.635628  8003 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:05.608047  8293 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.614801  8081 consensus_queue.cc:260] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:05.658818  8257 consensus_meta.cc:287] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.530s	user 0.016s	sys 0.000s
I20260709 12:26:05.684392  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:05.682719  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 153044us
I20260709 12:26:05.646613  8232 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.695354  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:05.695926  4680 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:05.684532  7981 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.702829  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.703593  4680 raft_consensus.cc:2410] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:05.697748  8081 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:05.694697  8327 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:05.709295  8309 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:05.711248  8219 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:05.712131  7945 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.714175  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.745851  4678 raft_consensus.cc:2393] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:05.746661  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.714632  4602 consensus_queue.cc:260] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.717905  8218 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:05.719010  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2166) took 1586 ms. Trace:
I20260709 12:26:05.696461  8232 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:05.732864  8019 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:05.737686  8259 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:05.742637  8294 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:05.742964  8290 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:05.743211  8269 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:05.791769  4600 rpcz_store.cc:276] 0709 12:26:04.132505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.132641 (+   136us) service_pool.cc:224] Handling call
0709 12:26:05.718763 (+1586122us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54002e54d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:05.718943 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3165961088}
W20260709 12:26:05.768661  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.795238  4602 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.730262  8327 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.727701  8273 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:05.817958  4602 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:05.804160  8269 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.818743  8269 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.821039  8269 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:05.821626  8269 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:26:05.824148  8238 consensus_meta.cc:287] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.728s	user 0.010s	sys 0.002s
W20260709 12:26:05.826445  8195 consensus_meta.cc:287] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.708s	user 0.013s	sys 0.001s
I20260709 12:26:05.827705  4756 raft_consensus.cc:2410] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:05.803644  8282 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.799216  8290 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:05.808532  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.788758  8288 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:05.816069  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.708658  8328 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:05.664839  5341 rpcz_store.cc:276] 0709 12:26:04.340485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.341018 (+   533us) service_pool.cc:224] Handling call
0709 12:26:04.360646 (+ 19628us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.757866 (+397220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.764428 (+  6562us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.857023 (+ 92595us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.324091 (+467068us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:05.329344 (+  5253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.345725 (+ 16381us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.664073 (+318348us) raft_consensus.cc:1685] finished
0709 12:26:05.664102 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.664281 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":365901,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":13153,"thread_start_us":684766,"threads_started":2,"wal-append.queue_time_us":606807}
W20260709 12:26:05.796623  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.730552  8298 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:05.836037  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 109689us
I20260709 12:26:05.824851  8238 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:05.835136  8280 consensus_meta.cc:287] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.771s	user 0.010s	sys 0.005s
I20260709 12:26:05.830739  8327 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.846701  7945 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:05.851001  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.851610  4748 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:05.838841  8328 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:05.852931  8328 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:05.853668  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2699) took 1798 ms. Trace:
W20260709 12:26:05.853938  8229 consensus_meta.cc:287] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.713s	user 0.008s	sys 0.006s
I20260709 12:26:05.854116  4681 rpcz_store.cc:276] 0709 12:26:04.055280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.325838 (+270558us) service_pool.cc:224] Handling call
0709 12:26:04.326236 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.426582 (+100346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.853337 (+1426755us) spinlock_profiling.cc:243] Waited 92.5 ms on lock 0x7b54001d29d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:05.853589 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212790656}
I20260709 12:26:05.854478  8229 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:05.855530  8224 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:05.855708  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:05.856550  4681 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:05.856777  8145 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:05.857079  8229 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:05.858043  8266 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:05.858433  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:05.859838  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:05.860414  4753 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:05.861346  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:05.862866  4753 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:05.644229  5342 rpcz_store.cc:276] 0709 12:26:04.341510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.345687 (+  4177us) service_pool.cc:224] Handling call
0709 12:26:04.391565 (+ 45878us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.467861 (+ 76296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.476445 (+  8584us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.712161 (+235716us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.487733 (+775572us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:05.503887 (+ 16154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.512750 (+  8863us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.643261 (+130511us) raft_consensus.cc:1685] finished
0709 12:26:05.643290 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.643421 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":374077,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":136,"thread_start_us":721303,"threads_started":2,"wal-append.queue_time_us":765903}
I20260709 12:26:05.864128  8309 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:05.864918  8309 consensus_queue.cc:1243] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:05.865034  5314 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:05.866429  5314 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:05.867666  8298 raft_consensus.cc:3055] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:05.868101  4678 raft_consensus.cc:2368] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11.
I20260709 12:26:05.868240  8298 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:05.869189  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.869885  4678 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:26:05.870183  7863 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:05.870414  7945 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:05.867759  8273 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:05.854483  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:05.871758  4747 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:05.872205  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:05.875803  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.857414  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.877490  4681 raft_consensus.cc:2410] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:26:05.866463  8180 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
W20260709 12:26:05.880239  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.881985  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2696) took 1862 ms. Trace:
I20260709 12:26:05.883754  8232 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.883065  4673 rpcz_store.cc:276] 0709 12:26:04.020131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.307932 (+287801us) service_pool.cc:224] Handling call
0709 12:26:04.308391 (+   459us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:04.387213 (+ 78822us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:04.765954 (+378741us) log.cc:844] Serialized 14 byte log entry
0709 12:26:04.998840 (+232886us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:04.999422 (+   582us) log.cc:844] Serialized 52 byte log entry
0709 12:26:04.999549 (+   127us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:04.999580 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.881530 (+881950us) spinlock_profiling.cc:243] Waited 53.3 ms on lock 0x7b54001bd354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:05.881609 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.881676 (+    67us) raft_consensus.cc:1685] finished
0709 12:26:05.881779 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.881924 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15329,"prepare.queue_time_us":74655,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":24730,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":11336,"spinlock_wait_cycles":122595584,"thread_start_us":501809,"threads_started":2,"wal-append.queue_time_us":184983}
I20260709 12:26:05.885461  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.886540  4673 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:05.887552  8180 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:05.887712  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2692) took 2104 ms. Trace:
I20260709 12:26:05.874300  7945 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:05.888157  8180 raft_consensus.cc:697] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:05.888561  7945 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.889299  8180 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.881976  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:05.891379  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:05.886587  8232 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:05.887753  4571 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:05.891499  4755 raft_consensus.cc:2410] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:26:05.892189  4681 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:05.894145  4747 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 5.
I20260709 12:26:05.897722  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.898600  4673 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:05.899794  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:05.913368  8281 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.317s	user 0.008s	sys 0.004s
W20260709 12:26:05.916443  8281 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.363s	user 0.009s	sys 0.004s
I20260709 12:26:05.896940  5314 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:05.888275  4676 rpcz_store.cc:276] 0709 12:26:03.783625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.929174 (+145549us) service_pool.cc:224] Handling call
0709 12:26:03.946273 (+ 17099us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.995966 (+ 49693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.996000 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.996511 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:26:04.056359 (+ 59848us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:04.358324 (+301965us) log.cc:844] Serialized 14 byte log entry
0709 12:26:04.358513 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.887290 (+1528777us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b5400200654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:05.887348 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.887403 (+    55us) raft_consensus.cc:1685] finished
0709 12:26:05.887494 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.887648 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1918,"prepare.queue_time_us":356787,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":3925,"raft.queue_time_us":5139,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":24942,"spinlock_wait_cycles":113336704,"thread_start_us":283042,"threads_started":2,"wal-append.queue_time_us":173180,"wal-append.run_cpu_time_us":4181,"wal-append.run_wall_time_us":1284222}
W20260709 12:26:05.934760  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 72557.9us
I20260709 12:26:05.937952  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.938365  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.939034  4605 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:05.938023  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.941222  4747 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:05.941933  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 574) took 1255 ms. Trace:
W20260709 12:26:05.942354  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 54678.8us
I20260709 12:26:05.944167  4604 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:05.944669  4571 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:05.942343  4747 rpcz_store.cc:276] 0709 12:26:04.686711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.906215 (+1219504us) service_pool.cc:224] Handling call
0709 12:26:05.941033 (+ 34818us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.941690 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.941874 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:05.946172  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.942905  4645 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:05.949954  4676 raft_consensus.cc:2393] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:05.950896  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.869309  8298 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:05.931373  8280 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:05.951968  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.940272  8281 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:05.961658  4597 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:05.958472  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:05.961828  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.959168  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2156) took 1225 ms. Trace:
I20260709 12:26:05.964172  4747 rpcz_store.cc:276] 0709 12:26:04.724457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.948747 (+1224290us) service_pool.cc:224] Handling call
0709 12:26:05.949031 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.949206 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.949235 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:05.949279 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:05.949322 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.949376 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.949471 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:05.953439  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:05.962792  4678 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:05.971671  8081 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:26:05.975870  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:05.976792  4605 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:05.947804  8018 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:05.978570  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.982692  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:05.983886  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.986843  4596 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
W20260709 12:26:05.987588  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 44770.6us
I20260709 12:26:05.987337  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.991343  4676 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:05.975917  8242 raft_consensus.cc:3055] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:06.003490  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.999927  8018 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.989248  8129 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:06.000689  8081 consensus_queue.cc:1243] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:05.975903  8282 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:06.006702  8242 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:06.009117  8129 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:06.009868  8242 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.010787  4601 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:06.007786  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:05.985064  8261 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:05.984946  8333 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:26:05.953213  8273 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:26:05.985383  8281 raft_consensus.cc:697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:06.038018  8281 consensus_queue.cc:237] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.012696  4678 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:06.000437  8062 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:06.025354  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:06.041409  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 67104.1us
I20260709 12:26:05.987805  8298 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:06.007645  8018 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:06.042618  4672 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:06.008133  4676 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:05.972448  8280 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.000619  8221 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:06.043337  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 60788.9us
I20260709 12:26:06.060871  4647 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:06.061285  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.062232  8229 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:06.041833  8062 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:06.067698  8129 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:06.068006  8062 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:06.060202  8221 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:06.068151  8265 consensus_meta.cc:287] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.924s	user 0.020s	sys 0.010s
I20260709 12:26:06.068845  8221 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:06.069103  8265 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.064172  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:06.082794  4647 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:06.085067  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:06.086647  4753 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:06.088922  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:06.090970  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 129147us
I20260709 12:26:06.099831  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:06.105213  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.112619  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:06.113806  4753 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:06.139523  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2154) took 1680 ms. Trace:
W20260709 12:26:06.145118  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 100448us
I20260709 12:26:06.145627  8224 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:06.156587  4681 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:06.148554  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:06.161945  4679 raft_consensus.cc:2368] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8.
W20260709 12:26:06.162629  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 100525us
I20260709 12:26:06.163657  4645 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:06.167243  4681 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:06.168910  4681 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:06.170210  4681 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:06.170506  8333 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.171224  8333 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.090417  8238 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:06.093137  8273 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:06.093129  8229 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:06.091300  8261 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:06.184628  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 84899.3us
I20260709 12:26:06.192679  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:06.191149  4645 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:06.187976  4750 rpcz_store.cc:276] 0709 12:26:04.458955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.685302 (+1226347us) service_pool.cc:224] Handling call
0709 12:26:05.685773 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.686066 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:05.837802 (+151736us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.100393 (+262591us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.138973 (+ 38580us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:06.139092 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.139214 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.139419 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3383,"prepare.queue_time_us":197247,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":27806,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":87,"thread_start_us":397007,"threads_started":3,"wal-append.queue_time_us":231497}
I20260709 12:26:06.197227  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:06.060616  8253 consensus_queue.cc:1059] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 12:26:06.203161  4673 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:06.204365  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48198 (request call id 532) took 1009 ms. Trace:
I20260709 12:26:06.205506  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:06.205502  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:06.209025  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:06.209903  4753 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:26:06.210281  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:06.207389  4673 rpcz_store.cc:276] 0709 12:26:05.194744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.899486 (+704742us) service_pool.cc:224] Handling call
0709 12:26:06.204290 (+304804us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:06.184329  8289 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.186904  8129 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:06.226994  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.205999  8157 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:06.238241  8337 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:26:06.238775  8253 raft_consensus.cc:3055] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:06.204495  8003 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:06.240797  4673 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:06.263517  4673 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
W20260709 12:26:06.271972  4754 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2155) took 1547 ms (client timeout 1746 ms). Trace:
W20260709 12:26:06.272529  4754 rpcz_store.cc:269] 0709 12:26:04.724159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.100415 (+376256us) service_pool.cc:224] Handling call
0709 12:26:06.271679 (+1171264us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b54001f1154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:06.271867 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1048353664}
I20260709 12:26:06.274745  4677 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:06.276355  4677 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:06.252084  8253 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:06.283692  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.204182  7945 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.289644  4677 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:06.206682  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:06.206141  8273 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:06.258823  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:06.212258  8224 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:06.240051  8266 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:06.292953  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:06.261128  8333 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:06.249531  8129 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.246088  8289 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.304301  8283 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:06.305410  8328 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.262849  8344 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:06.264703  8229 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:06.306958  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 101552us
W20260709 12:26:06.308380  4599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2169) took 2123 ms (client timeout 1722 ms). Trace:
W20260709 12:26:06.390973  4599 rpcz_store.cc:269] 0709 12:26:04.175773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.175917 (+   144us) service_pool.cc:224] Handling call
0709 12:26:06.298999 (+2123082us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b5400300a54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:06.299154 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2060189440}
I20260709 12:26:06.252436  8157 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.310917  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:06.311160  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:06.312490  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:06.313088  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:06.313376  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:06.314373  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:06.316393  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.324334  7945 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:06.301088  8219 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:06.337698  8250 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:06.351495  4677 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:26:06.318349  8287 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:06.417810  8287 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.419101  8328 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:06.377811  8224 raft_consensus.cc:697] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:06.420202  8287 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:06.345556  7981 consensus_meta.cc:287] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.638s	user 0.006s	sys 0.006s
I20260709 12:26:06.420567  8224 consensus_queue.cc:237] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:06.347497  8290 consensus_meta.cc:287] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.000s
I20260709 12:26:06.422426  8290 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:06.425077  8290 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:06.384214  8232 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:06.361608  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.359875  4750 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:06.371835  4754 raft_consensus.cc:2393] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:06.466061  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:06.467010  4754 raft_consensus.cc:2410] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:06.376310  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:06.350759  8195 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.353477  8265 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:06.368916  8280 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:06.411298  5345 raft_consensus.cc:2393] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:06.376935  5342 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:06.376478  8266 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:06.413805  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:06.416630  8018 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.417068  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:06.426219  8229 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.369841  8253 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.430119  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2700) took 2117 ms. Trace:
I20260709 12:26:06.432240  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:06.506775  4674 rpcz_store.cc:276] 0709 12:26:04.312664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.355818 (+ 43154us) service_pool.cc:224] Handling call
0709 12:26:04.368399 (+ 12581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.011102 (+642703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.429843 (+1418741us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b540007c454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:06.430047 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476250368}
I20260709 12:26:06.432368  8062 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.433853  7863 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.434348  8344 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.518141  4750 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:06.441159  4599 raft_consensus.cc:1217] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Deduplicated request from leader. Original: 9.7->[10.8-10.8]   Dedup: 10.8->[]
I20260709 12:26:06.441365  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:06.519068  4599 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:26:06.442282  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:06.519538  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:06.443470  8221 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.445248  5341 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:26:06.446501  5340 raft_consensus.cc:2410] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:26:06.448414  8289 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:06.536463  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:06.536772  8266 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.537760  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:06.539116  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:06.539351  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:06.539682  5341 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:06.540693  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:06.453922  8279 consensus_meta.cc:287] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.979s	user 0.010s	sys 0.002s
I20260709 12:26:06.454432  8273 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.456813  8129 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:06.459892  8284 consensus_meta.cc:287] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.794s	user 0.011s	sys 0.001s
I20260709 12:26:06.460870  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:06.525446  8253 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:06.464318  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:06.391711  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.472005  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.495582  4719 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:06.552587  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 57106.8us
I20260709 12:26:06.553941  4719 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:06.498909  4601 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:26:06.501423  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:06.503183  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:06.503641  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2707) took 1352 ms. Trace:
W20260709 12:26:06.469476  8327 consensus_meta.cc:287] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.599s	user 0.000s	sys 0.018s
I20260709 12:26:06.435316  8333 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:06.519372  4600 raft_consensus.cc:2410] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:26:06.521451  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:06.522617  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:06.523429  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:06.524116  8317 log.cc:927] Time spent T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.945s	user 0.005s	sys 0.023s
I20260709 12:26:06.530303  8344 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.532544  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:06.502202  8246 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.541007  4644 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:06.474748  8195 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.426254  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:06.546849  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:06.557873  4754 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:06.520586  7863 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:06.586104  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 45198.5us
I20260709 12:26:06.559734  4603 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:06.587999  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:06.588737  4720 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.513s (SENT) [suppressed 27 similar messages]
I20260709 12:26:06.558101  4676 rpcz_store.cc:276] 0709 12:26:05.151237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.043828 (+892591us) service_pool.cc:224] Handling call
0709 12:26:06.045089 (+  1261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:06.060672 (+ 15583us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:06.082646 (+ 21974us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.499785 (+417139us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.503104 (+  3319us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.503239 (+   135us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.503281 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:06.503346 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.503432 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.503576 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12713,"prepare.queue_time_us":25436,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":3129,"raft.queue_time_us":81197,"raft.run_cpu_time_us":425,"raft.run_wall_time_us":31857,"spinlock_wait_cycles":49876736,"thread_start_us":408091,"threads_started":2,"wal-append.queue_time_us":242194}
I20260709 12:26:06.487943  8232 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.561306  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2708) took 1400 ms. Trace:
I20260709 12:26:06.498799  8003 raft_consensus.cc:697] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:06.558254  8019 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:06.566695  8327 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.566532  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:06.568533  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.559990  8269 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:06.596933  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:06.560618  8309 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:06.595999  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:06.574496  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:06.601922  4754 raft_consensus.cc:2410] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:26:06.602320  4719 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:06.603897  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:06.604692  4751 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:06.543973  8279 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.591459  4672 rpcz_store.cc:276] 0709 12:26:05.151473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.066474 (+915001us) service_pool.cc:224] Handling call
0709 12:26:06.066783 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:06.066953 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:06.238743 (+171790us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.551307 (+312564us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:06.551383 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:06.551446 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.551638 (+   192us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.551800 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19122,"prepare.queue_time_us":77447,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":112,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":7366,"thread_start_us":449983,"threads_started":2,"wal-append.queue_time_us":344313}
I20260709 12:26:06.580919  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:06.579838  8298 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:06.608829  4750 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:06.552496  8284 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.594846  8265 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:06.617789  8298 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.619400  8284 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:06.620503  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.621407  4753 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:06.577966  8280 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:26:06.594918  8289 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:06.623698  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:06.624433  4603 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:06.600387  7975 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:06.578192  8328 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:26:06.560400  8129 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 10.8, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:26:06.582026  8333 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.628997  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:06.627254  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:06.638105  4644 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:06.640192  5343 raft_consensus.cc:1240] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:26:06.609395  8350 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:06.624959  8298 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:06.607805  8283 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:06.589880  8246 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.658679  4751 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:06.672283  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 462075us
I20260709 12:26:06.693749  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:06.686748  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.673262  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 482223us
I20260709 12:26:06.626703  8289 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.635191  7975 raft_consensus.cc:697] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:06.699941  7975 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.700433  7863 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:06.703383  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:06.704133  4600 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:06.706789  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1743) took 1233 ms. Trace:
W20260709 12:26:06.707154  4756 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2658) took 1639 ms (client timeout 1676 ms). Trace:
I20260709 12:26:06.707368  4681 rpcz_store.cc:276] 0709 12:26:05.473899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.170952 (+697053us) service_pool.cc:224] Handling call
0709 12:26:06.171306 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:06.191113 (+ 19807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.196999 (+  5886us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:06.198384 (+  1385us) log.cc:844] Serialized 52 byte log entry
0709 12:26:06.597348 (+398964us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:06.597406 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.597494 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.706474 (+108980us) raft_consensus.cc:1685] finished
0709 12:26:06.706506 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.706706 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13625,"raft.queue_time_us":18531,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":343182,"threads_started":1,"wal-append.queue_time_us":462150}
I20260709 12:26:06.706391  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:06.707883  4756 rpcz_store.cc:269] 0709 12:26:05.067989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.239719 (+171730us) service_pool.cc:224] Handling call
0709 12:26:06.696232 (+1456513us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54000bf9d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:06.707065 (+ 10833us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1348469120}
I20260709 12:26:06.643415  8265 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:06.709110  5342 raft_consensus.cc:2393] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:06.718467  4645 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:06.690308  8129 consensus_queue.cc:1243] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:26:06.762980  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 44609.4us
I20260709 12:26:06.708756  8289 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:06.764853  4645 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:06.771085  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:06.772475  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:06.732044  4603 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:06.715888  8335 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:06.727545  8358 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:06.783736  4720 leader_election.cc:336] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.721s (SENT)
I20260709 12:26:06.800734  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:06.730252  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:06.801604  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:06.765319  8062 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:06.772751  8265 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:06.726576  7863 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:06.775262  8221 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:06.766323  8357 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:06.784809  8328 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.821098  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:06.803776  8235 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.818s	user 0.022s	sys 0.053s
I20260709 12:26:06.769423  8250 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:26:06.808835  8129 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:06.805419  8355 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:06.774266  8359 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:26:06.810626  8238 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:06.809158  8265 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:06.829885  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 121238us
I20260709 12:26:06.883766  8265 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:06.884670  4571 leader_election.cc:336] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.676s (SENT)
I20260709 12:26:06.834287  4647 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:06.887328  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 53118.1us
I20260709 12:26:06.889304  8224 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:06.870440  8333 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:06.889942  8224 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.890622  8224 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:06.846503  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.888481  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:06.855051  4720 leader_election.cc:336] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.799s (SENT)
I20260709 12:26:06.899130  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:06.900272  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 45260.2us
I20260709 12:26:06.812507  7863 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:06.832818  8235 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.854s	user 0.022s	sys 0.060s
I20260709 12:26:06.835588  8221 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:06.907248  8250 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:06.840804  8356 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:06.824354  8357 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:06.863940  8246 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:06.830349  8328 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.825007  8266 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:06.804358  8358 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.905653  8333 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:06.870682  8232 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:06.784703  8335 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.879944  5348 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:06.928570  5348 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 12:26:06.882200  8346 log.cc:927] Time spent T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.557s	user 0.003s	sys 0.000s
I20260709 12:26:06.896888  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:06.909286  8250 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.931056  5345 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 5.
W20260709 12:26:06.898151  4755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2164) took 1723 ms (client timeout 2165 ms). Trace:
W20260709 12:26:06.828786  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 64036.3us
I20260709 12:26:06.905380  4751 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:06.802731  8003 consensus_queue.cc:237] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.907266  4675 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:06.907450  8221 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.906296  7863 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:06.866075  8348 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:26:06.922963  8114 raft_consensus.cc:3055] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:06.942013  8114 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:06.943008  8114 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:06.911177  8249 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.923921  8328 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:06.924243  8358 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:06.913638  8357 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:06.914953  8279 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:06.925638  8081 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:06.939936  8250 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:06.933007  4755 rpcz_store.cc:269] 0709 12:26:05.174571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.619490 (+444919us) service_pool.cc:224] Handling call
0709 12:26:06.897924 (+1278434us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54001b1054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:06.898087 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":525655808}
I20260709 12:26:06.934041  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:06.938247  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:06.915179  8327 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:06.940815  5348 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:06.999223  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 65248.6us
W20260709 12:26:06.954399  4720 leader_election.cc:336] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.899s (SENT)
I20260709 12:26:06.904861  8355 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.928813  8195 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:06.940035  8221 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:06.944047  8114 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:07.000656  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 46285.9us
I20260709 12:26:07.012265  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:07.015152  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2706) took 1864 ms. Trace:
W20260709 12:26:07.017550  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.006840  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:07.012265  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:07.029471  4751 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:07.030728  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:07.032915  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:07.033921  4755 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:06.956873  8081 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:07.037169  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:07.038036  4600 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:07.015738  4678 rpcz_store.cc:276] 0709 12:26:05.151008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.040127 (+889119us) service_pool.cc:224] Handling call
0709 12:26:06.062720 (+ 22593us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:06.083288 (+ 20568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.083338 (+    50us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:06.092216 (+  8878us) log.cc:844] Serialized 78 byte log entry
0709 12:26:06.379453 (+287237us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:06.379516 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.379627 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.014784 (+635157us) raft_consensus.cc:1685] finished
0709 12:26:07.014816 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.015066 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14700,"raft.queue_time_us":10351,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":6706,"thread_start_us":202882,"threads_started":1,"wal-append.queue_time_us":436661}
I20260709 12:26:07.044306  4645 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.092017  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:07.012564  8358 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:07.096766  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:07.106498  4681 raft_consensus.cc:2393] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:07.107381  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:07.108064  8371 log.cc:927] Time spent T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:26:07.014400  8195 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.108748  8195 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.111167  8195 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:07.097450  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:07.092290  8333 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.113245  8333 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:07.013159  8346 log.cc:927] Time spent T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:26:07.115303  8312 log.cc:927] Time spent T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 1.547s	user 0.007s	sys 0.000s
I20260709 12:26:07.115763  8333 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:07.116936  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 539) took 1560 ms. Trace:
I20260709 12:26:07.104399  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.103094  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.102387  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.103734  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.103859  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.120883  4676 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:07.121786  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.107997  4681 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:07.105647  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:07.044900  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.122865  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 78700.2us
I20260709 12:26:07.102202  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:07.099781  4601 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:07.123656  4596 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:06.954795  8249 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.117482  4673 rpcz_store.cc:276] 0709 12:26:05.557005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.283763 (+726758us) service_pool.cc:224] Handling call
0709 12:26:06.284166 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.594093 (+309927us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.594136 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.609151 (+ 15015us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.834261 (+225110us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:06.834925 (+   664us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.835184 (+   259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.835264 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.116728 (+281464us) raft_consensus.cc:1685] finished
0709 12:26:07.116747 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.116880 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4957,"prepare.queue_time_us":239,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":132168,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":2916,"thread_start_us":446967,"threads_started":2,"wal-append.queue_time_us":308236}
I20260709 12:26:07.138411  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:07.139387  4603 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:07.041209  8260 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.140544  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:07.147789  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.137355  4751 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:07.149348  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.140961  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:07.142472  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.150250  4674 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:26:07.151631  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:07.152320  4673 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:07.154264  5312 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:07.144820  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.157120  4678 raft_consensus.cc:2368] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11.
I20260709 12:26:07.036602  8356 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.163635  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:07.164932  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:07.167659  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.168596  4681 raft_consensus.cc:2368] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8.
I20260709 12:26:07.169919  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:07.121476  8358 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.174058  4676 raft_consensus.cc:2393] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:07.174472  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.177804  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.178889  4674 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
W20260709 12:26:07.180014  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 46685.8us
I20260709 12:26:07.181502  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.182847  4676 raft_consensus.cc:2410] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 12:26:07.153733  5341 consensus_meta.cc:287] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.000s
I20260709 12:26:07.178864  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.190824  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:07.184665  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.192044  4751 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:07.192610  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2167) took 1211 ms. Trace:
I20260709 12:26:07.192916  4751 rpcz_store.cc:276] 0709 12:26:05.980909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.171166 (+1190257us) service_pool.cc:224] Handling call
0709 12:26:07.192551 (+ 21385us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:07.193192  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:07.173098  7835 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:07.173604  8281 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:26:07.173316  8129 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.147114  8283 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:07.195374  8129 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.195936  8283 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.198253  8283 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:07.198534  8350 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:07.199105  8350 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:07.202143  8366 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 12:26:07.202908  8374 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:26:07.204008  8309 consensus_meta.cc:287] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.596s	user 0.009s	sys 0.001s
W20260709 12:26:07.204386  8368 log.cc:927] Time spent T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:26:07.204512  8309 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:07.205986  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:07.206676  4751 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:07.201421  8350 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:07.115988  8081 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.210906  8081 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:07.211090  8362 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.341s	user 0.000s	sys 0.002s
I20260709 12:26:07.147094  8195 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.215343  8195 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.164491  8273 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:07.158564  8333 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.164677  8328 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:26:07.216727  4756 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:07.230587  4596 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:07.219329  8249 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:07.233716  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:07.227128  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:07.237784  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 59029.8us
I20260709 12:26:07.183553  8358 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:07.239856  4673 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:07.140344  8266 consensus_queue.cc:1059] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.084s
W20260709 12:26:07.246230  4679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2710) took 2063 ms (client timeout 1569 ms). Trace:
W20260709 12:26:07.246829  4679 rpcz_store.cc:269] 0709 12:26:05.174958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.608103 (+433145us) service_pool.cc:224] Handling call
0709 12:26:07.232275 (+1624172us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b540007b554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:07.238733 (+  6458us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1184059264}
I20260709 12:26:07.248677  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:07.224172  8375 log.cc:927] Time spent T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:26:07.239815  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:07.253719  4596 raft_consensus.cc:3055] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:07.253931  4679 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:07.255184  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:07.256882  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:07.219877  8327 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:07.265861  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.291860  4678 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:07.258826  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:07.292237  4645 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:07.245447  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:07.219694  8273 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:07.224473  8298 consensus_queue.cc:1175] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3a5aeedd11fe49938079fe86cb01232c" responder_term: 10 status { last_received { term: 10 index: 8 } last_committed_idx: 6 last_received_current_leader { term: 10 index: 8 } }
I20260709 12:26:07.166343  8260 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.222203  8289 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:07.254565  4751 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:07.288859  4596 raft_consensus.cc:740] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:07.177886  8356 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:07.222602  8333 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:07.290544  4647 leader_election.cc:336] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 2.165s (SENT)
I20260709 12:26:07.283877  4601 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:07.292990  4602 raft_consensus.cc:2393] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:07.191406  8269 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:26:07.219381  7863 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:06.878686  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:07.332190  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.407390  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.252437  8283 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:07.405671  4673 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:07.314551  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 75357.9us
I20260709 12:26:07.412907  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:07.353174  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.415740  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.383486  8345 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 1.060s	user 0.000s	sys 0.010s
I20260709 12:26:07.314006  8246 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:26:07.386652  8265 consensus_meta.cc:287] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.003s
I20260709 12:26:07.420925  4679 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:07.419142  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.294605  5341 raft_consensus.cc:2410] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:26:07.424886  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:07.439375  4751 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 5.
I20260709 12:26:07.440745  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:07.441481  4751 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 6.
I20260709 12:26:07.442447  4673 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:07.256078  8281 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:07.293713  8374 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:07.322636  8195 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:07.427414  7975 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.335095  8335 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:07.427414  8036 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:07.287011  7986 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:07.427124  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 234002us
I20260709 12:26:07.454109  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:07.455659  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.341348  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:07.456271  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.442577  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:07.457180  4673 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:26:07.439265  4596 consensus_queue.cc:260] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.339310  8221 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.458891  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:07.430202  8247 raft_consensus.cc:3055] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:07.452526  8374 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.430814  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:07.457130  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:07.457720  4751 raft_consensus.cc:2410] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:26:07.457917  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.464357  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2676) took 1288 ms. Trace:
I20260709 12:26:07.464475  4679 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:07.464723  4751 rpcz_store.cc:276] 0709 12:26:06.176025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.442275 (+1266250us) service_pool.cc:224] Handling call
0709 12:26:07.464300 (+ 22025us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:07.424782  8355 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:07.445997  8279 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:07.451704  5341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 560) took 1134 ms. Trace:
I20260709 12:26:07.408955  8273 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:07.460184  4596 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:07.465652  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:07.442940  8246 raft_consensus.cc:697] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:07.458287  7986 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.470278  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:07.438596  8265 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.473281  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.473769  4680 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:07.456776  8235 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:07.463721  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:07.466240  8374 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:07.462930  8036 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.500451  8036 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:07.501749  8285 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:07.502081  8336 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:07.460166  8335 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:07.514071  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:07.460098  8247 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:07.491575  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.493676  7986 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:07.458956  7975 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:07.495095  8265 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:07.484802  8279 raft_consensus.cc:697] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:07.472126  5341 rpcz_store.cc:276] 0709 12:26:06.315495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.538840 (+223345us) service_pool.cc:224] Handling call
0709 12:26:07.449763 (+910923us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:07.496229  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1743) took 1534 ms. Trace:
I20260709 12:26:07.495901  8357 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.496795  8283 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:07.502339  8285 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.550840  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:07.495915  8242 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:07.505430  4681 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:07.508018  8374 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:07.509266  4601 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:07.469434  7945 raft_consensus.cc:2764] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:26:07.486871  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:07.579602  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 92823.8us
I20260709 12:26:07.580850  4720 leader_election.cc:400] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11.
I20260709 12:26:07.581128  4720 leader_election.cc:403] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:07.469653  8195 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:07.545809  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 54357.9us
I20260709 12:26:07.551342  4598 rpcz_store.cc:276] 0709 12:26:06.040294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.044774 (+  4480us) service_pool.cc:224] Handling call
0709 12:26:06.066660 (+ 21886us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.067018 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:06.067308 (+   290us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.186510 (+119202us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.187654 (+  1144us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.209406 (+ 21752us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.209625 (+   219us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:06.209685 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.209797 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.428826 (+219029us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b18003d5fa0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:26:07.382847 (+954021us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:07.496005 (+113158us) raft_consensus.cc:1685] finished
0709 12:26:07.496029 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.496166 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":827,"prepare.queue_time_us":1054,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":643,"raft.queue_time_us":6817,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":15391,"spinlock_wait_cycles":68419072,"thread_start_us":118398,"threads_started":1,"wal-append.queue_time_us":255190}
I20260709 12:26:07.482177  8269 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:07.553913  4673 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:07.457736  7835 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:07.526139  8336 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:07.503252  8333 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:07.604849  8357 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:07.587564  8269 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.606101  8269 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:07.585875  8327 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11.
I20260709 12:26:07.586371  8298 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:07.611312  8298 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:07.572890  8285 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:07.577324  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 286799us
I20260709 12:26:07.546682  8114 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:07.528970  4596 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:07.614632  4596 raft_consensus.cc:3055] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:07.614987  4596 raft_consensus.cc:740] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:07.615623  8221 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:07.615803  4596 consensus_queue.cc:260] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:07.616664  8265 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.616793  4596 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:07.583331  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.617192  8265 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.585528  5347 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:07.573696  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:07.586017  4673 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:07.615690  4647 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.620080  8357 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:07.620321  4673 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:26:07.589020  4601 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:26:07.591038  7835 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:07.622042  8279 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:07.616818  8221 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:07.623394  8221 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.588510  4604 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:07.591682  7975 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:07.592481  8336 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.592312  8019 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:07.526261  8036 consensus_queue.cc:1059] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
W20260709 12:26:07.594034  5340 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 495) took 4657 ms (client timeout 2320 ms). Trace:
I20260709 12:26:07.594728  5349 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:07.594660  7863 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:07.595780  8018 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:07.597817  8281 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:07.602339  8379 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 12:26:07.582599  8335 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.611843  8298 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:07.618026  4598 raft_consensus.cc:3055] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:07.622560  8327 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:07.593600  8356 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:07.591923  8374 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:07.625331  4673 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:07.642541  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 59281.1us
I20260709 12:26:07.644025  4681 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:26:07.626000  8019 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:07.645838  4647 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:07.609427  8224 consensus_meta.cc:287] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.718s	user 0.010s	sys 0.000s
I20260709 12:26:07.658610  4598 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:07.659683  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 231695us
W20260709 12:26:07.660501  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.742054  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2685) took 1178 ms. Trace:
W20260709 12:26:07.739044  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 93289us
I20260709 12:26:07.735543  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:07.693874  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 590) took 1350 ms. Trace:
I20260709 12:26:07.766618  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.766844  4756 rpcz_store.cc:276] 0709 12:26:06.343824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.804822 (+460998us) service_pool.cc:224] Handling call
0709 12:26:07.693620 (+888798us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54000550d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:07.693790 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860503808}
W20260709 12:26:07.702960  4748 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2649) took 2645 ms (client timeout 1648 ms). Trace:
W20260709 12:26:07.767910  4748 rpcz_store.cc:269] 0709 12:26:05.057413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.176949 (+119536us) service_pool.cc:224] Handling call
0709 12:26:07.702663 (+2525714us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b5400183b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:07.702817 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1549120896}
I20260709 12:26:07.768451  5314 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.669716  4720 leader_election.cc:400] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8.
I20260709 12:26:07.774330  4720 leader_election.cc:403] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:07.705255  4755 raft_consensus.cc:2410] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 12:26:07.672528  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.780068  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.765908  4750 rpcz_store.cc:276] 0709 12:26:06.563191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.566154 (+  2963us) service_pool.cc:224] Handling call
0709 12:26:07.737477 (+1171323us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b540030a554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:07.741943 (+  4466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30402304}
I20260709 12:26:07.663328  4596 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:26:07.658404  8289 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:07.821382  4673 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:07.784502  4750 pending_rounds.cc:85] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 3
I20260709 12:26:07.657207  8260 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:07.823289  4750 pending_rounds.cc:107] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:26:07.825455  4750 raft_consensus.cc:2889] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:07.811481  4571 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:07.828709  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.720216  8293 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:07.708092  8157 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:07.661703  8018 raft_consensus.cc:3055] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:26:07.823526  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 57055.2us
I20260709 12:26:07.771330  4598 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:07.665033  8245 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:07.810075  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 44546.7us
I20260709 12:26:07.826099  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:07.840914  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:07.686862  8351 raft_consensus.cc:3055] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:07.827445  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:07.845322  5344 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:07.770092  8281 consensus_queue.cc:1059] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 12:26:07.830242  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:07.850808  5347 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:07.786279  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:07.772204  8081 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.647997  8247 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:07.770479  8224 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:07.588099  8246 consensus_queue.cc:237] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.775436  8356 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:07.844996  8351 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:07.859397  8081 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.662325  8335 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.843377  4598 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:07.843581  5349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 555) took 4731 ms. Trace:
I20260709 12:26:07.848851  8228 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:07.870942  5349 rpcz_store.cc:276] 0709 12:26:03.111533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.124936 (+ 13403us) service_pool.cc:224] Handling call
0709 12:26:07.594404 (+4469468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.633765 (+ 39361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.839492 (+205727us) spinlock_profiling.cc:243] Waited 4.47 s on lock 0x7b5400893850. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:07.839709 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10274632064}
I20260709 12:26:07.851612  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:07.826311  8219 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:07.825893  8327 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8.
I20260709 12:26:07.848948  4647 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.848626  4571 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:07.830914  7986 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:26:07.821292  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 151810us
I20260709 12:26:07.841035  8388 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:26:07.886384  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:07.854162  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 194402us
I20260709 12:26:07.838624  8386 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.692715  8344 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:07.856310  4673 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:07.846812  8245 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:07.852656  8157 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:07.865944  8221 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:07.819420  4596 pending_rounds.cc:85] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 5
I20260709 12:26:07.866159  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:07.898314  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:07.900079  4569 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:07.867527  4672 raft_consensus.cc:2393] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:07.827389  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2652) took 2769 ms. Trace:
I20260709 12:26:07.901078  4748 rpcz_store.cc:276] 0709 12:26:05.058429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.778013 (+2719584us) service_pool.cc:224] Handling call
0709 12:26:07.778805 (+   792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.827048 (+ 48243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.827079 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:07.827124 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:07.827159 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.827222 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.827334 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:07.851629  8289 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:07.872782  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 575) took 3103 ms. Trace:
I20260709 12:26:07.878774  4681 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:07.633559  5340 rpcz_store.cc:269] 0709 12:26:02.936109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.936307 (+   198us) service_pool.cc:224] Handling call
0709 12:26:07.593749 (+4657442us) spinlock_profiling.cc:243] Waited 3.5 s on lock 0x7b5400893854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:07.593924 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8055526272}
W20260709 12:26:07.882169  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.853299  8265 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:07.883679  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 569) took 3904 ms. Trace:
I20260709 12:26:07.871718  8374 raft_consensus.cc:1064] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:26:07.887961  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:07.890498  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:07.871030  8373 raft_consensus.cc:3055] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:07.842787  8018 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:07.870864  8385 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.903807  4596 pending_rounds.cc:107] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:26:07.936136  4596 raft_consensus.cc:2889] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:07.907542  4747 rpcz_store.cc:276] 0709 12:26:04.769148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.968063 (+1198915us) service_pool.cc:224] Handling call
0709 12:26:05.968472 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.968747 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:05.997327 (+ 28580us) log.cc:844] Serialized 14 byte log entry
0709 12:26:07.702186 (+1704859us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b180010e8e0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:26:07.872404 (+170218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:07.872471 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:07.872519 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.872590 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.872719 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20926,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":217,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":3964,"spinlock_wait_cycles":3868544,"thread_start_us":169769,"threads_started":1,"wal-append.queue_time_us":1901764}
I20260709 12:26:07.891424  8293 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:07.915340  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:07.915812  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.868640  8351 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.894794  8219 raft_consensus.cc:697] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:07.893703  8229 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:07.925762  8213 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:07.895052  7863 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:07.899207  8279 raft_consensus.cc:3055] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:26:07.947067  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.897975  8273 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:07.933207  8373 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:07.906817  8320 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:26:07.982824  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 82819.5us
W20260709 12:26:07.988435  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 98015.9us
I20260709 12:26:07.938130  4747 raft_consensus.cc:1240] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:26:07.936728  8265 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:07.891319  7986 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.939162  8246 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:07.925239  4749 rpcz_store.cc:276] 0709 12:26:03.974198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.979211 (+  5013us) service_pool.cc:224] Handling call
0709 12:26:03.993998 (+ 14787us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:04.028193 (+ 34195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.028228 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:06.426425 (+2398197us) spinlock_profiling.cc:243] Waited 6.38 ms on lock 0x7b18000d88e0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3972a6 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a
0709 12:26:07.222679 (+796254us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b2c000d0108. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:26:07.223425 (+   746us) log.cc:844] Serialized 52 byte log entry
0709 12:26:07.409319 (+185894us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:07.663000 (+253681us) log.cc:844] Serialized 14 byte log entry
0709 12:26:07.664903 (+  1903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.665064 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.878454 (+213390us) raft_consensus.cc:1685] finished
0709 12:26:07.878475 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.878635 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13014,"prepare.queue_time_us":434623,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":2358,"raft.queue_time_us":31574,"raft.run_cpu_time_us":469,"raft.run_wall_time_us":106553,"spinlock_wait_cycles":21064320,"thread_start_us":418036,"threads_started":2,"wal-append.queue_time_us":219786}
I20260709 12:26:07.897588  8386 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.950086  8157 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:07.999720  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2656) took 2940 ms. Trace:
I20260709 12:26:08.052421  4747 rpcz_store.cc:276] 0709 12:26:05.059454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.937505 (+2878051us) service_pool.cc:224] Handling call
0709 12:26:07.937897 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.999454 (+ 61557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.999665 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:07.913729  8081 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:08.003562  8019 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:07.988863  8229 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:08.090718  8229 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:08.100322  4602 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:07.992558  7863 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:08.104005  7863 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:07.996933  8397 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:08.110893  8397 consensus_queue.cc:1059] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:08.111987  8397 consensus_queue.cc:1243] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:07.995440  8279 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:08.113700  8018 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:08.115213  8018 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:08.118325  8373 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:08.120083  8373 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:08.131042  5342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 510) took 1758 ms (client timeout 2139 ms). Trace:
W20260709 12:26:08.131724  5342 rpcz_store.cc:269] 0709 12:26:06.373687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.545840 (+172153us) service_pool.cc:224] Handling call
0709 12:26:08.130705 (+1584865us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b540020dad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:08.130941 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35979648}
W20260709 12:26:08.133702  8391 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 12:26:08.135077  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2207) took 1008 ms. Trace:
I20260709 12:26:08.135723  4599 rpcz_store.cc:276] 0709 12:26:07.126291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.458305 (+332014us) service_pool.cc:224] Handling call
0709 12:26:07.458710 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.458983 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.459017 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.459528 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.830071 (+370543us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:07.830111 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.830194 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.134785 (+304591us) raft_consensus.cc:1685] finished
0709 12:26:08.134813 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.135003 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":367500,"threads_started":1,"wal-append.queue_time_us":403273}
I20260709 12:26:08.139312  4603 raft_consensus.cc:3055] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:08.139780  4603 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:08.140560  4603 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:08.143723  8364 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 1.277s	user 0.003s	sys 0.006s
I20260709 12:26:08.145282  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2146) took 4188 ms. Trace:
I20260709 12:26:08.145828  4752 rpcz_store.cc:276] 0709 12:26:03.956379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.999320 (+ 42941us) service_pool.cc:224] Handling call
0709 12:26:03.999674 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:04.028335 (+ 28661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.028368 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:06.423685 (+2395317us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b1800079ae0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3972a6 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a
0709 12:26:06.425189 (+  1504us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b2c000d0108. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:26:06.425840 (+   651us) log.cc:844] Serialized 52 byte log entry
0709 12:26:06.715465 (+289625us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:06.715531 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.715659 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.143161 (+1427502us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:08.144816 (+  1655us) raft_consensus.cc:1685] finished
0709 12:26:08.144848 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.145101 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6768,"raft.queue_time_us":11550,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":75393024,"thread_start_us":289102,"threads_started":1,"wal-append.queue_time_us":439941}
I20260709 12:26:08.149296  4752 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:08.152654  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2162) took 3076 ms. Trace:
W20260709 12:26:08.155362  8363 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 1.293s	user 0.006s	sys 0.001s
I20260709 12:26:08.155413  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2654) took 3096 ms. Trace:
I20260709 12:26:08.156288  4680 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:26:08.156111  4750 rpcz_store.cc:276] 0709 12:26:05.058966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.783636 (+2724670us) service_pool.cc:224] Handling call
0709 12:26:07.784179 (+   543us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:07.844353 (+ 60174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.844404 (+    51us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:07.851921 (+  7517us) log.cc:844] Serialized 78 byte log entry
0709 12:26:07.998050 (+146129us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:07.998119 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.998236 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.155175 (+156939us) raft_consensus.cc:1685] finished
0709 12:26:08.155194 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.155355 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":5047,"thread_start_us":105292,"threads_started":1,"wal-append.queue_time_us":262058}
I20260709 12:26:08.155016  4756 raft_consensus.cc:1217] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[8.5-8.5]   Dedup: 8.5->[]
I20260709 12:26:08.159129  4750 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 12:26:08.159703  4756 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:08.160248  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2663) took 3083 ms. Trace:
I20260709 12:26:08.102984  8260 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.160727  4750 rpcz_store.cc:276] 0709 12:26:05.076627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.158397 (+3081770us) service_pool.cc:224] Handling call
0709 12:26:08.158771 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:08.159814 (+  1043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.160137 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.161819  8036 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.162101  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 583875us
I20260709 12:26:08.154981  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1748) took 1119 ms. Trace:
I20260709 12:26:08.162853  4750 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 12:26:08.162941  4672 rpcz_store.cc:276] 0709 12:26:07.035960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.037048 (+  1088us) service_pool.cc:224] Handling call
0709 12:26:08.154760 (+1117712us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b54002df354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:08.154915 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1716841216}
I20260709 12:26:08.163650  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 579) took 3008 ms. Trace:
I20260709 12:26:08.058959  8293 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:07.979943  8385 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:08.060916  8396 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:26:08.062407  8219 consensus_queue.cc:237] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:08.064797  8398 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:26:08.067927  4747 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [5.7-8.8]
I20260709 12:26:08.070379  8335 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:08.072567  8273 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:08.041689  8265 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:08.042172  8246 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:08.082525  8157 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:08.052332  8221 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.042827  8351 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:08.088981  8395 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
W20260709 12:26:08.090126  8390 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
W20260709 12:26:08.049309  8288 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.726s	user 0.025s	sys 0.060s
I20260709 12:26:08.161230  8260 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:08.057984  4569 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:08.137164  7796 raft_consensus.cc:3055] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:08.164021  4750 rpcz_store.cc:276] 0709 12:26:05.154894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.162227 (+3007333us) service_pool.cc:224] Handling call
0709 12:26:08.162614 (+   387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:08.163394 (+   780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.163597 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.164441  4603 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:08.153934  4752 rpcz_store.cc:276] 0709 12:26:05.076333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.148306 (+3071973us) service_pool.cc:224] Handling call
0709 12:26:08.148692 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.151317 (+  2625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.152288 (+   971us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.164587  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:08.164017  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.492620  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.494462  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:08.254715  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 316762us
W20260709 12:26:08.252588  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 246650us
W20260709 12:26:08.497066  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 332539us
I20260709 12:26:08.453619  8081 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:08.005363  8269 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:08.482110  8298 consensus_meta.cc:287] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.813s	user 0.014s	sys 0.000s
W20260709 12:26:08.460007  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.165771  8279 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:08.524125  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:08.533140  4750 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:08.154191  8224 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:08.464054  4602 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:08.162406  8036 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:08.467103  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2659) took 3398 ms. Trace:
I20260709 12:26:08.545920  4747 rpcz_store.cc:276] 0709 12:26:05.068349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.067321 (+2998972us) service_pool.cc:224] Handling call
0709 12:26:08.067686 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:08.466024 (+398338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.467039 (+  1015us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.547097  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:08.465361  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 540) took 1334 ms. Trace:
W20260709 12:26:08.472615  8358 consensus_meta.cc:287] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.208s	user 0.011s	sys 0.008s
W20260709 12:26:08.469090  8285 consensus_meta.cc:287] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.856s	user 0.013s	sys 0.005s
I20260709 12:26:08.552613  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:08.490121  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 432219us
W20260709 12:26:08.469149  8250 consensus_meta.cc:287] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.438s	user 0.009s	sys 0.009s
I20260709 12:26:08.475245  8246 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:08.506336  5349 raft_consensus.cc:3055] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:08.464250  8293 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:08.245324  4598 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:26:08.475708  8335 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.567296  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 405074us
W20260709 12:26:08.491425  8288 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.163s	user 0.025s	sys 0.060s
I20260709 12:26:08.547744  4679 rpcz_store.cc:276] 0709 12:26:07.130991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.536689 (+405698us) service_pool.cc:224] Handling call
0709 12:26:07.571543 (+ 34854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.571826 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:07.829516 (+257690us) log.cc:844] Serialized 14 byte log entry
0709 12:26:08.460951 (+631435us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:08.465066 (+  4115us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:08.465115 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.465178 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.465303 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":297801,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":221951,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":8912,"thread_start_us":652978,"threads_started":2}
I20260709 12:26:08.505369  8211 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:08.550222  4751 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:26:08.458694  5347 consensus_meta.cc:287] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.607s	user 0.014s	sys 0.000s
I20260709 12:26:08.491124  7796 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:08.544775  8281 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:26:08.552080  8358 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:08.552706  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:08.498214  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.554046  4569 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:08.503696  8403 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:08.553186  8285 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:08.562294  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:08.562654  8250 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:08.687984  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 551) took 1043 ms. Trace:
I20260709 12:26:08.688495  4679 rpcz_store.cc:276] 0709 12:26:07.644806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.602512 (+957706us) service_pool.cc:224] Handling call
0709 12:26:08.687405 (+ 84893us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.687613 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.687639 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:08.687684 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:08.687717 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.687777 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.687899 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.566020  4598 pending_rounds.cc:85] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 7
I20260709 12:26:08.689777  4598 pending_rounds.cc:107] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:26:08.690176  4598 raft_consensus.cc:2889] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:08.691134  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 548) took 1501 ms. Trace:
I20260709 12:26:08.691563  4598 rpcz_store.cc:276] 0709 12:26:07.189677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.583244 (+393567us) service_pool.cc:224] Handling call
0709 12:26:07.617891 (+ 34647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.690763 (+1072872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.691057 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.565333  5349 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:08.541882  4604 raft_consensus.cc:3055] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:08.693189  4604 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:08.482939  8221 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:08.580561  5347 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:08.569058  4603 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:08.695554  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:08.696350  4747 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:08.514108  8081 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:08.601613  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1744) took 2561 ms. Trace:
I20260709 12:26:08.699810  4597 rpcz_store.cc:276] 0709 12:26:06.041399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.041513 (+   114us) service_pool.cc:224] Handling call
0709 12:26:06.064266 (+ 22753us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:06.066684 (+  2418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.066718 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:07.387129 (+1320411us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b2c000014c8. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:26:07.418198 (+ 31069us) log.cc:844] Serialized 52 byte log entry
0709 12:26:07.648202 (+230004us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:07.648240 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.648308 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.595607 (+947299us) raft_consensus.cc:1685] finished
0709 12:26:08.595630 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.601518 (+  5888us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3913,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"spinlock_wait_cycles":35838720,"thread_start_us":203554,"threads_started":1,"wal-append.queue_time_us":420867}
I20260709 12:26:08.661944  8242 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:26:08.603039  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.702157  4568 connection.cc:256] RPC call timeout handler was delayed by -1.4434273239999129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21250000000000002s for a response
I20260709 12:26:08.702389  8081 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:08.702656  8293 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:08.535212  8298 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:08.703001  8081 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:08.703204  8221 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:08.703588  8081 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:08.569680  8224 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.694054  4604 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:08.704668  4604 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:08.704466  8224 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:08.699818  4677 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:08.656901  4750 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:08.546969  8394 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:26:08.707116  8224 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:08.571805  8211 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.707759  8211 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:08.708033  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:08.709038  4750 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:08.710028  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:08.711127  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:08.711359  8350 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:26:08.562362  8404 consensus_queue.cc:1175] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49277a7653524787a715232ab908bbc6" responder_term: 8 status { last_received { term: 8 index: 9 } last_committed_idx: 8 last_received_current_leader { term: 8 index: 9 } }
I20260709 12:26:08.550182  7945 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:26:08.577782  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:08.662297  4753 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2165) took 2682 ms (client timeout 1987 ms). Trace:
W20260709 12:26:08.713752  4753 rpcz_store.cc:269] 0709 12:26:05.980183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.246036 (+265853us) service_pool.cc:224] Handling call
0709 12:26:08.662018 (+2415982us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b540004d654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:08.662204 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582384384}
I20260709 12:26:08.714205  7945 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.715112  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:08.714989  7945 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:08.662886  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 108966us
W20260709 12:26:08.715721  4755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2678) took 2539 ms (client timeout 2713 ms). Trace:
W20260709 12:26:08.716107  4755 rpcz_store.cc:269] 0709 12:26:06.176683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.034512 (+857829us) service_pool.cc:224] Handling call
0709 12:26:07.704668 (+670156us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b5400054c20. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee42ddc0 00007effee4b9e18 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6
0709 12:26:08.715549 (+1010881us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b540030af54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:08.715668 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651190400}
I20260709 12:26:08.717119  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:08.571513  8288 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 12:26:08.718189  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2694) took 4767 ms. Trace:
I20260709 12:26:08.703969  4672 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:08.718806  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 547) took 1306 ms. Trace:
I20260709 12:26:08.718611  4675 rpcz_store.cc:276] 0709 12:26:03.951031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.951781 (+   750us) service_pool.cc:224] Handling call
0709 12:26:03.960900 (+  9119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:06.938893 (+2977993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.992367 (+1053474us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b54004815a0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee42ddc0 00007effee4b9a02 00007effee4b5dd6 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:26:08.717951 (+725584us) spinlock_profiling.cc:243] Waited 2.95 s on lock 0x7b54000b27d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:08.718103 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6788083328}
I20260709 12:26:08.719204  4680 rpcz_store.cc:276] 0709 12:26:07.412833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.696169 (+283336us) service_pool.cc:224] Handling call
0709 12:26:08.156026 (+459857us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.197255 (+ 41229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.718623 (+521368us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b54002df350. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:08.718755 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1056551808}
I20260709 12:26:08.699131  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:08.719834  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 550) took 1135 ms. Trace:
I20260709 12:26:08.720297  4747 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:08.720330  4672 rpcz_store.cc:276] 0709 12:26:07.584121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.163976 (+579855us) service_pool.cc:224] Handling call
0709 12:26:08.461245 (+297269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.718871 (+257626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.719566 (+   695us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54001d1ad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:08.719767 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557455232}
I20260709 12:26:08.721462  8242 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:08.718473  7945 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:08.722523  8269 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:08.702880  4568 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:08.727324  4568 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:08.718221  4569 leader_election.cc:400] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8.
I20260709 12:26:08.729056  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2171) took 2688 ms. Trace:
I20260709 12:26:08.729380  4751 rpcz_store.cc:276] 0709 12:26:06.040863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.469962 (+1429099us) service_pool.cc:224] Handling call
0709 12:26:08.728866 (+1258904us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000e56d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:08.728987 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2465814912}
I20260709 12:26:08.730063  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:08.730386  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:08.730960  4751 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:08.731173  4755 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:08.610157  8389 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.002s	sys 0.011s
I20260709 12:26:08.571960  8335 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:08.736908  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:08.737620  4678 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:08.715683  4753 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:08.738622  5346 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:08.737744  4751 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:08.711694  4750 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:08.742133  5314 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:08.699576  8386 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:08.735948  4568 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:08.737001  8389 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.003s	sys 0.011s
I20260709 12:26:08.765693  4680 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:08.743953  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:08.767231  4751 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:08.768410  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:08.747030  4747 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:08.733974  8403 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:08.785982  8419 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:08.786478  8419 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.786855  8419 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:08.760848  8386 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:08.793782  8386 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.793819  8420 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:08.735387  8245 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:08.744012  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 233447us
I20260709 12:26:08.794783  8420 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.794991  8245 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.795336  8420 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:08.795580  8245 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:08.759558  4755 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:08.797334  4755 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:08.797191  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:08.773958  4753 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:08.798025  4672 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:08.799088  4753 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:08.799729  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 582) took 3488 ms. Trace:
I20260709 12:26:08.800071  4753 rpcz_store.cc:276] 0709 12:26:05.310987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.798536 (+3487549us) service_pool.cc:224] Handling call
0709 12:26:08.798799 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.799455 (+   656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.799675 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.801314  4753 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:08.783504  4751 raft_consensus.cc:1240] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:08.794207  8386 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:08.772591  8228 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.798033  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2668) took 3594 ms. Trace:
I20260709 12:26:08.805076  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 581) took 3649 ms. Trace:
I20260709 12:26:08.809473  4751 rpcz_store.cc:276] 0709 12:26:05.155420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.783091 (+3627671us) service_pool.cc:224] Handling call
0709 12:26:08.783340 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.804836 (+ 21496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.805013 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.808952  4755 rpcz_store.cc:276] 0709 12:26:05.203309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.796894 (+3593585us) service_pool.cc:224] Handling call
0709 12:26:08.797174 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.797811 (+   637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.797981 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.810753  4751 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:08.810932  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 281200us
I20260709 12:26:08.813326  8245 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:08.805550  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2670) took 3422 ms. Trace:
I20260709 12:26:08.814083  4753 rpcz_store.cc:276] 0709 12:26:05.383539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.800906 (+3417367us) service_pool.cc:224] Handling call
0709 12:26:08.801158 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.803932 (+  2774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.805486 (+  1554us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.781610  4750 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
W20260709 12:26:08.801558  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.815984  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 583) took 3340 ms. Trace:
I20260709 12:26:08.816390  4751 rpcz_store.cc:276] 0709 12:26:05.475936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.810298 (+3334362us) service_pool.cc:224] Handling call
0709 12:26:08.810592 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.815726 (+  5134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.815930 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.749917  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:08.817432  4751 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:08.818253  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 297985us
I20260709 12:26:08.818346  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2672) took 2643 ms. Trace:
I20260709 12:26:08.822513  4751 rpcz_store.cc:276] 0709 12:26:06.174680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.817119 (+2642439us) service_pool.cc:224] Handling call
0709 12:26:08.817313 (+   194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.818113 (+   800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.818291 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.803663  8293 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:08.817446  4750 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:08.824335  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 74536.1us
I20260709 12:26:08.827005  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2169) took 2845 ms. Trace:
I20260709 12:26:08.827062  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2671) took 2652 ms. Trace:
W20260709 12:26:08.829991  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 99182.9us
I20260709 12:26:08.830904  4569 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:26:08.831911  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 96044.6us
I20260709 12:26:08.829356  4750 rpcz_store.cc:276] 0709 12:26:06.174308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.816624 (+2642316us) service_pool.cc:224] Handling call
0709 12:26:08.816883 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.826868 (+  9985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.827013 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.833228  4755 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:08.834100  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2168) took 2852 ms. Trace:
I20260709 12:26:08.834313  8425 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:08.834537  4755 rpcz_store.cc:276] 0709 12:26:05.981234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.810760 (+2829526us) service_pool.cc:224] Handling call
0709 12:26:08.825844 (+ 15084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.833743 (+  7899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.834001 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:08.746399  7796 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:08.809302  8385 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:08.873307  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:08.874361  4680 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:08.820417  8298 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:08.879465  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 48632.3us
I20260709 12:26:08.880882  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:08.881850  4750 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:08.877208  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 44434.4us
I20260709 12:26:08.879432  4604 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:08.826407  8036 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:08.886128  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2651) took 3827 ms. Trace:
I20260709 12:26:08.886675  4756 rpcz_store.cc:276] 0709 12:26:05.058177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.789587 (+2731410us) service_pool.cc:224] Handling call
0709 12:26:08.154845 (+365258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.160159 (+  5314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.885845 (+725686us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b5400264150. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:08.886034 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838639360}
I20260709 12:26:08.831542  8421 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.893121  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:08.837702  8425 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:26:08.895310  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:08.821089  8273 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:26:08.921581  8288 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:08.923380  8288 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:08.925675  8081 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:08.819557  8228 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:08.831610  8260 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:08.832160  8265 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:08.840277  8415 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.900337  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:08.894809  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:08.886123  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 580) took 3731 ms. Trace:
I20260709 12:26:08.917429  4599 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:26:08.918139  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:08.935025  4600 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:08.934073  4752 rpcz_store.cc:276] 0709 12:26:05.155154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.507214 (+3352060us) service_pool.cc:224] Handling call
0709 12:26:08.507623 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.507896 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:08.755125 (+247229us) log.cc:844] Serialized 14 byte log entry
0709 12:26:08.885733 (+130608us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:08.885795 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:08.885852 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.885930 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.886054 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":277729,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":12685,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":11557,"thread_start_us":322920,"threads_started":2}
I20260709 12:26:08.822126  8394 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:26:08.914728  8129 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:08.919183  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:08.919639  8250 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:08.919855  8431 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:08.828943  8410 consensus_queue.cc:1059] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:26:08.938175  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:08.926574  7796 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:08.852560  8388 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 12:26:08.958349  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:08.955966  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:08.999588  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:08.960841  8408 log.cc:927] Time spent T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:26:09.019222  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:08.955876  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:08.965197  8018 consensus_meta.cc:287] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.836s	user 0.007s	sys 0.008s
I20260709 12:26:08.876757  8424 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.119s
I20260709 12:26:08.944789  8358 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:09.018682  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.125941  4599 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:09.021184  4600 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:09.022012  4598 raft_consensus.cc:2410] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:26:08.941002  8273 consensus_queue.cc:1243] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:08.982513  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:08.927522  8211 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:09.015934  8407 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:26:08.914387  8293 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:26:08.929559  8421 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.140355  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:09.136317  8373 consensus_meta.cc:287] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.016s	user 0.017s	sys 0.001s
I20260709 12:26:09.141927  4597 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:26:09.142571  8411 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:26:09.143136  8211 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:09.147320  8434 log.cc:927] Time spent T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:26:09.190616  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2662) took 4114 ms. Trace:
I20260709 12:26:09.202993  4749 rpcz_store.cc:276] 0709 12:26:05.075995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.078347 (+3002352us) service_pool.cc:224] Handling call
0709 12:26:08.078934 (+   587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.079480 (+   546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.079528 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:08.080284 (+   756us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.578408 (+498124us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:08.578472 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.578553 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.190350 (+611797us) raft_consensus.cc:1685] finished
0709 12:26:09.190377 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.190537 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7699,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":110,"thread_start_us":497644,"threads_started":1,"wal-append.queue_time_us":703471}
I20260709 12:26:09.137543  8335 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:09.126340  8422 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.017777  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.216755  4749 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:26:09.020771  8431 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:09.106241  8285 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:09.168421  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.159870  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:09.170254  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.223918  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2655) took 4164 ms. Trace:
W20260709 12:26:09.188026  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.104477  8281 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:26:09.206429  4751 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:08.960736  8415 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.217494  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2680) took 3040 ms. Trace:
I20260709 12:26:09.229147  4749 rpcz_store.cc:276] 0709 12:26:06.177367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.204502 (+3027135us) service_pool.cc:224] Handling call
0709 12:26:09.214892 (+ 10390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.217214 (+  2322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.217434 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:09.206580  7863 consensus_meta.cc:287] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.090s	user 0.010s	sys 0.005s
I20260709 12:26:09.216259  4604 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:26:09.224325  4748 rpcz_store.cc:276] 0709 12:26:05.059213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.902311 (+2843098us) service_pool.cc:224] Handling call
0709 12:26:07.902660 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.902910 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:07.902938 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.903402 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.431881 (+528479us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:08.432032 (+   151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.432143 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.223654 (+791511us) raft_consensus.cc:1685] finished
0709 12:26:09.223680 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.223852 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":450068,"threads_started":1,"wal-append.queue_time_us":727295}
I20260709 12:26:09.241159  4748 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:09.241914  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 589) took 2984 ms. Trace:
I20260709 12:26:09.242352  4748 rpcz_store.cc:276] 0709 12:26:06.257385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.240656 (+2983271us) service_pool.cc:224] Handling call
0709 12:26:09.240932 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.241520 (+   588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.241853 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.243709  4748 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:09.244379  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 591) took 2684 ms. Trace:
I20260709 12:26:09.244855  4748 rpcz_store.cc:276] 0709 12:26:06.559805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.243196 (+2683391us) service_pool.cc:224] Handling call
0709 12:26:09.243505 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.244127 (+   622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.244322 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.246356  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.223613  4599 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:09.226511  4598 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:09.239766  4749 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:26:09.248694  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.139302  8293 consensus_queue.cc:1243] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:09.250162  4598 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:26:09.253248  4602 raft_consensus.cc:1217] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Deduplicated request from leader. Original: 9.7->[10.8-10.8]   Dedup: 10.8->[]
I20260709 12:26:09.254047  4602 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:26:09.249228  4749 pending_rounds.cc:85] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 7
I20260709 12:26:09.264015  4749 pending_rounds.cc:107] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:26:09.264456  4749 raft_consensus.cc:2889] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:09.269796  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:09.271322  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 350385us
I20260709 12:26:09.271653  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:09.272449  4680 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:09.279064  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:09.284248  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.295475  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2681) took 3108 ms. Trace:
I20260709 12:26:09.296054  4749 rpcz_store.cc:276] 0709 12:26:06.187463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.230112 (+3042649us) service_pool.cc:224] Handling call
0709 12:26:09.239373 (+  9261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.285646 (+ 46273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.295387 (+  9741us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.222851  8422 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:09.320223  8430 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:26:09.321707  8389 consensus_queue.cc:1059] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:26:09.271142  8229 consensus_meta.cc:287] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.180s	user 0.015s	sys 0.000s
I20260709 12:26:09.241777  8285 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:26:09.247023  7863 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.283067  8228 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:09.324468  8431 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.329241  8229 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:09.335548  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 420160us
W20260709 12:26:09.338554  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:09.350360  4672 consensus_meta.cc:287] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.000s
I20260709 12:26:09.350996  4672 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:09.351593  4680 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:09.354009  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:09.357391  4747 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:09.356724  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:09.366114  4749 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:09.367064  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2683) took 2804 ms. Trace:
I20260709 12:26:09.366979  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:09.367723  4749 rpcz_store.cc:276] 0709 12:26:06.562630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.297066 (+2734436us) service_pool.cc:224] Handling call
0709 12:26:09.358681 (+ 61615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.366706 (+  8025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.366988 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:09.364207  5314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.016s	sys 0.032s
W20260709 12:26:09.369481  5314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.016s	sys 0.032s
W20260709 12:26:09.369856  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 558123us
W20260709 12:26:09.370357  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.368671  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:09.364017  8433 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:09.352294  8389 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.375926  8081 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:09.376423  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.385785  8433 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.386590  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.387356  4602 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:09.369263  8335 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:09.389183  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.391323  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.394928  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:09.370910  8298 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:09.397747  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.400978  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:09.401422  4602 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:26:09.398518  4599 raft_consensus.cc:2376] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14. Re-sending same reply.
I20260709 12:26:09.386612  8036 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.406843  8036 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:09.408978  8246 consensus_meta.cc:287] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.813s	user 0.010s	sys 0.002s
I20260709 12:26:09.409539  8246 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.410679  4673 raft_consensus.cc:1240] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: []
W20260709 12:26:09.410888  7975 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:26:09.411681  8246 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:09.413156  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 549) took 1995 ms. Trace:
I20260709 12:26:09.400915  8298 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.415441  8298 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.418263  4604 raft_consensus.cc:2410] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 10 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:09.421367  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2650) took 4363 ms. Trace:
I20260709 12:26:09.388514  8389 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.421849  4754 rpcz_store.cc:276] 0709 12:26:05.057885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.510127 (+2452242us) service_pool.cc:224] Handling call
0709 12:26:07.510492 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.584754 (+ 74262us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:26:07.584786 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.590527 (+  5741us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.840368 (+249841us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:07.840417 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.420969 (+1580552us) spinlock_profiling.cc:243] Waited 73.9 ms on lock 0x7b5400263254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:09.421020 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.421076 (+    56us) raft_consensus.cc:1685] finished
0709 12:26:09.421092 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.421311 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":467,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":5669,"spinlock_wait_cycles":169882880,"thread_start_us":249430,"threads_started":1,"wal-append.queue_time_us":266484,"wal-append.run_cpu_time_us":2065,"wal-append.run_wall_time_us":1002267}
I20260709 12:26:09.424266  4754 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:09.424990  8389 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:09.425418  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 592) took 2611 ms. Trace:
I20260709 12:26:09.425987  4754 rpcz_store.cc:276] 0709 12:26:06.814608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.423770 (+2609162us) service_pool.cc:224] Handling call
0709 12:26:09.424089 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.425195 (+  1106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.425364 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.385821  8081 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:09.392997  8433 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.418136  8229 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:09.408999  8036 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:09.433645  4673 rpcz_store.cc:276] 0709 12:26:07.418118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.926443 (+508325us) service_pool.cc:224] Handling call
0709 12:26:07.949764 (+ 23321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.411055 (+1461291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.412901 (+  1846us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540007c954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:09.413092 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3359341312}
I20260709 12:26:09.436250  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2687) took 2550 ms. Trace:
I20260709 12:26:09.439592  4754 rpcz_store.cc:276] 0709 12:26:06.886048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.435229 (+2549181us) service_pool.cc:224] Handling call
0709 12:26:09.435549 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.435799 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.435840 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.435895 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:09.435946 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.436016 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.436153 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.436331  4749 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:26:09.442395  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2686) took 2879 ms. Trace:
I20260709 12:26:09.442956  4749 rpcz_store.cc:276] 0709 12:26:06.563465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.368768 (+2805303us) service_pool.cc:224] Handling call
0709 12:26:09.436091 (+ 67323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:09.442039 (+  5948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.442323 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.441239  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:09.446694  4754 raft_consensus.cc:2410] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:09.447870  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:09.448776  4754 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:09.450114  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:09.445371  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:09.450941  4749 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:09.451100  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.451676  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:09.452011  4749 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:09.452682  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2179) took 2548 ms. Trace:
I20260709 12:26:09.452996  4749 rpcz_store.cc:276] 0709 12:26:06.904215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.451508 (+2547293us) service_pool.cc:224] Handling call
0709 12:26:09.451830 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.452476 (+   646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.452635 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.453521  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.454675  4749 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:26:09.458129  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 593) took 1596 ms. Trace:
I20260709 12:26:09.458846  4749 rpcz_store.cc:276] 0709 12:26:07.861340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.453808 (+1592468us) service_pool.cc:224] Handling call
0709 12:26:09.454238 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.457726 (+  3488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.457992 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:09.461427  8289 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.473s	user 0.009s	sys 0.023s
I20260709 12:26:09.463616  4749 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:09.437095  8081 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:09.468686  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.476028  8445 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:09.476887  8399 log.cc:927] Time spent T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 1.425s	user 0.008s	sys 0.016s
I20260709 12:26:09.477847  5341 raft_consensus.cc:2410] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:26:09.478150  8439 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.478657  8415 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:09.480679  7796 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:26:09.474416  8289 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.489s	user 0.009s	sys 0.023s
W20260709 12:26:09.499051  8420 consensus_meta.cc:287] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.006s
I20260709 12:26:09.520051  4675 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:26:09.520787  4672 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:09.512904  8289 raft_consensus.cc:697] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:09.544256  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 102681us
I20260709 12:26:09.546222  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:09.548907  4571 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:09.552276  8036 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:09.552742  8036 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.553102  8036 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.555760  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:09.553056  8351 consensus_meta.cc:287] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.070s	user 0.013s	sys 0.000s
I20260709 12:26:09.558027  4571 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:09.559206  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2206) took 2462 ms. Trace:
I20260709 12:26:09.559574  4605 rpcz_store.cc:276] 0709 12:26:07.097225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.426612 (+329387us) service_pool.cc:224] Handling call
0709 12:26:07.444916 (+ 18304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.662277 (+217361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.662308 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.662863 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.908696 (+245833us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:07.908754 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.908857 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.476326 (+1567469us) spinlock_profiling.cc:243] Waited 78.1 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:09.553678 (+ 77352us) raft_consensus.cc:1685] finished
0709 12:26:09.553698 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.559128 (+  5430us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109982,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":12757,"spinlock_wait_cycles":179739008,"thread_start_us":295724,"threads_started":2,"wal-append.queue_time_us":183529}
I20260709 12:26:09.522305  8298 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.522986  8420 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.511070  8439 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.522768  8441 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:26:09.551682  8447 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:09.563967  8293 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:09.564236  8389 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:09.603605  8439 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:09.582139  7796 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:26:09.499249  8445 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.608594  8081 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.612368  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:09.617671  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:09.620321  8289 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:09.621886  7796 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:09.621969  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.622495  7796 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:09.622717  4605 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:09.623382  8336 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:09.578526  8440 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:26:09.623441  7796 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.581403  8320 consensus_queue.cc:1059] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:26:09.625555  8320 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:09.625959  8320 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:09.626694  8320 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.627352  8298 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:09.627840  8320 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:09.629381  8444 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:09.629937  8444 raft_consensus.cc:697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:09.630424  8419 consensus_meta.cc:287] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.843s	user 0.011s	sys 0.002s
I20260709 12:26:09.618925  8439 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:26:09.630904  8444 consensus_queue.cc:237] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.631942  4754 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:09.631351  8081 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:09.619136  8389 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:09.634485  8389 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.611860  8293 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:09.637512  8433 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:09.637794  8293 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.638149  8433 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.638634  8433 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.641070  8433 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:09.634963  8428 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.497s	user 0.005s	sys 0.004s
W20260709 12:26:09.616190  8386 consensus_meta.cc:287] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.797s	user 0.010s	sys 0.002s
I20260709 12:26:09.611680  8447 raft_consensus.cc:697] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:09.655169  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:09.656678  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:09.657436  4605 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.662503  8445 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:09.663880  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2185) took 1169 ms. Trace:
W20260709 12:26:09.665386  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.664499  4754 rpcz_store.cc:276] 0709 12:26:08.494824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.456392 (+961568us) service_pool.cc:224] Handling call
0709 12:26:09.457441 (+  1049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.632418 (+174977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.663628 (+ 31210us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400264654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:09.663821 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400724224}
W20260709 12:26:09.670657  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 151638us
I20260709 12:26:09.675001  4748 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:26:09.675799  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:09.676697  4754 raft_consensus.cc:2410] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:26:09.681775  4605 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:09.688289  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.692987  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:09.675736  8350 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.693964  8350 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.695101  4754 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:09.686690  8435 raft_consensus.cc:3055] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:09.697934  8435 raft_consensus.cc:740] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:09.698439  8293 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.698832  8435 consensus_queue.cc:260] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.700188  8435 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:09.701073  8293 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:09.701967  4602 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:09.702829  7863 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:09.695804  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2193) took 1197 ms. Trace:
I20260709 12:26:09.722043  4754 rpcz_store.cc:276] 0709 12:26:08.498667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.694382 (+1195715us) service_pool.cc:224] Handling call
0709 12:26:09.694711 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.695535 (+   824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.695742 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:09.689918  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 72362.1us
I20260709 12:26:09.630834  8419 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.723795  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2194) took 1224 ms. Trace:
I20260709 12:26:09.727793  4754 rpcz_store.cc:276] 0709 12:26:08.499053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.722843 (+1223790us) service_pool.cc:224] Handling call
0709 12:26:09.723148 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.723460 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.723488 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.723529 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:09.723564 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.723621 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.723742 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97}
I20260709 12:26:09.666723  8386 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.729417  4754 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:09.741135  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 594) took 1238 ms. Trace:
I20260709 12:26:09.741516  4754 rpcz_store.cc:276] 0709 12:26:08.503458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.728815 (+1225357us) service_pool.cc:224] Handling call
0709 12:26:09.729108 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.740860 (+ 11752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.741077 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.725659  8336 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.744946  7835 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:09.745868  8382 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:09.739665  8436 raft_consensus.cc:3055] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:09.747460  8419 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:09.748387  4601 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:26:09.749284  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.749476  8420 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:09.749930  8456 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:09.737377  8447 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.751915  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.753108  8036 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.763226  4599 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:09.758636  4598 raft_consensus.cc:1240] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:09.765898  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2725) took 2414 ms. Trace:
I20260709 12:26:09.766497  4681 rpcz_store.cc:276] 0709 12:26:07.351545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.738546 (+387001us) service_pool.cc:224] Handling call
0709 12:26:07.821872 (+ 83326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.925353 (+103481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.765549 (+1840196us) spinlock_profiling.cc:243] Waited 49 ms on lock 0x7b540007b554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:09.765814 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112648064}
I20260709 12:26:09.768523  8019 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:09.758386  8288 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:09.773985  8288 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.774842  8386 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:09.777133  4720 leader_election.cc:336] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.888s (SENT)
I20260709 12:26:09.777630  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:09.787844  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:09.758452  8436 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:09.776189  8288 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:09.744490  8213 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.685s	user 0.018s	sys 0.063s
W20260709 12:26:09.801923  8213 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.742s	user 0.019s	sys 0.063s
I20260709 12:26:09.787268  7863 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.803133  7863 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:09.804809  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:09.806537  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.809851  4596 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:09.795619  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:09.811558  4599 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:09.823462  8436 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.824199  8019 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:26:09.828871  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:09.837606  8374 raft_consensus.cc:1068] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: Unable to promote non-voter dbe960c35c84451faf4e5b5c9233fa39: Illegal state: Replica 3a5aeedd11fe49938079fe86cb01232c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:26:09.838454  8433 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.839040  8433 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:08.827570  4753 rpcz_store.cc:276] 0709 12:26:05.981586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.814850 (+2833264us) service_pool.cc:224] Handling call
0709 12:26:08.815140 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.826678 (+ 11538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.826717 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:08.826759 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:08.826792 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.826850 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.826954 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4670,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":6964}
W20260709 12:26:09.857404  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 1.95384e+06us
I20260709 12:26:09.869388  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:09.757709  8336 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:09.878192  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:09.879313  5346 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:09.879495  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:09.846717  8397 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:26:09.848238  4677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1765) took 2804 ms (client timeout 1654 ms). Trace:
I20260709 12:26:09.898813  4596 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:09.900131  4644 connection.cc:256] RPC call timeout handler was delayed by -1.6650907839998581s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
I20260709 12:26:09.899891  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:09.905241  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:09.904088  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:09.907980  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:09.909010  4600 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:09.910957  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.906734  4673 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:09.915899  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:09.916781  4674 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:09.919080  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:09.921887  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:09.922821  7863 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:09.899129  8320 consensus_queue.cc:1059] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:26:09.924711  8320 consensus_queue.cc:1243] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:09.925220  8320 raft_consensus.cc:3055] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:09.889772  8420 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.929877  8420 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.932178  8420 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:09.932305  8336 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:09.907039  8288 consensus_queue.cc:1059] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:26:09.932951  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2682) took 3370 ms. Trace:
I20260709 12:26:09.951165  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:09.952040  4672 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:09.952889  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:09.933326  8288 consensus_queue.cc:1243] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:26:09.942140  8320 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:09.956954  7986 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:09.957474  8320 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:09.958007  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:09.958493  8422 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:09.958799  4680 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:09.960114  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.933367  4748 rpcz_store.cc:276] 0709 12:26:06.562316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.248570 (+2686254us) service_pool.cc:224] Handling call
0709 12:26:09.248906 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.675477 (+426571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.932710 (+257233us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54000555d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:09.932889 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979045632}
I20260709 12:26:09.889772  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:09.904801  4677 rpcz_store.cc:269] 0709 12:26:07.043402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.120352 (+ 76950us) service_pool.cc:224] Handling call
0709 12:26:09.847914 (+2727562us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b540038f4d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:09.848101 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3562770176}
I20260709 12:26:09.899361  8436 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:09.874317  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 97209.5us
I20260709 12:26:09.866860  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:09.876049  4599 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:09.881529  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:09.882637  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:09.796247  8081 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.905333  8246 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:09.873347  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.898953  8397 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:10.102583  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.063405  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:09.966670  4677 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:09.966171  4748 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:26:10.047031  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.103580  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:10.113487  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:09.966225  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:09.984251  4676 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:09.939009  5347 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:10.111671  4753 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.983636  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1762) took 2941 ms. Trace:
I20260709 12:26:10.114857  4678 rpcz_store.cc:276] 0709 12:26:07.042548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.331801 (+289253us) service_pool.cc:224] Handling call
0709 12:26:09.983342 (+2651541us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54001bbf54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:09.983555 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3047276800}
I20260709 12:26:09.905618  8374 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.118642  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:10.118939  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.123811  4748 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:26:10.117678  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.916206  8019 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:09.982419  5349 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:10.104765  8410 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.000s	user 0.004s	sys 0.015s
I20260709 12:26:10.132895  5349 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:10.130667  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:10.135285  4720 leader_election.cc:336] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.901s (SENT)
I20260709 12:26:10.135948  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:10.131757  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:10.137243  4748 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:10.138706  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:10.139573  4748 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:10.115978  8229 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.963232  8320 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:09.920706  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:09.882378  8289 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.979408  8242 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:10.157534  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:10.154398  4753 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:10.158252  4678 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:10.158329  4720 leader_election.cc:336] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.928s (SENT)
I20260709 12:26:10.158906  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:10.159694  5314 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:10.154277  4748 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:10.155993  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:09.976495  8433 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:10.009685  8380 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:10.165110  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.163967  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 264126us
W20260709 12:26:10.146979  8410 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.046s	user 0.004s	sys 0.016s
I20260709 12:26:10.155826  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.162247  4750 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:10.164376  4748 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 12:26:10.146132  8336 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.934131  8036 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:10.124771  8081 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:10.164261  7796 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:10.173537  4644 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:10.167924  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.171765  4677 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:26:10.150169  8229 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:10.158068  8242 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:10.159938  8335 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:10.177983  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:10.188737  8336 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:10.191641  4750 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:26:10.174889  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:10.204133  4756 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:10.127571  8246 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:10.171701  8433 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:10.137020  8374 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:10.212759  4644 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:10.225535  8336 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:10.233879  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:10.234946  4677 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:10.215206  4678 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:10.161043  8289 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:10.240350  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 224643us
I20260709 12:26:10.237973  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.208007  8464 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:10.242172  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:10.246790  4604 raft_consensus.cc:2410] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:26:10.210706  8435 consensus_meta.cc:287] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.510s	user 0.012s	sys 0.004s
I20260709 12:26:10.250844  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.252753  4677 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:10.216215  7796 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:10.173669  8380 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:10.258711  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 83889.6us
I20260709 12:26:10.259651  4647 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:10.260819  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:10.263543  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:10.264308  4677 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:10.264473  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 106173us
I20260709 12:26:10.267242  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2674) took 4092 ms. Trace:
I20260709 12:26:10.267853  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.268615  4678 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:10.268565  4749 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:10.175563  8397 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:10.267699  4751 rpcz_store.cc:276] 0709 12:26:06.175238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.823430 (+2648192us) service_pool.cc:224] Handling call
0709 12:26:08.823796 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.225414 (+401618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.266941 (+1041527us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54001f1654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:10.267181 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":879286144}
I20260709 12:26:10.233136  8451 consensus_queue.cc:1059] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:26:10.274542  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.275538  4677 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
W20260709 12:26:10.273878  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 61238.3us
I20260709 12:26:10.276747  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:10.278164  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:10.261210  8464 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:10.280545  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:10.284584  4604 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:10.286921  5349 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:26:10.287223  5349 pending_rounds.cc:85] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 6
I20260709 12:26:10.287362  5349 pending_rounds.cc:107] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:26:10.287515  5349 raft_consensus.cc:2889] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:10.288043  5349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 574) took 1793 ms. Trace:
I20260709 12:26:10.288246  5349 rpcz_store.cc:276] 0709 12:26:08.495096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.505695 (+ 10599us) service_pool.cc:224] Handling call
0709 12:26:08.506156 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.981917 (+1475761us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400295020. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee43406f 00007effee4acf60 00007effee4ae7de 00007effee4b8b49 00007effee4b9416 00007effee4b5d8f 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62
0709 12:26:10.287858 (+305941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.287999 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":286657408}
I20260709 12:26:10.250751  8466 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:26:10.293962  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:10.292532  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:10.297214  4677 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:10.297828  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.300508  4569 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:10.310652  8427 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
W20260709 12:26:10.320240  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 59528.7us
I20260709 12:26:10.322870  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2667) took 5120 ms. Trace:
I20260709 12:26:10.323450  4747 rpcz_store.cc:276] 0709 12:26:05.202910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.782428 (+3579518us) service_pool.cc:224] Handling call
0709 12:26:08.782758 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.365675 (+582917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.322496 (+956821us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b54000726d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:10.322782 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1320993664}
I20260709 12:26:10.319518  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:10.313144  8389 raft_consensus.cc:3055] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:10.307224  8213 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:10.289692  8433 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:26:10.314218  8435 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:10.333734  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.289973  8420 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.310010  8397 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:10.292160  8476 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:10.345046  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:10.346168  4604 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:26:10.347262  8293 raft_consensus.cc:3055] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:10.351702  8405 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.426921  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:10.424692  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49840.6us
I20260709 12:26:10.429111  4598 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:10.430673  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:10.431545  4602 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:26:10.432719  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:10.433434  4602 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:26:10.435038  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.435864  4677 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:10.437880  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:10.335644  8389 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:10.522795  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:10.523137  4755 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 12:26:10.342840  8213 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:10.526863  4604 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:26:10.531467  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:10.504287  4747 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:10.417511  8293 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:10.326211  7796 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:10.610175  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 55624.2us
I20260709 12:26:10.616173  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.632192  4747 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:10.614995  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:10.616353  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.541127  4604 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:26:10.633641  4755 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:10.540517  4602 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:10.418059  8447 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:10.435303  8420 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:10.649716  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.435063  8435 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:10.542488  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.426201  8477 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:10.638984  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:10.649497  4720 leader_election.cc:336] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.693s (SENT)
I20260709 12:26:10.652957  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:10.551800  8465 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.345s	user 0.001s	sys 0.000s
I20260709 12:26:10.609858  8466 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.660630  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:10.667585  4755 raft_consensus.cc:2410] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:26:10.609683  8157 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.541307  8081 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:10.655826  4602 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:26:10.645777  8229 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:10.562587  8478 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.546165  8213 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:10.677397  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.640620  8374 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:10.677973  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.504669  8405 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:10.537348  8373 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:10.683106  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.750869  4681 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:10.751909  8373 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:10.752899  8373 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:10.754921  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2694) took 1930 ms. Trace:
I20260709 12:26:10.756847  4754 rpcz_store.cc:276] 0709 12:26:08.825241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.742339 (+917098us) service_pool.cc:224] Handling call
0709 12:26:09.742625 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.742861 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:09.855942 (+113081us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.109729 (+253787us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.732155 (+622426us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b18000e4100. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007effeb3cba40 00007effeb3c64e4 00007effeb3c6455 00007efff1eae3c7 00007effee4c09a7 00007effee4d00bf 00007effee4d0038
0709 12:26:10.732446 (+   291us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.732652 (+   206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:10.732718 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:10.732772 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.747208 (+ 14436us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b18000e4100. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c9522 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:26:10.754715 (+  7507us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.754860 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10166,"prepare.queue_time_us":854665,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":114,"raft.queue_time_us":20190,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":14094,"spinlock_wait_cycles":33478016,"thread_start_us":345082,"threads_started":2,"wal-append.queue_time_us":304761}
I20260709 12:26:10.638827  8464 raft_consensus.cc:697] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:10.685999  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:10.767879  4755 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:10.660501  8289 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:10.664513  8335 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:10.696753  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.673280  8293 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:10.698523  8474 log.cc:927] Time spent T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
W20260709 12:26:10.699311  8470 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 12:26:10.699488  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.671046  8477 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:10.679263  8380 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:10.692284  8475 log.cc:927] Time spent T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.372s	user 0.001s	sys 0.000s
I20260709 12:26:10.819761  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:10.820607  4755 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:10.822059  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49795.9us
W20260709 12:26:10.712627  8472 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:26:10.676214  8435 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:10.674290  8466 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:10.692482  8427 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:10.707024  8467 log.cc:927] Time spent T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
W20260709 12:26:10.728808  8019 consensus_meta.cc:287] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.581s	user 0.008s	sys 0.004s
W20260709 12:26:10.732784  8436 consensus_meta.cc:287] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.000s
I20260709 12:26:10.732972  8445 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:10.734346  8320 consensus_meta.cc:287] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.002s
W20260709 12:26:10.844795  8468 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:26:10.744382  8482 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.747805  8405 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:10.748607  5341 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 573) took 2253 ms (client timeout 1522 ms). Trace:
I20260709 12:26:10.749856  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.883560  4677 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:10.654098  8447 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:10.764561  8409 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 2.065s	user 0.000s	sys 0.013s
W20260709 12:26:10.707960  8463 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.430s	user 0.000s	sys 0.001s
I20260709 12:26:10.766072  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:10.664111  8389 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:10.676334  8157 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:10.691161  8488 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:10.822841  4645 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:10.832885  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 59354.5us
I20260709 12:26:10.836269  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.696383  8478 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:10.902216  4680 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:26:10.848728  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.904057  4673 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:10.820282  8293 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:10.908097  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 85347.5us
I20260709 12:26:10.817354  8350 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:10.740222  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.913324  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.913812  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.843149  8487 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:26:10.877991  5344 raft_consensus.cc:2410] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:10.781370  8335 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:10.840934  8486 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.888829  4754 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:10.957855  8486 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:10.958523  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:10.959597  4754 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:10.819312  8490 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 12:26:10.960500  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:10.908437  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:10.961437  4754 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:10.887952  8482 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:10.962342  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.962679  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:10.962965  4677 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:10.963596  8482 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:10.910441  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:10.910749  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.963882  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.964499  4751 raft_consensus.cc:2410] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:10.965020  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.965368  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:10.966236  4751 raft_consensus.cc:2410] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:10.967026  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:10.967664  4751 raft_consensus.cc:2393] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 6: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:10.910936  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 599) took 1237 ms. Trace:
I20260709 12:26:10.967778  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:10.968360  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:10.968685  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:10.969041  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:10.909533  8476 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:10.941301  8388 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:10.944097  8242 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:10.968005  4748 rpcz_store.cc:276] 0709 12:26:09.673299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.177703 (+504404us) service_pool.cc:224] Handling call
0709 12:26:10.193725 (+ 16022us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.194151 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.194176 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.194644 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.251777 (+ 57133us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:10.251846 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.251941 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.910725 (+658784us) raft_consensus.cc:1685] finished
0709 12:26:10.910745 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.910879 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":125,"thread_start_us":43517,"threads_started":1,"wal-append.queue_time_us":66347}
I20260709 12:26:10.973093  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:10.969043  4751 raft_consensus.cc:2393] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 7: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:10.973284  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:10.945461  5344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 521) took 1064 ms. Trace:
I20260709 12:26:10.945719  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.976534  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:10.934371  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.973029  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:10.978179  4680 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:10.977236  5344 rpcz_store.cc:276] 0709 12:26:09.881141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.881253 (+   112us) service_pool.cc:224] Handling call
0709 12:26:10.945188 (+1063935us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b54005749d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:10.945383 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1760021376}
I20260709 12:26:10.853454  8427 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:10.948266  8335 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:10.950946  8421 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:10.951856  8452 raft_consensus.cc:3055] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:10.941274  8481 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:26:10.988610  4748 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:10.963703  4754 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:10.880366  8019 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:10.781755  8289 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:26:10.877810  8374 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:26:10.970443  8388 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:10.975586  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:10.986212  5345 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:11.032099  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.983440  8484 log.cc:927] Time spent T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:26:10.883280  5347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:11.036342  5347 raft_consensus.cc:606] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:10.910876  8389 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.000530  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:11.015453  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 261031us
I20260709 12:26:10.973156  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.910357  8229 consensus_queue.cc:1059] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.120s
I20260709 12:26:10.943184  8157 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:10.988343  8425 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.986976  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:11.032583  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57099.2us
I20260709 12:26:11.054052  4673 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:10.891824  8464 consensus_queue.cc:237] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:10.919230  8320 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.970467  8476 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.036718  8492 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 12:26:10.916152  8491 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:10.985102  8472 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:26:10.883144  5341 rpcz_store.cc:269] 0709 12:26:08.494767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.514143 (+ 19376us) service_pool.cc:224] Handling call
0709 12:26:10.748323 (+2234180us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b5400640354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:10.748487 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1686277504}
W20260709 12:26:11.040210  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.041875  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2738) took 1096 ms. Trace:
I20260709 12:26:11.039861  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:11.043375  4672 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: [9.7-10.8]
W20260709 12:26:11.033078  8397 consensus_meta.cc:287] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.592s	user 0.012s	sys 0.000s
I20260709 12:26:11.043882  4677 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:10.999501  8242 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:11.087337  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.088631  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:11.044399  5347 raft_consensus.cc:1217] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 3.6->[4.7-10.10]   Dedup: 6.8->[8.9-10.10]
I20260709 12:26:11.089960  5347 pending_rounds.cc:85] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 8
I20260709 12:26:11.090240  5347 pending_rounds.cc:107] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:26:11.090468  5347 raft_consensus.cc:2889] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:10.984097  8452 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:11.092666  4677 raft_consensus.cc:2393] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 7: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:11.092247  8452 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.018352  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.093603  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:11.093729  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 173521us
I20260709 12:26:11.046530  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:11.094563  4677 raft_consensus.cc:2410] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:26:11.094800  5345 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:11.049785  8500 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.050272  8229 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:11.051497  8478 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.052704  8246 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.062649  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:11.063823  4755 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:11.037983  8421 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:10.911391  8488 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.098481  8421 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:11.098770  8245 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:11.098968  8229 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:11.099208  8421 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.099395  8245 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.099941  8245 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:11.100035  8229 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.102074  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:11.090391  5349 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:11.056272  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.103472  8335 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:11.103811  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.103977  8335 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:11.104772  4680 raft_consensus.cc:2410] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:26:11.104334  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:11.098825  8500 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.104951  8335 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.093331  8452 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.108867  8350 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:11.104403  8488 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:11.078776  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 164463us
I20260709 12:26:11.095377  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:10.990554  8213 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.068351  4679 rpcz_store.cc:276] 0709 12:26:09.946094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.948864 (+  2770us) service_pool.cc:224] Handling call
0709 12:26:09.949306 (+   442us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:09.949707 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.949733 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:09.950346 (+   613us) log.cc:844] Serialized 52 byte log entry
0709 12:26:10.244030 (+293684us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:10.244091 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.244193 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.041589 (+797396us) raft_consensus.cc:1685] finished
0709 12:26:11.041619 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.041802 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27395,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"thread_start_us":293255,"threads_started":1,"wal-append.queue_time_us":363333}
I20260709 12:26:11.109668  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.093899  4675 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:11.113699  4755 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:11.112008  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.114782  4673 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:11.115303  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:11.113945  4680 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:26:11.112531  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:11.116199  4677 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:11.116875  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.118038  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.116528  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:11.119520  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.119879  8350 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:11.120154  8481 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:11.122159  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 562) took 1368 ms. Trace:
I20260709 12:26:11.122107  8427 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:11.122637  8213 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:11.123184  8427 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:11.123701  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:11.101394  8229 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.124069  8427 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.124939  4602 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:11.125759  8491 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:11.125975  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:11.126389  8447 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:10.975915  8477 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.127308  5348 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEARNER]: Advancing to term 11
I20260709 12:26:11.128693  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.129894  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.130585  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1776) took 1007 ms. Trace:
I20260709 12:26:11.130905  4679 rpcz_store.cc:276] 0709 12:26:10.122890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.129644 (+1006754us) service_pool.cc:224] Handling call
0709 12:26:11.130534 (+   890us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.131875  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.132463  8427 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.122556  4605 rpcz_store.cc:276] 0709 12:26:09.753786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.754034 (+   248us) service_pool.cc:224] Handling call
0709 12:26:09.754509 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:09.755021 (+   512us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:09.755069 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:09.755709 (+   640us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.948157 (+1192448us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b180047a940. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:26:11.031591 (+ 83434us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:11.031653 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.031747 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.121866 (+ 90119us) raft_consensus.cc:1685] finished
0709 12:26:11.121884 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.122083 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":977,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":2195,"spinlock_wait_cycles":2934144,"thread_start_us":83155,"threads_started":1,"wal-append.queue_time_us":1359342}
I20260709 12:26:11.124336  8213 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
W20260709 12:26:11.137666  4570 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.011s	sys 0.015s
W20260709 12:26:11.139711  4570 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.011s	sys 0.016s
W20260709 12:26:11.142232  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.143697  4647 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:11.144469  5348 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:11.145285  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:11.058025  8425 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.058322  8320 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.113361  8388 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:11.158300  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 603) took 1242 ms. Trace:
I20260709 12:26:11.146622  4679 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:11.160846  4675 raft_consensus.cc:2393] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 6: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:11.147523  4720 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:11.148085  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.160820  4747 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:26:11.116988  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 237217us
I20260709 12:26:11.171741  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:11.162701  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 253803us
I20260709 12:26:11.159961  4755 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:26:11.159751  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 48242.3us
I20260709 12:26:11.146261  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1772) took 1024 ms. Trace:
I20260709 12:26:11.175614  4680 rpcz_store.cc:276] 0709 12:26:10.121527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.116649 (+995122us) service_pool.cc:224] Handling call
0709 12:26:11.146194 (+ 29545us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.175921  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:11.169313  4750 rpcz_store.cc:276] 0709 12:26:09.915935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.226368 (+310433us) service_pool.cc:224] Handling call
0709 12:26:10.226769 (+   401us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:10.227185 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:10.732540 (+505355us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.872821 (+140281us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:10.880506 (+  7685us) log.cc:844] Serialized 52 byte log entry
0709 12:26:10.880696 (+   190us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:10.880747 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.880842 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.158085 (+277243us) raft_consensus.cc:1685] finished
0709 12:26:11.158103 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.158240 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":505044,"prepare.queue_time_us":520053,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":6055,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":6795,"thread_start_us":138767,"threads_started":1,"wal-append.queue_time_us":322452}
I20260709 12:26:11.176914  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.164220  5348 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:11.170446  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1783) took 1039 ms. Trace:
I20260709 12:26:11.179044  4679 rpcz_store.cc:276] 0709 12:26:10.130758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.131636 (+1000878us) service_pool.cc:224] Handling call
0709 12:26:11.170386 (+ 38750us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.178284  4680 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:11.159526  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:11.180882  4677 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.178462  5348 raft_consensus.cc:606] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
W20260709 12:26:11.187481  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.145407  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.162123  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:11.188946  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.189347  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:11.187232  4645 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:11.190200  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:11.192013  5341 raft_consensus.cc:2410] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:26:11.194090  8425 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.198104  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:11.175127  8229 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:11.207006  4677 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:11.207667  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2753) took 1050 ms. Trace:
I20260709 12:26:11.208056  4571 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:11.208869  4677 rpcz_store.cc:276] 0709 12:26:10.157408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.159203 (+1001795us) service_pool.cc:224] Handling call
0709 12:26:11.207604 (+ 48401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.210492  4677 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:11.211278  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2749) took 1095 ms. Trace:
I20260709 12:26:11.211788  4677 rpcz_store.cc:276] 0709 12:26:10.116029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.209944 (+1093915us) service_pool.cc:224] Handling call
0709 12:26:11.210256 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.211002 (+   746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.211216 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.213610  4677 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:11.214359  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2754) took 1042 ms. Trace:
I20260709 12:26:11.214828  4677 rpcz_store.cc:276] 0709 12:26:10.172285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.212829 (+1040544us) service_pool.cc:224] Handling call
0709 12:26:11.213162 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.214069 (+   907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.214296 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.199369  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:11.200018  5348 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:11.176959  8335 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:11.219903  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:11.220762  4603 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:11.220994  4600 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:26:11.204218  4673 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:11.204346  4680 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:11.185894  8380 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:11.204739  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:11.221804  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 59775.5us
I20260709 12:26:11.190765  8269 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:11.222369  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2744) took 1188 ms. Trace:
I20260709 12:26:11.224689  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:11.225044  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2748) took 1122 ms. Trace:
I20260709 12:26:11.227685  4673 rpcz_store.cc:276] 0709 12:26:10.102973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.199498 (+1096525us) service_pool.cc:224] Handling call
0709 12:26:11.203988 (+  4490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.224747 (+ 20759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.224975 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.222402  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.219569  8350 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:11.242625  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:11.226581  4680 rpcz_store.cc:276] 0709 12:26:10.033895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.180557 (+1146662us) service_pool.cc:224] Handling call
0709 12:26:11.203125 (+ 22568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.222127 (+ 19002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.222313 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:11.239574  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.234977  4571 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:11.243480  4603 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:11.239460  4679 raft_consensus.cc:1217] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[4.3-7.4]   Dedup: 4.3->[7.4-7.4]
I20260709 12:26:11.237861  8466 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:11.223361  8380 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.205922  8229 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:11.238507  8269 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:11.240000  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:11.240381  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:11.253618  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:11.239326  8503 consensus_queue.cc:1059] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:26:11.241998  8320 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.237914  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:11.153604  8245 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.244333  4679 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: [4.3-7.4]
I20260709 12:26:11.244894  4645 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.184700  8464 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:26:11.198933  8433 consensus_queue.cc:1059] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:26:11.237366  8427 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:11.228250  8421 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.243499  8491 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:11.250506  8350 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.255234  4603 raft_consensus.cc:2393] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 6: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:11.227451  8293 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:11.264616  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.273029  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:11.310384  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2728) took 3929 ms. Trace:
I20260709 12:26:11.311136  4676 rpcz_store.cc:276] 0709 12:26:07.381259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.694663 (+313404us) service_pool.cc:224] Handling call
0709 12:26:09.983981 (+2289318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.114320 (+130339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.310165 (+1195845us) spinlock_profiling.cc:243] Waited 2.29 s on lock 0x7b54001bbf50. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:11.310308 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5263928576}
I20260709 12:26:11.309755  4750 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:11.312433  4571 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:11.259475  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2188) took 2763 ms. Trace:
I20260709 12:26:11.227761  8478 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:11.250445  8476 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:11.253613  8229 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:11.252662  8380 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:11.253654  8489 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:11.250046  8519 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:11.314365  4597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1770) took 2409 ms (client timeout 1765 ms). Trace:
I20260709 12:26:11.314916  4747 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:26:11.326560  4597 rpcz_store.cc:269] 0709 12:26:08.905122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.958011 (+ 52889us) service_pool.cc:224] Handling call
0709 12:26:11.314172 (+2356161us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54002760d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:11.314303 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330963712}
I20260709 12:26:11.255654  8388 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:11.307669  8320 raft_consensus.cc:3055] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:11.308879  8350 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.317381  4571 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:11.303328  8427 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:11.314239  8478 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.303328  8213 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
W20260709 12:26:11.316610  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 127345us
I20260709 12:26:11.350345  5341 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:11.361338  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:11.362030  5341 raft_consensus.cc:606] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:11.350234  8380 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.362447  8335 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:11.362579  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:11.362579  5341 raft_consensus.cc:614] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:11.334719  8245 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:11.363752  8245 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:11.363934  8491 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:11.364140  8245 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.364566  8245 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.335216  8279 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.320350  8503 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:11.330789  8320 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:11.366837  8512 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:26:11.367651  8320 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.360496  8427 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.310065  8516 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:26:11.324214  8435 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:11.350169  5348 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:11.331713  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.313309  4749 rpcz_store.cc:276] 0709 12:26:08.495651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.510719 (+1015068us) service_pool.cc:224] Handling call
0709 12:26:10.268269 (+757550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.269052 (+   783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.259181 (+990129us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b54001f1650. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:11.259393 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1741355904}
I20260709 12:26:11.315177  8476 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.362991  8335 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.298866  8269 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.339445  8421 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:11.351125  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 106358us
I20260709 12:26:11.372031  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.379566  4571 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:11.375906  8289 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:26:11.376518  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.382292  8213 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:11.375766  8335 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.387964  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2741) took 1440 ms. Trace:
I20260709 12:26:11.388370  4672 rpcz_store.cc:276] 0709 12:26:09.947215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.961078 (+ 13863us) service_pool.cc:224] Handling call
0709 12:26:10.064922 (+103844us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:11.069908 (+1004986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.387679 (+317771us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b540019f354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:11.387896 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2249496320}
I20260709 12:26:11.390961  8380 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:11.392860  8516 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:11.393301  8516 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:11.394214  8516 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.372852  8435 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:11.374256  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.397873  8435 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:11.398296  4755 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:11.402060  8335 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:11.402971  4601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 560) took 2504 ms (client timeout 1922 ms). Trace:
I20260709 12:26:11.364017  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:11.414901  8350 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:11.452639  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 88684.7us
I20260709 12:26:11.369379  8320 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:11.438308  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 61894.4us
I20260709 12:26:11.314991  8293 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:11.442848  4601 rpcz_store.cc:269] 0709 12:26:08.898119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.388253 (+490134us) service_pool.cc:224] Handling call
0709 12:26:11.402725 (+2014472us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54002e54d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:11.402891 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796318464}
W20260709 12:26:11.445287  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 65809.6us
I20260709 12:26:11.476823  8335 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:11.477820  8420 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.361232  8478 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.479446  8245 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.483309  8350 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:11.451808  8421 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.372031  8519 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:11.484532  8421 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.485339  8519 raft_consensus.cc:697] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:11.486310  8519 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.486903  8421 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:11.488026  8486 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.453729  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:11.491236  4720 leader_election.cc:336] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.654s (SENT)
I20260709 12:26:11.491819  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.454761  8520 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.497974  8520 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.374081  8476 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.468792  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:11.474632  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.445297  8516 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:11.451704  8289 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:11.495569  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2745) took 1459 ms. Trace:
I20260709 12:26:11.442641  8247 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:11.500913  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.504024  4645 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.522285  8245 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:11.534114  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.535792  4645 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.536463  8213 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.537009  8213 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.472009  8229 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.543210  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2735) took 1598 ms. Trace:
I20260709 12:26:11.543654  4674 rpcz_store.cc:276] 0709 12:26:09.944933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.946256 (+  1323us) service_pool.cc:224] Handling call
0709 12:26:09.982240 (+ 35984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.982524 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:09.982857 (+   333us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.342924 (+360067us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.542753 (+1199829us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b18003cd600. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effee05e4bc 00007effeb3ac48b 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c6455 00007effee4ae060 00007effee4c0716 00007effee4d1287
0709 12:26:11.542851 (+    98us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.542901 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:11.542956 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.543019 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.543148 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123349,"prepare.queue_time_us":3598,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":10636,"raft.queue_time_us":525298,"raft.run_cpu_time_us":469,"raft.run_wall_time_us":27161,"spinlock_wait_cycles":32980736,"thread_start_us":448909,"threads_started":3,"wal-append.queue_time_us":352035}
I20260709 12:26:11.547816  4645 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.527887  4675 rpcz_store.cc:276] 0709 12:26:10.035877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.181553 (+1145676us) service_pool.cc:224] Handling call
0709 12:26:11.206923 (+ 25370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.223277 (+ 16354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.223310 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.250808 (+ 27498us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.495201 (+244393us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:11.495243 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.495319 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.495361 (+    42us) raft_consensus.cc:1685] finished
0709 12:26:11.495373 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.495506 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93080,"raft.queue_time_us":22166,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":8023,"thread_start_us":232890,"threads_started":1,"wal-append.queue_time_us":192813}
I20260709 12:26:11.522454  8464 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:11.552554  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:11.540782  8260 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.690s	user 0.032s	sys 0.026s
I20260709 12:26:11.512975  8421 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:11.550261  8491 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:11.566618  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.553645  8488 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:26:11.563174  8260 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.711s	user 0.033s	sys 0.026s
I20260709 12:26:11.555402  8489 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:11.571745  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.572992  4601 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:11.581676  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.582371  4747 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:26:11.583304  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.562230  8479 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:11.570542  8507 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:11.589179  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:11.584424  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.590807  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.588225  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:11.591981  4601 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:11.592662  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.577090  8421 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:11.572383  5340 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:11.604206  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.600540  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:11.602064  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:11.606802  4597 raft_consensus.cc:2393] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:11.609629  4756 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:26:11.613744  8245 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:11.614214  8245 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:11.614596  8245 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.602909  8273 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.618346  8293 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.586670  8213 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.591298  8520 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:11.622254  4599 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:11.622747  8229 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.591485  8531 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:11.618086  8273 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.610090  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:11.661995  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 51956us
I20260709 12:26:11.663023  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.668536  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.669433  4754 raft_consensus.cc:2393] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 7: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:11.670539  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:11.671721  4754 raft_consensus.cc:2368] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12.
I20260709 12:26:11.676118  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:11.677088  8269 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:11.677644  8520 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:11.677723  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:11.654390  8293 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:11.678689  8293 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.681017  8293 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.681159  8485 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:11.682432  8478 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.686636  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.660444  8479 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.685146  8478 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:11.690357  8520 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:11.702154  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:11.707741  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:11.707473  8269 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:11.722955  8435 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.724649  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.725556  4750 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:11.705763  8479 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.726925  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.727770  4750 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:11.728957  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.708166  8520 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:11.665254  8421 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:26:11.735888  8522 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 12:26:11.736456  8273 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.737344  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.734812  8380 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:11.736709  8129 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.727841  8479 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.740319  4747 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.725430  8435 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.747867  8129 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.757611  4752 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
W20260709 12:26:11.760433  4752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2695) took 2930 ms (client timeout 3084 ms). Trace:
W20260709 12:26:11.760937  4752 rpcz_store.cc:269] 0709 12:26:08.829705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.937737 (+108032us) service_pool.cc:224] Handling call
0709 12:26:11.760222 (+2822485us) spinlock_profiling.cc:243] Waited 2.77 s on lock 0x7b54000730d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:11.760359 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6380011904}
I20260709 12:26:11.764107  8530 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:11.765213  8500 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:11.767791  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 568455us
I20260709 12:26:11.768157  8397 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:11.768575  8397 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.768908  8397 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.769631  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:11.770831  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:11.771242  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2703) took 2121 ms. Trace:
I20260709 12:26:11.771659  8424 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:11.772068  8424 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.772480  8424 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.773490  8437 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:11.774029  8437 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:11.749804  8538 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 12:26:11.775398  5340 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53808 (request call id 511) took 1897 ms (client timeout 2191 ms). Trace:
W20260709 12:26:11.775838  5340 rpcz_store.cc:269] 0709 12:26:09.878400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.889360 (+ 10960us) service_pool.cc:224] Handling call
0709 12:26:11.775226 (+1885866us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b5400712354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:11.775337 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3794041216}
I20260709 12:26:11.777786  8436 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.772125  8466 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:11.778421  8436 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.780860  8436 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:11.782162  8281 raft_consensus.cc:3055] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:11.782680  8281 raft_consensus.cc:740] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:11.774392  8437 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.740808  8380 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:11.783595  8281 consensus_queue.cc:260] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.786206  8476 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.824227  8476 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:11.862644  4569 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 2.179s (SENT) [suppressed 43 similar messages]
W20260709 12:26:11.871533  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.880471  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:11.882483  4596 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:11.882853  4603 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:11.883901  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:11.888775  4600 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.894209  4601 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:11.896183  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 567) took 1654 ms. Trace:
I20260709 12:26:11.896752  4602 rpcz_store.cc:276] 0709 12:26:10.241368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.678895 (+437527us) service_pool.cc:224] Handling call
0709 12:26:10.679236 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.125328 (+446092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.895548 (+770220us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b540032bed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:11.895977 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1024588672}
I20260709 12:26:11.900231  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:11.771679  4753 rpcz_store.cc:276] 0709 12:26:09.649826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.160030 (+510204us) service_pool.cc:224] Handling call
0709 12:26:10.160346 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.234299 (+ 73953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.234344 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:10.234403 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:10.234458 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.770881 (+1536423us) spinlock_profiling.cc:243] Waited 73.7 ms on lock 0x7b540004c754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:11.771027 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.771171 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169437056}
I20260709 12:26:11.923683  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.937685  8421 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:11.938092  4600 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:11.794457  8380 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:11.957743  8421 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.945938  8281 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:11.957790  8260 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.961381  8260 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.961885  8485 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:11.966950  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2214) took 2092 ms. Trace:
I20260709 12:26:11.967303  8421 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:11.965196  8260 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:11.969218  8260 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:11.969817  8260 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.970263  8260 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:11.972010  4598 rpcz_store.cc:276] 0709 12:26:09.874548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.874675 (+   127us) service_pool.cc:224] Handling call
0709 12:26:10.428856 (+554181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.429758 (+   902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.966446 (+1536688us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b5400141ed0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:11.966721 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1273756416}
I20260709 12:26:11.972824  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:11.976140  4681 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:11.946185  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.977548  4678 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:11.978420  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:11.949748  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 268454us
I20260709 12:26:11.782135  8427 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.982232  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:11.984503  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 62041.8us
I20260709 12:26:11.974939  8476 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:11.967187  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:11.952165  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.986618  8476 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.986964  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:11.987535  8476 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:11.988101  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:11.988411  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:11.989102  4754 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:11.944410  8245 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:11.947732  8242 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.958458  8479 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:11.984606  8427 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:11.985103  5314 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.989266  4753 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.983335  4748 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:11.991549  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:11.997967  8420 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:12.000006  4756 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:11.998205  4677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:26:12.001420  4750 raft_consensus.cc:1240] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:11.994802  8479 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.026084  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:12.026587  4600 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:12.028861  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.037328  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:12.019119  8479 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:12.001976  8242 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.020826  4677 raft_consensus.cc:606] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:12.040578  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:12.048022  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.049213  4678 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:12.050395  8036 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:12.051019  8036 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:12.051913  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:12.054525  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.066205  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 81324us
I20260709 12:26:12.067194  5314 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:12.070063  8421 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:12.071472  8450 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:12.071976  8450 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.072332  8450 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:12.075685  4750 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:12.074842  8424 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.078393  8245 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:12.080595  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:12.081421  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:12.082123  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:12.090922  4679 raft_consensus.cc:2410] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 15 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:12.091890  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:12.092414  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:12.087693  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:12.094377  8397 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.095110  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:12.096964  8397 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:12.081246  4678 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:12.099709  4753 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:12.106386  8380 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.101822  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:12.109983  8531 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:12.110498  8531 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:12.111541  4676 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:12.114296  8327 raft_consensus.cc:3055] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:12.114784  4676 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:12.115474  8242 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:12.114802  8327 raft_consensus.cc:740] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:12.117632  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 566) took 1888 ms. Trace:
I20260709 12:26:12.117213  8327 consensus_queue.cc:260] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.118695  8327 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:12.118014  4604 rpcz_store.cc:276] 0709 12:26:10.229208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.645167 (+415959us) service_pool.cc:224] Handling call
0709 12:26:11.402881 (+757714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.403197 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.403224 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.403265 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:11.403295 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.403351 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.117314 (+713963us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b54002e54d0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:12.117453 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":443,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":1741526400}
I20260709 12:26:12.123507  4604 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:12.124927  8424 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:12.125792  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:12.126663  4672 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:12.126804  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:12.128176  4604 raft_consensus.cc:2410] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:12.131457  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:12.133131  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:12.134162  4600 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:12.135761  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:12.047358  4677 raft_consensus.cc:614] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:12.138248  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 71196.4us
I20260709 12:26:12.138532  8476 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:12.139963  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 148663us
I20260709 12:26:12.140707  8273 raft_consensus.cc:993] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:26:12.141090  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:12.141556  8404 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:12.142133  8404 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:12.142290  8273 raft_consensus.cc:1081] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:26:12.142740  8404 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:12.144390  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:12.145728  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 86752.7us
I20260709 12:26:12.146920  8530 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:12.147437  4675 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:26:12.148777  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:12.148811  5312 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:12.089362  4680 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:12.151393  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:12.154757  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:12.152509  8437 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.160236  4678 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
W20260709 12:26:12.162909  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.158740  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:12.165824  4679 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:12.140897  8129 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:12.109107  8380 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:12.174214  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:12.161959  8427 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.174574  4680 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:12.177065  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:12.177968  4673 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:12.201349  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:12.180557  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:12.204880  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:12.212945  4596 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:12.214282  4747 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:12.216037  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:12.217020  4747 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:12.217854  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:12.218328  4673 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:26:12.222020  4750 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:12.223367  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:12.224012  4750 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:12.226024  8422 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:12.227180  5346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 586) took 1152 ms. Trace:
I20260709 12:26:12.230988  8466 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:12.232721  5344 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:12.233741  8435 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:12.234599  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.234407  8435 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:12.236423  8242 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:26:12.237488  8557 log.cc:927] Time spent T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:26:12.237251  8242 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:12.239094  8547 log.cc:927] Time spent T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.060s	user 0.002s	sys 0.000s
I20260709 12:26:12.242552  4679 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:26:12.242978  4753 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:12.244268  5344 raft_consensus.cc:606] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:26:12.244837  5344 raft_consensus.cc:614] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:12.245450  8397 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:12.247128  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:26:12.250092  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.252396  8320 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:12.254262  8485 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:26:12.236444  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 134488us
I20260709 12:26:12.227561  5346 rpcz_store.cc:276] 0709 12:26:11.074462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.386112 (+311650us) service_pool.cc:224] Handling call
0709 12:26:11.439731 (+ 53619us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.781684 (+341953us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:11.782018 (+   334us) log.cc:844] Serialized 14 byte log entry
0709 12:26:12.050215 (+268197us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:12.050980 (+   765us) log.cc:844] Serialized 26 byte log entry
0709 12:26:12.051190 (+   210us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:12.051226 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.051317 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.226842 (+175525us) raft_consensus.cc:1685] finished
0709 12:26:12.226864 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.227016 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":341410,"prepare.queue_time_us":2992,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":342364,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":267785,"threads_started":1,"wal-append.queue_time_us":437306}
I20260709 12:26:12.300513  4596 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:12.301324  4680 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:26:12.272928  8530 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:12.313342  4600 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:26:12.314030  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:12.324822  4672 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:12.257205  8427 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:12.325639  4678 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
W20260709 12:26:12.328248  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 179527us
I20260709 12:26:12.334408  4499 catalog_manager.cc:5697] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 8 to 11, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 11 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:12.336355  4499 catalog_manager.cc:5697] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:12.350347  8519 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:12.350762  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 56433.3us
I20260709 12:26:12.351140  8519 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:12.351814  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:12.353452  8405 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:12.338716  4678 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:12.273435  8476 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
W20260709 12:26:12.339257  4500 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52818 (request call id 302) took 28529 ms (client timeout 15000 ms). Trace:
I20260709 12:26:12.342494  8242 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:12.345518  8036 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:12.378654  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 157592us
I20260709 12:26:12.346069  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:12.346292  8437 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:12.355949  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 167393us
I20260709 12:26:12.359570  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:12.360857  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:12.355422  8516 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:12.360376  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:12.354022  8410 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:12.353550  8519 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:12.408059  8410 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.407624  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:12.397068  8260 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:12.355171  8435 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:12.353981  8405 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:12.362743  4500 rpcz_store.cc:269] 0709 12:25:43.817246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.844829 (+ 27583us) service_pool.cc:224] Handling call
0709 12:26:12.338832 (+28494003us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:59.670533 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:59.671108 (+   575us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:59.671144 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:25:59.671162 (+    18us) tablet.cc:662] Decoding operations
0709 12:25:59.672102 (+   940us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:59.672181 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:59.672207 (+    26us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:25:59.672934 (+   727us) tablet.cc:701] Row locks acquired
0709 12:25:59.672959 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:25:59.673115 (+   156us) write_op.cc:270] Start()
0709 12:25:59.673235 (+   120us) write_op.cc:276] Timestamp: P: 1783599959673051 usec, L: 0
0709 12:25:59.673254 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:25:59.673972 (+   718us) log.cc:844] Serialized 3605 byte log entry
0709 12:26:00.139507 (+465535us) op_driver.cc:464] REPLICATION: finished
0709 12:26:01.353748 (+1214241us) write_op.cc:301] APPLY: starting
0709 12:26:01.353872 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:01.354198 (+   326us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:01.354362 (+   164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:01.355738 (+  1376us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:01.356097 (+   359us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 12:26:01.356176 (+    79us) write_op.cc:312] APPLY: finished
0709 12:26:01.356594 (+   418us) log.cc:844] Serialized 71 byte log entry
0709 12:26:01.359379 (+  2785us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:01.359580 (+   201us) write_op.cc:454] Released schema lock
0709 12:26:01.359667 (+    87us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:01.627960 (+268293us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:01.652813 (+ 24853us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":802553,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":1184190,"apply.run_cpu_time_us":4393,"apply.run_wall_time_us":6450,"num_ops":9,"prepare.queue_time_us":115133,"prepare.run_cpu_time_us":10569,"prepare.run_wall_time_us":143618,"raft.queue_time_us":166925,"raft.run_cpu_time_us":41679,"raft.run_wall_time_us":1660637,"replication_time_us":379269,"thread_start_us":1682440,"threads_started":5,"wal-append.queue_time_us":140967,"wal-append.run_cpu_time_us":17290,"wal-append.run_wall_time_us":2171613}]]}
I20260709 12:26:12.354774  4499 catalog_manager.cc:5697] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 5 to 7, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 7 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:12.416065  4681 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:12.415381  8410 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:12.401914  8516 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.428099  4754 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:12.423975  4672 raft_consensus.cc:1240] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:12.357124  4600 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:12.420571  8405 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:12.435745  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:12.432624  4681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1795) took 2175 ms (client timeout 1835 ms). Trace:
I20260709 12:26:12.441830  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:12.439711  4681 rpcz_store.cc:269] 0709 12:26:10.245343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.972104 (+1726761us) service_pool.cc:224] Handling call
0709 12:26:12.420110 (+448006us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.443351  5342 raft_consensus.cc:2393] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:12.418542  8437 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:26:12.422180  8531 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:12.455272  4499 catalog_manager.cc:5697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 5 to 7, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193), no longer has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 7 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:26:12.463338  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:12.482151  4568 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:12.470671  8435 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.503700  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:12.463536  8481 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 12:26:12.512360  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:12.467867  8382 consensus_queue.cc:1059] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:26:12.521771  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:12.524034  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.531998  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:12.531159  8441 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:12.527456  8435 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:12.541460  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:12.525584  8427 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:12.545955  8260 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:12.508507  5340 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:12.554863  8382 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:12.555281  8531 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:12.555485  8382 raft_consensus.cc:697] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:12.555994  8350 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:12.556732  8382 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.528698  4499 catalog_manager.cc:5697] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 3 to 4, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 4 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:12.573251  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.534610  8519 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:12.557713  8441 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:12.583913  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:12.584594  4681 raft_consensus.cc:2376] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10. Re-sending same reply.
W20260709 12:26:12.579012  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.580108  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 118026us
W20260709 12:26:12.585443  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 399048us
I20260709 12:26:12.596891  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:12.612648  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:12.613449  4672 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:12.639340  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2727) took 1048 ms. Trace:
I20260709 12:26:12.636070  4600 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:12.639724  4754 rpcz_store.cc:276] 0709 12:26:11.583860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.007332 (+423472us) service_pool.cc:224] Handling call
0709 12:26:12.045982 (+ 38650us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:12.434229 (+388247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.629357 (+195128us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54001f2054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:12.631953 (+  2596us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":871339776}
I20260709 12:26:12.644582  4750 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:12.638775  4499 catalog_manager.cc:5697] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 6 to 8, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 8 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:12.647181  4681 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:26:12.650844  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 52166.4us
I20260709 12:26:12.646206  4568 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:12.655822  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:12.659214  8450 consensus_meta.cc:287] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.586s	user 0.016s	sys 0.003s
I20260709 12:26:12.674577  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:12.660159  8350 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:12.662799  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2234) took 1043 ms. Trace:
W20260709 12:26:12.667906  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 432433us
I20260709 12:26:12.680333  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:12.646812  8404 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.700207  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2726) took 1116 ms. Trace:
I20260709 12:26:12.700838  4756 rpcz_store.cc:276] 0709 12:26:11.583600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.781229 (+197629us) service_pool.cc:224] Handling call
0709 12:26:11.781642 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.000618 (+218976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.683333 (+682715us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b5400263254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:12.700118 (+ 16785us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":501084544}
I20260709 12:26:12.692472  8435 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:12.689595  8350 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:12.667860  8519 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 12:26:12.711248  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.712042  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 400885us
I20260709 12:26:12.705204  5340 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:12.705719  8427 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:12.689139  8450 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.687309  4597 rpcz_store.cc:276] 0709 12:26:11.619254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.654089 (+ 34835us) service_pool.cc:224] Handling call
0709 12:26:11.654465 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.883611 (+229146us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:11.933291 (+ 49680us) log.cc:844] Serialized 14 byte log entry
0709 12:26:12.232947 (+299656us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.233008 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:12.233056 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.254456 (+ 21400us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b4800080058. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff33dfaa1 00007efff33d6b37 00007efff33d69ca 00007efff34ce26f 00007efff34cdf37 00007efff34e255c 00007efff34e2508 00007efff34e2488 00007efff34e2443 00007efff34e16ec 00007efff240ba96
0709 12:26:12.662152 (+407696us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b5400034154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:12.662240 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.662546 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51647,"prepare.queue_time_us":156880,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":73,"raft.queue_time_us":70986,"raft.run_cpu_time_us":3286,"raft.run_wall_time_us":21513,"spinlock_wait_cycles":586133760,"thread_start_us":264153,"threads_started":2,"wal-append.queue_time_us":372268}
I20260709 12:26:12.740137  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2211) took 1899 ms. Trace:
W20260709 12:26:12.750658  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 135508us
I20260709 12:26:12.745549  4751 rpcz_store.cc:276] 0709 12:26:10.742145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.001724 (+259579us) service_pool.cc:224] Handling call
0709 12:26:11.002026 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.002238 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:11.036978 (+ 34740us) log.cc:844] Serialized 14 byte log entry
0709 12:26:12.001979 (+965001us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b180018d420. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:26:12.527105 (+525126us) log.cc:844] Serialized 14 byte log entry
0709 12:26:12.568481 (+ 41376us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.574841 (+  6360us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:12.592522 (+ 17681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.615473 (+ 22951us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.640838 (+ 25365us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45961,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":5303,"raft.queue_time_us":10018,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":9361,"spinlock_wait_cycles":4030720,"thread_start_us":453162,"threads_started":1,"wal-append.queue_time_us":1614918}
W20260709 12:26:12.754922  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 106862us
I20260709 12:26:12.764079  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:12.775588  4756 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:26:12.758205  8559 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:26:12.705966  8441 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.769444  4499 catalog_manager.cc:5697] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:12.773892  4752 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:42632
I20260709 12:26:12.799760  4600 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
W20260709 12:26:12.804477  8560 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:26:12.789517  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:12.807806  4568 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:12.808632  8281 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:12.809621  8479 consensus_meta.cc:287] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.652s	user 0.006s	sys 0.011s
I20260709 12:26:12.809341  8281 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.810030  8479 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.812234  8479 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:12.812582  8281 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:12.814690  8404 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:12.821981  5340 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:12.823994  8424 raft_consensus.cc:2764] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:12.825289  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:12.829967  4674 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:12.831115  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2223) took 1125 ms. Trace:
I20260709 12:26:12.831532  4748 rpcz_store.cc:276] 0709 12:26:11.706339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.019484 (+313145us) service_pool.cc:224] Handling call
0709 12:26:12.068299 (+ 48815us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:12.068777 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.068803 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:12.069436 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:26:12.433510 (+364074us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:12.456033 (+ 22523us) log.cc:844] Serialized 14 byte log entry
0709 12:26:12.460896 (+  4863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.460973 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.830914 (+369941us) raft_consensus.cc:1685] finished
0709 12:26:12.830932 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.831058 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22601,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":2432,"raft.queue_time_us":5675,"spinlock_wait_cycles":4840704,"thread_start_us":363651,"threads_started":1,"wal-append.queue_time_us":524495}
W20260709 12:26:12.835867  8556 log.cc:927] Time spent T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:26:12.837059  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2220) took 1541 ms. Trace:
I20260709 12:26:12.803112  8036 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 12:26:12.777416  8485 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.020s	sys 0.013s
W20260709 12:26:12.854527  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 46530us
W20260709 12:26:12.846903  8485 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.021s	sys 0.014s
W20260709 12:26:12.752260  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 442830us
I20260709 12:26:12.866683  4672 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:12.837466  4749 rpcz_store.cc:276] 0709 12:26:11.250667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.378993 (+128326us) service_pool.cc:224] Handling call
0709 12:26:11.379313 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.446035 (+ 66722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.446084 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.446711 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.992287 (+545576us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b180015b680. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:26:12.465467 (+473180us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:12.465536 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.470057 (+  4521us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.773619 (+303562us) raft_consensus.cc:1685] finished
0709 12:26:12.773644 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.790393 (+ 16749us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66264,"raft.queue_time_us":68306,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":20053,"spinlock_wait_cycles":35126144,"thread_start_us":438595,"threads_started":1,"wal-append.queue_time_us":1259818}
I20260709 12:26:12.857205  8567 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:12.893002  4499 catalog_manager.cc:5697] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:12.887967  8397 consensus_meta.cc:287] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.642s	user 0.015s	sys 0.002s
W20260709 12:26:12.915449  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.856645  8566 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:26:12.924971  4499 catalog_manager.cc:5697] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 5, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 5 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:12.918010  8397 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:12.948326  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 210963us
W20260709 12:26:12.949059  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 235464us
W20260709 12:26:12.933061  8410 consensus_meta.cc:287] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.503s	user 0.013s	sys 0.000s
I20260709 12:26:12.929180  8566 consensus_queue.cc:1243] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:12.966480  4752 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:12.966936  4752 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:12.963266  4499 catalog_manager.cc:5697] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 7, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 7 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:12.956580  8394 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:12.969973  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 48954.4us
I20260709 12:26:12.958742  8450 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:12.975219  8229 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
W20260709 12:26:12.970990  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.956640  8242 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.986385  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:12.956655  8410 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:12.988957  4499 catalog_manager.cc:5697] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 6 to 9, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 9 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:13.000797  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:13.001425  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:13.006016  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:13.017901  4747 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:13.022009  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:13.004105  8242 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.026917  4753 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:26:13.027753  8573 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:26:13.028537  8516 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:13.033134  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 87171.2us
I20260709 12:26:13.030889  8242 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:13.036226  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:13.040918  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 66713.3us
I20260709 12:26:13.033674  8394 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.044850  8394 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.045821  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:13.049947  8410 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:13.049085  8516 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:13.051362  8516 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:13.051815  8516 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:13.051927  8554 log.cc:927] Time spent T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.698s	user 0.008s	sys 0.000s
I20260709 12:26:13.052986  8516 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.057474  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2760) took 1016 ms. Trace:
I20260709 12:26:13.057936  4674 rpcz_store.cc:276] 0709 12:26:12.040673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.051426 (+ 10753us) service_pool.cc:224] Handling call
0709 12:26:13.057243 (+1005817us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54002dee54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:13.057386 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1787427200}
I20260709 12:26:13.058275  8516 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:13.060267  8575 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.256s	user 0.001s	sys 0.005s
I20260709 12:26:13.065137  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:13.066030  4756 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:13.066977  8476 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:13.067225  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:13.067436  8476 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:13.074122  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 86018.9us
I20260709 12:26:13.073522  4597 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:13.031599  8572 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.068939  8427 raft_consensus.cc:2764] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:26:13.086194  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:13.088294  4602 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.090602  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.088282  4499 catalog_manager.cc:5697] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 2 to 5, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 5 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:13.096010  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:13.097522  4749 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:13.098433  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:13.100901  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:13.104429  4678 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:13.098804  4752 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:13.120901  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.121184  4603 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.113515  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:13.095638  8572 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:13.114933  8405 consensus_meta.cc:287] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.648s	user 0.007s	sys 0.007s
I20260709 12:26:13.084004  8566 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:26:13.130153  8439 raft_consensus.cc:3055] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:13.132710  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:13.136857  4600 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 10
W20260709 12:26:13.142127  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 56366.7us
I20260709 12:26:13.119915  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:13.138550  4499 catalog_manager.cc:5697] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 2 to 5, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 5 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:13.142442  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:13.105669  8242 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:26:13.149657  4499 catalog_manager.cc:5697] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 7, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 7 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:13.136303  8439 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:13.151485  5341 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.155076  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:13.169238  8466 raft_consensus.cc:3055] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:13.169816  8466 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:13.171200  8572 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:13.170863  8466 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.172001  8466 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.185111  4603 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:13.188247  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:13.123070  8476 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.191159  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:13.192312  4603 raft_consensus.cc:2410] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:26:13.194541  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:13.195233  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 578) took 1169 ms. Trace:
I20260709 12:26:13.195691  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:13.198379  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:13.200338  8320 consensus_queue.cc:1059] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:13.201275  8435 raft_consensus.cc:3055] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:13.201683  8435 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:13.201874  4605 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:13.195621  4598 rpcz_store.cc:276] 0709 12:26:12.026187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.027576 (+  1389us) service_pool.cc:224] Handling call
0709 12:26:12.027983 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:12.223692 (+195709us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.223722 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:12.224234 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 12:26:12.579680 (+355446us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:12.579720 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.194844 (+615124us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400034b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:13.194902 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:13.194945 (+    43us) raft_consensus.cc:1685] finished
0709 12:26:13.194957 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.195165 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63287,"raft.queue_time_us":63624,"spinlock_wait_cycles":281939968,"thread_start_us":355068,"threads_started":1,"wal-append.queue_time_us":530208}
I20260709 12:26:13.202630  8435 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:13.203038  8320 consensus_queue.cc:1059] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:13.203974  8427 raft_consensus.cc:3055] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:13.204185  8435 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:13.204403  8427 raft_consensus.cc:740] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:13.205431  4597 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:13.207516  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:13.208284  4750 raft_consensus.cc:2368] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10.
I20260709 12:26:13.208751  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:13.209586  4749 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:13.209659  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:13.210026  8439 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.210906  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:13.211122  8439 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:13.211551  4748 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
W20260709 12:26:13.213061  8587 log.cc:927] Time spent T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.166s	user 0.001s	sys 0.002s
I20260709 12:26:13.212992  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:13.213769  4749 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:13.214717  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:13.205539  8427 consensus_queue.cc:260] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.216903  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:13.216648  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:13.217540  4751 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:26:13.217962  4597 raft_consensus.cc:1240] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:26:13.218307  5344 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:13.220171  8596 log.cc:927] Time spent T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:26:13.220501  5344 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 12:26:13.220927  8427 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.136885  8405 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.222581  5346 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:13.223276  8405 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:13.225250  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:13.225986  4605 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:13.226879  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:13.227157  8567 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:13.184101  5342 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:13.215927  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:13.220628  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.230907  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:13.238583  4750 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:13.220319  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:13.232226  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:13.231724  4499 catalog_manager.cc:5697] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 6, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 6 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:13.246528  4600 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:13.231359  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:13.196888  8577 log.cc:927] Time spent T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.228s	user 0.001s	sys 0.001s
I20260709 12:26:13.254653  4598 raft_consensus.cc:2410] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:13.254534  4602 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:13.257210  4599 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:13.258306  4600 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:26:13.234328  8598 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:26:13.235091  8590 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:26:13.229547  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:13.265136  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:13.270175  4597 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:26:13.277525  8380 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:13.230221  8394 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:13.165266  8350 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:13.299222  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 69774.6us
I20260709 12:26:13.307529  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:13.309758  4597 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:13.309087  8519 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:13.312657  8519 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.314083  8422 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:13.308413  8516 raft_consensus.cc:2764] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:13.316491  8601 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.317096  8601 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.318475  4568 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:13.319607  8601 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:13.320142  8601 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:13.320657  4568 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:13.320627  8601 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.322265  8242 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:13.322520  8500 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:13.322961  8500 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.323386  8500 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:13.323868  8586 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
I20260709 12:26:13.324016  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:13.326124  4597 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:13.306593  8380 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.328259  8380 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:13.328817  8380 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.330539  4752 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:13.351193  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:42632 (request call id 622) took 1196 ms. Trace:
I20260709 12:26:13.332530  8519 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:13.354436  4752 rpcz_store.cc:276] 0709 12:26:12.155249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.643765 (+488516us) service_pool.cc:224] Handling call
0709 12:26:13.329038 (+685273us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b540030a054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ada3d 00007efff347dcd0 00007effeca02254 00007effeca021e2 00007effeca02111 00007effeca0208c 00007effeca01342 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:13.351090 (+ 22052us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384877824}
I20260709 12:26:13.358314  5314 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:13.359092  4676 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:13.329290  8380 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.334594  8293 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:13.335842  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:13.335750  8516 consensus_queue.cc:1059] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:13.336830  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:13.339197  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:13.341071  8601 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:13.341794  8580 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.358s	user 0.003s	sys 0.000s
I20260709 12:26:13.347148  4600 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:26:13.347431  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.348382  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:13.349187  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:13.350394  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:13.352104  8481 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:13.353780  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 46308.3us
I20260709 12:26:13.330767  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:13.307848  4499 catalog_manager.cc:5697] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 5 to 6, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 6 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:13.370213  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:13.398774  5345 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:13.383191  4597 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:13.384848  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.399708  4679 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:13.386029  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.395331  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.401219  4674 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:13.402463  4605 raft_consensus.cc:2410] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:26:13.402526  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.381942  5346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 526) took 1001 ms. Trace:
I20260709 12:26:13.403311  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.403976  4673 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:13.403748  5346 rpcz_store.cc:276] 0709 12:26:12.359077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.359254 (+   177us) service_pool.cc:224] Handling call
0709 12:26:13.360189 (+1000935us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b5400712354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:13.360317 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1898307328}
I20260709 12:26:13.396525  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:13.405854  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.407768  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:13.408133  4747 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:13.408517  4753 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:13.410665  4571 leader_election.cc:400] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10.
I20260709 12:26:13.411098  4571 leader_election.cc:403] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:13.407382  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.414791  4499 catalog_manager.cc:5697] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:13.419595  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 88917.2us
I20260709 12:26:13.420269  5344 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:13.422609  8445 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:13.383183  8293 raft_consensus.cc:3055] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:13.380992  8466 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:13.423640  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:13.425782  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:13.425105  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 63027.8us
I20260709 12:26:13.355768  8229 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:13.432132  5314 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:13.380589  8424 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:26:13.435525  4602 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:13.429842  8466 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.436672  4569 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:13.437039  8466 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:13.437721  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:13.438205  4569 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:13.438948  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:13.439877  8601 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:13.440618  8601 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:13.440831  7835 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:13.387985  8519 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:26:13.441385  7835 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:13.442464  8382 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:13.442683  8397 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:13.442584  7835 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.429774  8435 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10.
I20260709 12:26:13.428721  8293 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:13.447470  8293 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.448398  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:13.449082  8293 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.439476  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:13.456785  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:13.453554  4644 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:13.454362  4571 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:13.454818  4753 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:26:13.455027  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 623) took 1159 ms. Trace:
I20260709 12:26:13.457835  4747 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:26:13.458412  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:13.452138  8380 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:13.470185  4675 raft_consensus.cc:2393] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:13.449363  4755 raft_consensus.cc:2393] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:26:13.470736  4570 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.3.89.254:36317, user_credentials={real_user=slave}} blocked reactor thread for 46927.8us
I20260709 12:26:13.402105  8422 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:13.477100  4753 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
W20260709 12:26:13.477739  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 45707.1us
I20260709 12:26:13.407032  8479 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:13.419238  8516 consensus_queue.cc:1243] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:13.482270  4678 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:13.481789  4499 catalog_manager.cc:5697] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:13.484221  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:13.487190  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.383684  8405 consensus_queue.cc:1059] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
W20260709 12:26:13.497773  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 58384us
I20260709 12:26:13.458014  4754 rpcz_store.cc:276] 0709 12:26:12.295188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.654875 (+359687us) service_pool.cc:224] Handling call
0709 12:26:12.675399 (+ 20524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:12.842793 (+167394us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:13.130068 (+287275us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.297918 (+167850us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:13.333887 (+ 35969us) log.cc:844] Serialized 26 byte log entry
0709 12:26:13.334027 (+   140us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:13.334065 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.334135 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:13.454816 (+120681us) raft_consensus.cc:1685] finished
0709 12:26:13.454835 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.454966 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":158359,"prepare.queue_time_us":204064,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":157117,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":2505,"thread_start_us":403088,"threads_started":2,"wal-append.queue_time_us":262846}
W20260709 12:26:13.503605  8581 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.464s	user 0.002s	sys 0.001s
I20260709 12:26:13.503677  8530 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.504495  8530 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:13.505728  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.506445  8482 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:13.507241  8482 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.509820  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:13.510259  8350 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:13.510460  5345 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:26:13.459265  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:13.511694  5342 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:13.512056  4719 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:13.484021  8422 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.512070  5342 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:13.513649  8593 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.514451  8422 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:13.514199  8593 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.513881  5342 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.515004  8422 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:13.515384  8422 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.516008  8422 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:13.516285  8593 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:13.516149  8530 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:13.517011  8584 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:13.517478  8584 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.517871  8584 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:13.519169  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 65703.7us
I20260709 12:26:13.519917  4644 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:13.520664  8594 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:13.521158  8594 raft_consensus.cc:697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:13.521744  8572 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:13.522511  8572 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:13.523816  8380 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:13.524225  8380 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:13.524605  8380 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:13.526402  4677 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:13.499272  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:13.531178  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:13.522148  8594 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.501624  4499 catalog_manager.cc:5697] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 2 to 6, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 6 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:13.515331  5342 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:13.431414  8445 raft_consensus.cc:697] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:13.541201  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:13.487360  8114 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:13.496954  8320 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:13.555836  4569 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:13.491405  8519 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:13.547887  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:13.509465  8482 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:13.539328  8036 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:13.532656  8516 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:13.482764  8229 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.459406  8481 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.568648  4499 catalog_manager.cc:5697] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 4 to 7, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 7 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:13.551250  8601 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:13.549170  8114 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.561671  8572 consensus_queue.cc:1243] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:26:13.561910  8320 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.605257  4499 catalog_manager.cc:5697] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 8 to 9, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 9 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:13.608520  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:13.531893  8479 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.571925  8516 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.605705  8601 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.620005  4499 catalog_manager.cc:5697] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 6 to 7, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 7 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:13.664079  4499 catalog_manager.cc:5697] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:13.609920  8114 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:13.624998  8531 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:13.637625  5341 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:13.642830  8479 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:13.647310  8500 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.649010  4749 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:26:13.628186  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.675776  4747 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:13.617213  8320 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:13.684412  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.604676  8519 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.655769  8516 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.693114  4599 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:13.675560  8601 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:13.698982  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 261406us
I20260709 12:26:13.677903  8572 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:13.699918  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:13.711519  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.712184  4677 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:13.713187  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.714051  4678 raft_consensus.cc:2410] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:26:13.690294  8479 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:13.726285  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:13.686667  8531 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.730856  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:13.711004  8566 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:13.736990  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:13.742823  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 195035us
I20260709 12:26:13.745136  4571 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:13.743490  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:13.745911  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 190203us
I20260709 12:26:13.699365  8380 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.748073  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:13.750523  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:13.751878  4680 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:13.732955  8350 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.687613  8445 consensus_queue.cc:237] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:13.752456  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 662444us
I20260709 12:26:13.721793  8466 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.723312  8572 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.727869  8579 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.192s
I20260709 12:26:13.747015  8476 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:13.738432  8479 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.772710  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:13.782847  8466 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:13.783721  8605 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:26:13.785713  5347 raft_consensus.cc:2368] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11.
W20260709 12:26:13.786144  8606 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:26:13.789987  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2247) took 1659 ms. Trace:
I20260709 12:26:13.790354  4601 rpcz_store.cc:276] 0709 12:26:12.130616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.130786 (+   170us) service_pool.cc:224] Handling call
0709 12:26:12.131324 (+   538us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:12.211432 (+ 80108us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:12.303220 (+ 91788us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.227994 (+924774us) spinlock_profiling.cc:243] Waited 25.6 ms on lock 0x7b1800062fe0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:26:13.531734 (+303740us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:13.549834 (+ 18100us) log.cc:844] Serialized 52 byte log entry
0709 12:26:13.550030 (+   196us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:13.550356 (+   326us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.550536 (+   180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.550607 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:13.789723 (+239116us) raft_consensus.cc:1685] finished
0709 12:26:13.789746 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.789926 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52351,"prepare.queue_time_us":92813,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":2396,"raft.queue_time_us":76926,"raft.run_cpu_time_us":422,"raft.run_wall_time_us":9744,"spinlock_wait_cycles":67543552,"thread_start_us":303396,"threads_started":1,"wal-append.queue_time_us":1362711}
W20260709 12:26:13.798574  8427 consensus_meta.cc:287] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.001s
W20260709 12:26:13.802119  8603 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:26:13.803915  8614 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:13.806244  8427 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:13.810585  8519 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:13.811445  8519 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:13.811921  8519 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.812309  8519 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:13.813386  8500 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:13.814345  8500 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:13.819165  4603 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:13.764179  8566 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:13.767488  8530 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:13.761304  8350 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.854017  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:13.854686  4678 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:13.813642  8427 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:13.868490  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 131577us
I20260709 12:26:13.832672  8566 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.843588  8530 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:13.849740  8579 consensus_queue.cc:1243] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:26:13.873413  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 431872us
I20260709 12:26:13.871210  8566 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:13.876139  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.877190  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:13.880086  4680 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:13.882622  8114 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.833117  8422 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.886023  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.886827  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.896874  4674 raft_consensus.cc:2393] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:13.889933  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:13.898186  4680 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.893793  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.894393  8626 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:13.893474  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:13.894451  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.893446  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:13.899101  4679 raft_consensus.cc:2410] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 12:26:13.911175  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 166128us
I20260709 12:26:13.912490  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.913555  4681 raft_consensus.cc:2410] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:26:13.886785  8530 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:13.846757  8481 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:13.915692  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.892529  8479 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:13.893134  8445 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:13.920053  4674 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:13.921861  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.922431  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.925800  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:13.926107  4673 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:13.922573  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 172129us
I20260709 12:26:13.910110  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:13.925830  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:13.871091  8585 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
W20260709 12:26:13.930970  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.932930  4680 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:13.932987  4678 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:13.933207  8242 raft_consensus.cc:3055] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:13.933705  8242 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:13.918288  8614 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 12 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:13.934486  8242 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.935545  8242 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.935678  8614 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.939718  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.941473  4644 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:13.918819  8579 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:13.949774  8380 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:13.950620  8422 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:13.950933  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:13.951210  8618 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.951901  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:13.951753  8618 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.952808  4674 raft_consensus.cc:2410] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:13.951898  8350 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:13.954200  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:13.959118  4647 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:13.963042  4676 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
W20260709 12:26:13.964046  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 65291.5us
I20260709 12:26:13.965095  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:13.966224  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:13.965021  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:13.970963  8628 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:26:13.971455  8624 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:13.971773  8628 raft_consensus.cc:697] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:13.972560  5347 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:26:13.975930  8404 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:13.976603  8404 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:13.977640  8404 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:13.978233  4673 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
W20260709 12:26:13.979589  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.973013  8628 consensus_queue.cc:237] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.981446  8633 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:13.982173  8633 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:13.983310  8593 raft_consensus.cc:3055] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:13.983762  8593 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:13.984953  8114 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:13.986171  8616 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:26:13.986827  8293 consensus_meta.cc:287] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.002s
I20260709 12:26:13.984778  8593 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.001799  4681 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:14.003346  4677 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:13.943885  8479 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:14.002897  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:14.019436  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 52279.6us
I20260709 12:26:13.957805  8481 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:13.956697  8516 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:13.918087  8584 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.023171  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:14.035686  4756 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:14.034739  5347 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 12:26:13.964171  8531 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:14.038609  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 58836.1us
I20260709 12:26:13.914826  8626 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.038671  4680 raft_consensus.cc:2376] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10. Re-sending same reply.
W20260709 12:26:14.044901  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 85862.7us
I20260709 12:26:13.996052  8572 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:14.049501  4672 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:14.053508  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:14.054316  4755 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:14.042088  8593 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:14.051424  8614 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.083454  8614 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.047559  8482 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:26:14.053001  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:14.086371  5340 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:14.086431  8618 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:14.051563  8380 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:14.081876  8615 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:14.082744  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:14.048297  8479 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:26:14.076457  8427 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:14.096846  4647 leader_election.cc:400] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12.
I20260709 12:26:14.122192  4647 leader_election.cc:403] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:26:14.110682  5347 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:14.070144  8404 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:26:14.072819  8579 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.120110  8632 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 12:26:14.120193  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:14.119976  8380 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.120568  8482 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.085551  8320 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:14.129223  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.119357  4681 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
W20260709 12:26:14.194136  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 147232us
I20260709 12:26:14.077751  8229 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:14.085523  8614 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:14.214493  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 144681us
I20260709 12:26:14.088973  8485 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 12:26:14.131937  8595 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.130211  8519 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.133066  8445 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:26:14.231246  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.203864  8615 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:14.228545  4678 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:14.214385  8585 raft_consensus.cc:2764] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:26:14.242844  4601 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:14.246035  4599 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [6.4-8.5]
W20260709 12:26:14.245898  8613 consensus_queue.cc:1175] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "dbe960c35c84451faf4e5b5c9233fa39" responder_term: 15 status { last_received { term: 13 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: [14.11-14.11]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:26:14.247756  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 335062us
I20260709 12:26:14.248317  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.249118  4600 raft_consensus.cc:2410] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:26:14.257582  4755 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:14.214093  5340 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:14.229984  8435 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:14.259788  5314 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:14.264079  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.279137  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.282987  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.288808  5347 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:26:14.206353  8579 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.289680  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.299013  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.229753  8482 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.280740  8382 raft_consensus.cc:3055] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:26:14.335711  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 400876us
I20260709 12:26:14.337076  4569 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:26:14.338660  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 75333.2us
I20260709 12:26:14.283202  8626 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:14.230737  8595 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.304528  8632 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:14.399802  4756 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:14.307652  8479 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:14.310926  8382 raft_consensus.cc:740] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:14.308056  8516 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:14.340813  8584 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:14.339346  8481 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:14.405064  8445 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.307809  8567 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:26:14.414201  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 107858us
W20260709 12:26:14.418926  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 81957.2us
I20260709 12:26:14.409744  8647 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.428834  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.380033  8293 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:26:14.430188  8632 raft_consensus.cc:697] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:14.430757  8637 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:26:14.431008  8482 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:14.431131  8632 consensus_queue.cc:237] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.431757  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.432493  8320 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:14.432716  8382 consensus_queue.cc:260] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.437994  8380 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:14.429384  8649 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:14.454639  4601 raft_consensus.cc:2393] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:14.455205  8649 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:14.456954  8519 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:14.456540  8649 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:14.457732  8638 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 12:26:14.458067  8643 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12.
I20260709 12:26:14.458235  8649 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:14.459394  8639 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:26:14.418423  8479 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.463697  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48198 (request call id 584) took 1046 ms. Trace:
I20260709 12:26:14.464214  4672 rpcz_store.cc:276] 0709 12:26:13.417102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.425476 (+  8374us) service_pool.cc:224] Handling call
0709 12:26:14.463442 (+1037966us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b54001d33d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:14.463621 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1432188672}
I20260709 12:26:14.334740  8566 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.469305  8566 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:14.470413  8476 consensus_meta.cc:287] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.695s	user 0.010s	sys 0.000s
W20260709 12:26:14.478933  5342 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.914s	user 0.007s	sys 0.004s
I20260709 12:26:14.428247  8567 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.479354  5342 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:14.479918  5342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53808 (request call id 527) took 1121 ms. Trace:
I20260709 12:26:14.480249  5342 rpcz_store.cc:276] 0709 12:26:13.358613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.386708 (+ 28095us) service_pool.cc:224] Handling call
0709 12:26:14.479828 (+1093120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:14.481074  8567 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:14.481464  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 434583us
I20260709 12:26:14.481848  8634 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:14.482715  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:14.462839  8516 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.484838  8516 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.486982  8516 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:14.487274  8519 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.487958  8519 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.490397  8634 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:14.490916  8634 raft_consensus.cc:697] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:14.491983  4601 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:14.491775  8634 consensus_queue.cc:237] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.497010  5314 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:14.423107  8615 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.429936  4568 leader_election.cc:400] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11.
I20260709 12:26:14.500370  4568 leader_election.cc:403] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:14.434249  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.433795  8647 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:14.426771  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 167407us
I20260709 12:26:14.433491  8481 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.431872  8629 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:14.438890  8579 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:14.433974  8635 log.cc:927] Time spent T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:26:14.453972  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.436980  4569 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:14.504514  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 74714.9us
I20260709 12:26:14.463274  8612 raft_consensus.cc:3055] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:14.506712  4677 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:26:14.503356  4719 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:14.543758  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:14.497859  8628 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:14.512486  8530 consensus_meta.cc:287] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.000s
I20260709 12:26:14.580758  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:14.515436  4601 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:26:14.534665  5312 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:14.535545  4602 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:26:14.540467  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 291941us
I20260709 12:26:14.540675  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:14.546986  4719 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:14.508329  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.555217  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:14.583446  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48884.2us
I20260709 12:26:14.583582  4596 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:26:14.584434  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:14.584636  5312 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:14.579269  4597 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:26:14.510674  8382 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:14.509658  7835 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:26:14.589080  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:14.533447  8567 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:26:14.590943  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:14.591382  4752 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:14.600425  4750 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:14.593425  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:14.601742  4681 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:26:14.593904  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 47012.9us
I20260709 12:26:14.601298  4675 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:14.594075  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.603304  4596 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:14.599432  4754 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 7: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:14.599858  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.544114  8612 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:14.608256  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:14.579843  8633 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:14.614048  4752 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:14.599807  8601 consensus_meta.cc:287] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.872s	user 0.012s	sys 0.000s
I20260709 12:26:14.615240  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:14.615176  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.620658  4596 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:14.593952  8424 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:26:14.621438  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 184597us
I20260709 12:26:14.592804  8650 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:14.655993  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:14.601902  8422 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:14.656759  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:14.664837  4751 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:14.604477  8566 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:14.657955  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:14.676826  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.679711  4752 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:14.666357  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:14.689145  8647 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:14.625005  8633 raft_consensus.cc:697] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 7 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:14.604367  8479 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:14.673713  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.599229  8405 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:14.681481  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:14.700975  4754 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:14.682780  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:14.686563  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.689237  8644 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:14.626403  8614 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:14.589514  8530 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.676014  8566 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:14.692760  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:14.693153  4571 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:14.589727  8628 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.700089  4750 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:14.700189  4596 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:14.702325  4605 raft_consensus.cc:2410] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:14.711766  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.703110  4747 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:14.664108  8519 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:14.691360  4603 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:14.705891  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:14.621315  8601 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:14.708223  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 167676us
I20260709 12:26:14.716068  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:14.709177  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 51321.7us
I20260709 12:26:14.684329  8422 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:14.710758  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:14.719818  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:14.717455  4647 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:14.730974  4749 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:14.738605  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:14.729144  4598 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:14.737550  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:14.740634  4571 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:14.742412  8636 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:14.743021  8636 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:14.743471  8636 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:14.729022  8622 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:14.746145  8622 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:14.747337  8622 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.751170  8601 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:14.703852  8656 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:14.708626  8595 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:14.676489  8650 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.711730  8566 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:14.764460  4603 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:26:14.710323  8644 raft_consensus.cc:697] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:14.718519  8614 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:14.776243  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:14.778558  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:14.783406  4750 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:14.776482  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:14.777613  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 60262.5us
I20260709 12:26:14.751838  8519 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:14.752516  8350 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:14.781468  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:14.715649  8427 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:26:14.769279  8405 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.798861  4599 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:14.800561  5342 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:14.774011  8572 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:14.801891  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:14.812568  5345 raft_consensus.cc:2393] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 7: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:14.802228  5347 raft_consensus.cc:1217] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Deduplicated request from leader. Original: 5.6->[7.7-9.8]   Dedup: 7.7->[9.8-9.8]
I20260709 12:26:14.813088  5347 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: [7.7-9.8]
I20260709 12:26:14.783941  8621 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:26:14.805558  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 94881.5us
I20260709 12:26:14.841889  4719 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:14.842468  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:14.785063  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 195222us
I20260709 12:26:14.773180  8623 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:14.803745  5346 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:26:14.843081  4568 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:14.805737  8614 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.800587  8519 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.808653  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:14.709542  8612 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:14.840502  8435 consensus_meta.cc:287] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.002s
W20260709 12:26:14.853904  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47806.9us
I20260709 12:26:14.812242  8572 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.812371  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:14.843753  8350 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:14.844313  8623 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:14.853562  8519 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:14.857828  4569 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:14.814137  8626 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:14.909264  8445 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:14.849903  8293 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:14.784708  8633 consensus_queue.cc:237] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.849478  8621 raft_consensus.cc:697] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:14.955653  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 43861.4us
W20260709 12:26:14.955703  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 96359.6us
I20260709 12:26:14.990795  4749 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
W20260709 12:26:14.946651  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.959692  4675 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
W20260709 12:26:14.963204  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 107195us
I20260709 12:26:14.812913  8405 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.911635  4750 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:26:14.954878  8447 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:15.039966  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:14.980612  4751 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:14.922920  8435 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:14.962968  8593 consensus_meta.cc:287] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.893s	user 0.012s	sys 0.000s
I20260709 12:26:14.942274  8530 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:14.939525  8663 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 12:26:14.938438  8658 log.cc:927] Time spent T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:26:14.955741  8572 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:15.054337  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 91265.2us
I20260709 12:26:15.057431  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:15.079866  4675 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
W20260709 12:26:14.954643  8665 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:26:15.080540  4681 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:26:14.926165  8654 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 12:26:14.941251  8601 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.055853  8405 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:15.101689  8601 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:15.103318  8567 consensus_meta.cc:287] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.000s
I20260709 12:26:15.103819  8601 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:14.922408  8662 consensus_queue.cc:1059] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.248s
I20260709 12:26:15.055244  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:14.909516  8397 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.132s
W20260709 12:26:15.104961  8664 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.297s	user 0.000s	sys 0.000s
I20260709 12:26:14.906383  8656 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.187s
I20260709 12:26:14.951788  8382 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.123s
I20260709 12:26:15.108976  8572 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:14.956007  8644 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.963673  8623 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:15.113996  8572 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:15.115682  8614 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:15.116238  8628 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:15.116508  8621 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:14.922461  8350 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:15.117839  8350 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:15.118813  8650 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:15.121014  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 65878.8us
I20260709 12:26:15.121251  8593 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:15.077353  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.081287  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:15.152351  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 71140.6us
I20260709 12:26:15.082441  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:15.154237  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:15.154901  4747 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:14.906281  8422 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:15.049945  8679 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:26:15.051384  4571 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:15.162340  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 111037us
I20260709 12:26:15.103771  8567 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11.
I20260709 12:26:15.055507  8661 consensus_queue.cc:1059] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 12:26:15.055706  8516 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:15.065276  8629 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:15.124796  8485 consensus_queue.cc:1059] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:15.125437  8656 consensus_queue.cc:1243] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:26:15.128094  8649 consensus_meta.cc:287] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.669s	user 0.008s	sys 0.001s
I20260709 12:26:15.067005  8447 raft_consensus.cc:697] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:15.128736  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 276619us
I20260709 12:26:15.130528  8479 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:15.132366  8643 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.133060  8572 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.134377  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:15.134050  8623 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.135545  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:15.136682  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:15.136984  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:15.138001  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:15.145519  8350 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:15.150060  4752 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:15.268184  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:15.269225  4752 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:15.164155  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:15.167125  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:15.272104  4750 raft_consensus.cc:2376] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9. Re-sending same reply.
I20260709 12:26:15.168344  4751 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:15.169000  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:15.169770  4754 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:15.170249  4755 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:15.181977  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:15.182284  8678 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
W20260709 12:26:15.185686  8660 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.309s	user 0.001s	sys 0.002s
I20260709 12:26:15.185884  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:15.279635  8657 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:15.280146  5341 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:15.165277  8632 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:15.192113  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 109757us
I20260709 12:26:15.166980  8601 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:15.164765  8626 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:15.299237  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:15.243553  4569 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:15.276894  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:15.278760  5346 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:15.190299  8293 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.303332  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:15.303591  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:15.304276  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:15.305018  4751 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:15.306262  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:15.282124  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:15.283643  4680 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:15.280545  8657 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:15.294926  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:15.300035  4676 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:15.300681  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:15.311975  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:15.313462  4752 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:15.313911  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:15.314611  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:15.315177  4674 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:15.316512  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:15.316993  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:15.320295  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.253331  8650 consensus_queue.cc:1059] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:26:15.335970  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:15.331290  4747 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:15.333828  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:15.338652  4750 raft_consensus.cc:2393] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:15.285367  8593 raft_consensus.cc:3055] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:15.288458  8632 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:15.390797  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 87284.2us
I20260709 12:26:15.391937  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:15.299741  8567 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.394372  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:15.395442  4755 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:15.398572  4754 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:15.325992  8445 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:15.400462  4676 raft_consensus.cc:2410] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:15.317301  8293 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.401926  4676 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:26:15.402258  8674 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.138s	user 0.007s	sys 0.002s
I20260709 12:26:15.402805  8293 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:15.403949  8293 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:15.403879  8441 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:15.404979  8662 raft_consensus.cc:3055] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:15.275708  8643 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.275878  8479 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:15.662441  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.403968  4755 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:15.419163  4748 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 12:26:15.643616  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 400116us
W20260709 12:26:15.643807  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 251964us
I20260709 12:26:15.405382  8662 raft_consensus.cc:740] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:15.406721  4647 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:15.437335  4680 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:15.284403  8572 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.317155  8601 raft_consensus.cc:697] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:15.365057  8593 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:15.460311  8445 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:15.699587  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:15.582369  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:15.502030  5346 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:15.730204  5346 raft_consensus.cc:606] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:15.395079  8632 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:15.581629  8685 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
W20260709 12:26:15.736651  4677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1836) took 1804 ms (client timeout 1685 ms). Trace:
W20260709 12:26:15.455811  8677 log.cc:927] Time spent T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
W20260709 12:26:15.456494  8680 log.cc:927] Time spent T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
W20260709 12:26:15.454728  8683 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
I20260709 12:26:15.298295  8566 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:15.749788  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 50297.8us
I20260709 12:26:15.750576  4720 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:15.751847  8566 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:15.455046  8636 consensus_meta.cc:287] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.711s	user 0.008s	sys 0.004s
I20260709 12:26:15.597146  4604 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:15.752528  8636 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.418506  8647 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.712250  4680 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:26:15.413350  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.567546  8441 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.714710  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 308049us
W20260709 12:26:15.768100  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.564636  8661 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:15.478821  8623 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:15.623390  8485 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.734086  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:15.699731  8519 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:15.737095  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.737362  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:15.737151  4677 rpcz_store.cc:269] 0709 12:26:13.910229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.916654 (+  6425us) service_pool.cc:224] Handling call
0709 12:26:15.713713 (+1797059us) spinlock_profiling.cc:243] Waited 76 ms on lock 0x7b5400084b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:15.714070 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174742272}
I20260709 12:26:15.781903  8593 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.743758  4679 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:15.748610  8626 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:15.704943  4569 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:15.744948  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:15.753161  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:15.720000  8350 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:15.710927  8293 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:26:15.797194  4677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:26:15.798018  4677 raft_consensus.cc:606] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:15.733482  8629 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:15.736909  8649 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:15.798456  4677 raft_consensus.cc:614] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:15.798678  8649 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:15.799104  8649 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:15.799214  8629 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.779367  8441 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:15.803701  8441 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:15.803879  8662 consensus_queue.cc:260] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.805007  8662 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:15.805755  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 590) took 1226 ms. Trace:
I20260709 12:26:15.417716  8567 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
W20260709 12:26:15.785051  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.788127  8601 consensus_queue.cc:237] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.789583  8661 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:15.792119  8479 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:15.793116  8636 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:15.805858  4755 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:15.482259  8657 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:15.817850  4568 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.015s	sys 0.043s
W20260709 12:26:15.818819  4568 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.015s	sys 0.043s
I20260709 12:26:15.417426  8447 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:15.817646  8621 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.018s	sys 0.014s
W20260709 12:26:15.824585  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 71468.7us
I20260709 12:26:15.806209  4673 rpcz_store.cc:276] 0709 12:26:14.579796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.579930 (+   134us) service_pool.cc:224] Handling call
0709 12:26:14.601895 (+ 21965us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.612465 (+ 10570us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:14.664575 (+ 52110us) log.cc:844] Serialized 14 byte log entry
0709 12:26:15.110987 (+446412us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:15.120185 (+  9198us) log.cc:844] Serialized 26 byte log entry
0709 12:26:15.120326 (+   141us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:15.120540 (+   214us) log.cc:844] Serialized 14 byte log entry
0709 12:26:15.165197 (+ 44657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.165273 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:15.805458 (+640185us) raft_consensus.cc:1685] finished
0709 12:26:15.805498 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.805667 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69661,"prepare.queue_time_us":56423,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":91970,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":5982,"thread_start_us":381139,"threads_started":1,"wal-append.queue_time_us":557127}
I20260709 12:26:15.806429  8593 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:15.808733  8623 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.821967  8655 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.857s	user 0.009s	sys 0.006s
I20260709 12:26:15.818404  8350 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:15.776319  8647 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:15.836069  5346 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8e62ff81eb34853811314530131292b"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:15.836678  5346 raft_consensus.cc:606] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:15.837420  8647 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:15.822139  8693 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.832271  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:15.820449  8293 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:15.784511  8485 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.835171  8686 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:15.823773  8657 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:15.853065  4755 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:15.827518  8579 raft_consensus.cc:3055] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:15.855554  8567 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:15.855736  8579 raft_consensus.cc:740] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:15.856184  8567 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.856635  8579 consensus_queue.cc:260] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.857095  8293 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.857715  8579 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:15.859323  8293 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:15.859861  8293 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:15.860337  8293 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:15.861826  5346 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:15.862401  5346 raft_consensus.cc:606] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:26:15.862707  5346 raft_consensus.cc:614] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:15.863188  8485 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:15.863870  8668 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.818s	user 0.002s	sys 0.014s
I20260709 12:26:15.856683  8567 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:15.830197  8621 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.019s	sys 0.015s
I20260709 12:26:15.849325  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:15.827292  8688 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:15.852773  8636 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:15.853032  8657 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:15.851600  8686 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:15.878549  5346 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:15.888298  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:15.888811  5346 raft_consensus.cc:606] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:26:15.889300  5346 raft_consensus.cc:614] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:15.833650  8623 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:15.877954  8481 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.851907  8693 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.895700  8445 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:15.899979  8422 raft_consensus.cc:2764] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:26:15.900389  8445 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:15.887167  8688 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:15.887313  8657 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:15.883800  8567 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:15.907491  8293 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:15.890597  8686 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:15.920890  8632 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:15.923234  8632 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:15.942543  8661 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:15.971407  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:15.975040  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.910413  8649 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:15.958642  8036 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:15.979406  8670 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:15.979724  8485 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:15.977662  8632 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.995812  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 290895us
I20260709 12:26:15.981946  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:15.959779  8519 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:15.910106  8481 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:15.998934  8647 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:16.003573  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.021695  4681 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 6.
I20260709 12:26:16.004308  8649 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.004482  4569 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:16.006296  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.023093  4677 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.023275  5312 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:16.024609  8441 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:16.025110  8441 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.989859  8036 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.007753  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:16.025650  8441 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:16.026646  4647 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.027365  8036 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.027827  4647 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.008042  8623 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:16.008828  8584 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:15.984570  8644 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:26:16.038460  8644 consensus_queue.cc:1243] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:26:16.038535  8693 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:15.977380  8661 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:26:16.041079  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.001796  8688 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:16.050783  4677 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
W20260709 12:26:16.051700  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 47375.9us
I20260709 12:26:16.052675  4569 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:16.053315  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:16.058140  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 159980us
I20260709 12:26:16.042119  4490 maintenance_manager.cc:419] P edba2778b94f4e5aa07d5f710879b375: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033440
I20260709 12:26:16.059054  4681 pending_rounds.cc:85] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 10
I20260709 12:26:16.064590  4681 pending_rounds.cc:107] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:26:16.064953  4681 raft_consensus.cc:2889] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:16.058974  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.055778  8450 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:26:15.999402  8632 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.000996  8485 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:16.051275  8649 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:16.051983  8626 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:16.050922  8479 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:16.055464  8584 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:16.054785  8647 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:16.079632  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.056087  8643 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.060412  8645 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:16.083098  8629 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.058861  8688 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:16.057542  8519 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:16.082775  8652 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:16.087914  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:16.093703  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.116257  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.089445  8479 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:16.054394  8644 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:26:16.114204  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.096514  4602 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:26:16.117336  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.119415  4673 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:26:16.118983  4599 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: [7.7-9.8]
I20260709 12:26:16.119227  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.119701  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:16.084837  8447 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:16.117363  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 76369.8us
I20260709 12:26:16.132236  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.115790  8629 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.136505  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.137357  8647 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.122983  8479 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.138008  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.138374  8629 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:16.139273  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.135591  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:16.114403  8519 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:16.143098  8612 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.143186  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.143929  8422 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:16.144343  8422 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:16.144721  8422 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.144760  8701 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:16.145928  8701 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:16.146513  4749 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2765) took 1536 ms (client timeout 2037 ms). Trace:
I20260709 12:26:16.146438  8701 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:16.146970  4749 rpcz_store.cc:269] 0709 12:26:14.610156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.615014 (+  4858us) service_pool.cc:224] Handling call
0709 12:26:16.146327 (+1531313us) spinlock_profiling.cc:243] Waited 67.9 ms on lock 0x7b540004c754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:16.146432 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156061952}
I20260709 12:26:16.148445  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 594) took 1632 ms. Trace:
I20260709 12:26:16.148705  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.149433  8701 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.132902  8652 raft_consensus.cc:697] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 12 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:16.150224  4596 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:16.150470  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.150842  8652 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:16.151602  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.152299  4674 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:16.152639  4596 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 12:26:16.150730  4647 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.153028  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.102865  8623 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:16.154762  4674 raft_consensus.cc:2410] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:26:16.148816  4604 rpcz_store.cc:276] 0709 12:26:14.516312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.538249 (+ 21937us) service_pool.cc:224] Handling call
0709 12:26:16.148256 (+1610007us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b5400344254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:16.148356 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390822656}
I20260709 12:26:16.158464  4602 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:26:16.118052  8694 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:26:16.161972  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:16.162575  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.143488  8643 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:16.116415  8700 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.155660  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.170917  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.134223  8636 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:16.098892  8626 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.118276  8645 raft_consensus.cc:697] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:16.156500  8707 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.066s	user 0.002s	sys 0.000s
I20260709 12:26:16.170064  4600 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:16.143151  8519 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:16.161453  8647 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:16.161633  8036 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:16.154187  8702 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:16.171552  4674 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:16.177183  8649 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:16.171589  4596 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:16.187427  8485 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.227973  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:16.229635  4747 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.169404  8643 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.190685  8439 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.191962  8632 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.198261  8481 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.101855  8706 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.077s
W20260709 12:26:16.204643  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:16.207283  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 87649.8us
I20260709 12:26:16.211742  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:16.118384  8688 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.213867  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:16.252316  4752 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:16.213867  4749 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:26:16.226895  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:16.189921  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:16.230602  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.254820  4596 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:16.236078  8643 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:16.237685  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:16.156666  8447 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.242208  8716 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.141465  8686 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.244923  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:16.274852  4750 raft_consensus.cc:2410] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:26:16.199064  8584 consensus_queue.cc:1059] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:26:16.170918  8701 consensus_queue.cc:1059] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:26:16.247702  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.248592  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:16.249043  4751 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:16.228864  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.254539  4647 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:16.259467  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:16.261899  4756 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:16.199880  8422 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.282934  4601 raft_consensus.cc:2410] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:26:16.230475  8441 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.282003  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:16.236903  8036 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.291000  4754 raft_consensus.cc:2393] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:16.292163  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:16.293207  4749 raft_consensus.cc:2393] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:16.294744  4600 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:16.240130  8439 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.301735  8636 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:16.302328  4597 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:16.290457  8584 raft_consensus.cc:3055] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:16.303063  8584 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:16.303922  8584 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:16.306094  8710 log.cc:927] Time spent T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:26:16.306772  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:16.307677  8712 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:16.307621  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.308995  8623 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:16.310652  8662 consensus_meta.cc:287] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.505s	user 0.013s	sys 0.000s
I20260709 12:26:16.311643  4673 raft_consensus.cc:2410] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:26:16.286068  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.312685  4601 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:26:16.199569  8700 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.308940  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.303949  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.319239  4680 raft_consensus.cc:2410] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:26:16.302310  8636 raft_consensus.cc:697] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:16.286841  8481 raft_consensus.cc:3055] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:16.290259  8350 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:16.323779  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.325878  4602 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:16.326797  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.328064  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.328603  8422 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:16.318317  8679 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.328888  8036 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.279016  4747 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:16.335712  4645 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.331643  4680 raft_consensus.cc:2410] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:16.337368  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.312525  8441 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.315737  8584 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:16.330523  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:16.348572  4597 raft_consensus.cc:2368] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11.
I20260709 12:26:16.334930  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.338409  4599 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:16.350557  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.339591  4680 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:16.324753  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 70300.7us
I20260709 12:26:16.342880  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.353051  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.353694  4601 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
W20260709 12:26:16.357219  8622 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.242s	user 0.006s	sys 0.041s
W20260709 12:26:16.357741  8622 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.242s	user 0.007s	sys 0.041s
I20260709 12:26:16.346748  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.374508  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:16.374656  8701 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:16.375865  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:16.331478  8439 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.377079  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:16.377858  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.383709  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 644) took 1883 ms. Trace:
I20260709 12:26:16.323447  8481 raft_consensus.cc:740] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:16.334630  8686 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:16.398576  4602 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:16.398742  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:16.356341  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:16.437743  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 81461.7us
I20260709 12:26:16.439363  4647 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:16.332446  8350 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.411826  4597 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:16.415680  4680 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:26:16.384568  8649 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:26:16.329530  8636 consensus_queue.cc:237] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.428829  8706 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:16.432348  4569 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:16.333487  8447 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:16.411345  8652 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:16.334820  8700 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.377063  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:16.451244  4597 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:26:16.451731  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 74763.8us
I20260709 12:26:16.314803  8657 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.343896  8579 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:16.285764  8716 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.442727  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.443285  8686 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:16.439662  8481 consensus_queue.cc:260] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.439565  8720 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:16.448777  4569 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:16.458777  4603 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:16.459810  4569 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:26:16.434429  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.432682  4753 rpcz_store.cc:276] 0709 12:26:14.499973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.500222 (+   249us) service_pool.cc:224] Handling call
0709 12:26:14.500588 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:14.754076 (+253488us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:14.762731 (+  8655us) log.cc:844] Serialized 14 byte log entry
0709 12:26:15.798529 (+1035798us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b180039f5e0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:26:15.984215 (+185686us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:15.984286 (+    71us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:15.984743 (+   457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.383390 (+398647us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400117e54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:16.383460 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:16.383637 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":152,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":2293,"raft.queue_time_us":117,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":2710,"spinlock_wait_cycles":588869120,"thread_start_us":185345,"threads_started":1,"wal-append.queue_time_us":1275685}
I20260709 12:26:16.446269  8350 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:16.472422  4674 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:16.331272  8679 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.277318  8645 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.445780  8702 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:16.457290  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:16.476228  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.454262  8711 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:16.469210  8657 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:16.455206  8686 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:16.454336  8662 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:26:16.451185  8447 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:16.477859  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.480952  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.481175  4676 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
W20260709 12:26:16.483407  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.446358  8479 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.463608  8714 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:16.472965  8622 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:16.496716  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.449658  8706 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.458848  8720 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:16.477921  8686 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:16.514412  4677 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:16.507350  4674 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:16.479961  8623 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:16.479709  8422 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:16.482733  8702 raft_consensus.cc:697] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:16.494593  8445 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:16.534696  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.512300  8720 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:16.550153  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 94542.1us
I20260709 12:26:16.527731  8422 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.527259  8404 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.551757  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.555868  4674 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:16.552050  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.527413  8688 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:16.526055  8652 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:16.560874  8479 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:16.561347  8671 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.631s	user 0.019s	sys 0.006s
I20260709 12:26:16.563884  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:16.555083  8422 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.572746  4672 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:26:16.597620  4672 pending_rounds.cc:85] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 7
I20260709 12:26:16.597874  4672 pending_rounds.cc:107] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:26:16.598135  4672 raft_consensus.cc:2889] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:16.565845  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.598775  4673 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:16.578083  8647 raft_consensus.cc:3055] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:16.578233  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:16.572790  8702 consensus_queue.cc:237] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:16.573338  8723 log.cc:927] Time spent T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:16.578572  8679 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.578795  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:16.603379  4754 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:16.603592  4674 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:16.582347  8720 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.600093  8647 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:16.583961  8706 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:16.590294  4645 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.607741  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:16.591013  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.571869  8519 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:16.595438  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 114540us
I20260709 12:26:16.559098  8404 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.612648  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:16.613176  4754 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:16.614393  8404 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:16.605240  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 585) took 2050 ms. Trace:
I20260709 12:26:16.605110  8647 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.615514  8519 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.616055  8647 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:16.615010  4678 rpcz_store.cc:276] 0709 12:26:14.554699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.582761 (+ 28062us) service_pool.cc:224] Handling call
0709 12:26:14.583081 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.603018 (+ 19937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.603056 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.613829 (+ 10773us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.804010 (+190181us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:14.804081 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.804218 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:16.560766 (+1756548us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:16.604921 (+ 44155us) raft_consensus.cc:1685] finished
0709 12:26:16.604945 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:16.605164 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11348,"raft.queue_time_us":11895,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":15632,"spinlock_wait_cycles":31997696,"thread_start_us":118953,"threads_started":1,"wal-append.queue_time_us":303889}
I20260709 12:26:16.618083  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.619081  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.620064  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.621023  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.601948  8721 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.621951  4754 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
W20260709 12:26:16.610736  8725 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:16.611960  8720 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:16.612195  8643 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.630549  8688 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:16.632225  8618 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:16.632845  8643 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.621515  4678 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:16.624094  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.634122  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:16.626856  4569 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:16.634588  8652 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:16.635699  4569 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:16.634619  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.637382  4645 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.637730  4673 raft_consensus.cc:2410] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:26:16.638067  8657 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:16.638458  8657 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:16.639242  8519 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:16.638828  8657 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:16.643110  8716 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:16.644526  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:16.646483  4569 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:16.606246  8650 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:16.648720  4569 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:16.629037  8679 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:16.621946  8721 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.661705  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1845) took 1525 ms. Trace:
I20260709 12:26:16.662062  4674 raft_consensus.cc:2368] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11.
I20260709 12:26:16.651871  8716 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:16.654835  8481 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:16.654968  4645 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:16.655722  8643 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.659400  8485 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:16.663534  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:16.662120  4679 rpcz_store.cc:276] 0709 12:26:15.136613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.136740 (+   127us) service_pool.cc:224] Handling call
0709 12:26:16.661509 (+1524769us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b540007bf54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:16.661645 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1095869824}
I20260709 12:26:16.646104  8623 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:16.666179  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:16.666759  8720 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:16.675458  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.676152  4679 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:16.647037  8652 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:16.660328  8688 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:16.683055  4645 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.662313  8519 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:16.685793  4645 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.686522  8729 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:16.695326  8647 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:16.697556  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:16.698118  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.698675  4681 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:16.665095  8601 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:16.699529  4644 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:16.691116  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.704972  4680 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:16.677213  8720 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.707242  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.707808  4603 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:16.706614  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:16.709827  4569 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:16.700248  8519 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.711189  4600 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:16.711211  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:16.705713  8669 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.713088  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:16.713388  8669 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.712888  8519 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.714121  8531 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:16.714537  8531 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:16.714915  8531 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:16.715508  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:16.713749  8593 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:16.700740  8716 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:16.700055  8652 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.718438  4748 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:16.675834  8643 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.723690  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:16.724355  5346 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:16.708500  8601 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:16.723291  8700 consensus_queue.cc:1059] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:26:16.726562  8700 raft_consensus.cc:3055] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:16.715080  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:16.726963  8700 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:16.726328  8601 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.727813  8700 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.728904  8700 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.715022  8500 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:16.730058  8500 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.730458  8500 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:16.709543  8720 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:16.723985  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.746129  4679 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:16.747323  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:16.723081  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:16.749910  8622 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:16.750465  8622 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.751209  8622 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:16.757485  4569 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:16.721127  8716 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.770053  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:16.770948  4677 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:26:16.776532  4681 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:16.778929  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.779251  4569 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:16.733530  8572 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.794550  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:16.795248  5346 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:16.796070  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:16.769513  8733 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:16.774871  8716 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:16.786149  8516 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.522s	user 0.018s	sys 0.020s
I20260709 12:26:16.769783  8629 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:16.780280  8479 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:16.798908  4678 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:16.799314  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.810228  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.799374  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:16.786163  8647 consensus_queue.cc:1059] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:26:16.815157  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:16.813622  4681 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:16.785312  8531 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.818569  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:16.820072  8435 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:26:16.804629  8516 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.556s	user 0.018s	sys 0.021s
I20260709 12:26:16.825206  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:16.809268  8479 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.828917  4756 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:16.780995  8422 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.815002  8629 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:16.823203  4681 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:26:16.824056  8516 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:16.826678  8623 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:16.830010  8479 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.861969  8531 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:16.863713  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:16.865108  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:16.865521  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:16.869027  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:16.811807  8733 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.837878  8669 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:16.899104  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:16.830984  8657 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.841845  8435 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:16.910912  8720 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.913933  5345 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:16.841262  8350 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.918892  5347 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:16.926632  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:16.903345  8441 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:16.932693  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:16.934128  4748 raft_consensus.cc:2410] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:16.935557  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:16.941455  4644 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:16.923271  8743 log.cc:927] Time spent T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:26:16.897604  8737 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:26:16.950244  4754 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:16.944998  4719 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:16.952049  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:16.952256  8642 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:16.914530  8686 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:16.954950  8686 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:16.959718  8350 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:16.962015  8716 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:16.964084  8729 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:16.967615  4677 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:16.967909  5347 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:16.969291  8716 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:16.970813  8682 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 1.722s	user 0.003s	sys 0.009s
W20260709 12:26:16.972288  8684 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 1.855s	user 0.010s	sys 0.032s
I20260709 12:26:16.974985  8422 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:16.949927  8531 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:26:16.986188  4681 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:16.964295  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 586) took 2409 ms. Trace:
I20260709 12:26:16.989806  4672 rpcz_store.cc:276] 0709 12:26:14.554984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.582589 (+ 27605us) service_pool.cc:224] Handling call
0709 12:26:14.593035 (+ 10446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.599869 (+2006834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.963907 (+364038us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54001bc954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:16.964191 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4552041216}
I20260709 12:26:16.992964  4672 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:16.965251  4719 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:16.938005  8447 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:16.953004  8642 raft_consensus.cc:697] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:17.000569  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:17.005026  4679 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:16.942545  8643 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:17.011523  4748 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:26:17.000049  8531 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:17.009306  8652 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:16.985314  8500 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.030679  4672 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:16.950426  8441 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.001082  8422 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:26:17.039948  8720 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:17.039538  8642 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.040810  8733 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:17.043546  4571 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:17.049299  4673 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:26:16.942096  8629 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:17.054258  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 112877us
I20260709 12:26:17.054550  4673 pending_rounds.cc:85] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 4
I20260709 12:26:17.055121  4673 pending_rounds.cc:107] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:26:17.055377  4673 raft_consensus.cc:2889] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:16.994486  8516 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.057421  8729 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:17.065779  8572 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:17.066340  8572 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.068399  8572 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:17.070443  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:17.072289  4755 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:17.076341  4747 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:17.077483  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:17.082809  8657 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:17.065133  8447 consensus_queue.cc:1059] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:26:17.085053  8447 raft_consensus.cc:3055] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:17.085405  8447 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:17.082547  8700 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:26:17.076270  8673 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 2.146s	user 0.032s	sys 0.062s
I20260709 12:26:17.116504  8479 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.117583  8447 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:17.125845  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 82370.2us
I20260709 12:26:17.136348  4748 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:17.137259  4571 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:17.148904  4751 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:17.151104  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:17.151916  4673 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:17.153515  4749 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:17.144884  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.158303  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.160081  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:17.161854  8036 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:17.162398  4571 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:17.164309  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:17.169749  8720 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:17.170275  8720 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.172441  8720 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:17.173827  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:17.175419  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.176582  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.177214  4750 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:17.177906  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.178966  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.180006  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:17.180518  4752 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.186220  8737 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:17.186810  8737 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:17.187706  8737 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.039146  8435 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.193159  8500 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:17.069759  8601 consensus_queue.cc:1059] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:26:17.196820  8601 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:17.197206  8601 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:17.198119  8601 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.199348  8601 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:17.200326  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:17.070097  8627 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 12:26:17.202064  8627 raft_consensus.cc:3055] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:26:17.197108  8681 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 1.902s	user 0.004s	sys 0.006s
I20260709 12:26:17.202416  8627 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:17.203297  8627 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.204334  8728 raft_consensus.cc:3055] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:17.204553  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1837) took 2483 ms. Trace:
I20260709 12:26:17.204782  8728 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:17.122035  8622 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:17.204957  4605 rpcz_store.cc:276] 0709 12:26:14.721057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.721183 (+   126us) service_pool.cc:224] Handling call
0709 12:26:14.756650 (+ 35467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.768511 (+ 11861us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:14.768561 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.769187 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 12:26:15.009027 (+239840us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:15.009085 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:15.009182 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.196564 (+2187382us) spinlock_profiling.cc:243] Waited 7.85 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:17.204256 (+  7692us) raft_consensus.cc:1685] finished
0709 12:26:17.204275 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.204468 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4152,"raft.queue_time_us":981,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":18800,"spinlock_wait_cycles":18047872,"thread_start_us":197156,"threads_started":1,"wal-append.queue_time_us":315883}
I20260709 12:26:17.206720  8622 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:17.102322  8519 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.207430  4674 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 12:26:17.208858  4674 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:26:17.209175  4674 pending_rounds.cc:85] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 5
I20260709 12:26:17.209386  4674 pending_rounds.cc:107] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:26:17.201493  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.212214  4753 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:17.155571  8700 consensus_queue.cc:1243] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:17.213729  8728 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.214706  8479 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:17.199250  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.215644  4568 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:17.216763  5340 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:17.217509  8623 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:17.217921  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:17.218443  8623 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.218628  8441 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:17.218817  8623 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:17.219514  8485 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:17.219960  8485 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.220844  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.221009  8721 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:17.226965  8645 raft_consensus.cc:993] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:26:17.227120  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:17.227375  8519 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:17.227522  8645 raft_consensus.cc:1081] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:26:17.228367  8447 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.217922  4568 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:17.229529  4568 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:17.229753  8622 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:17.230356  8622 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:17.230826  8622 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.231194  8622 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.229298  8626 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:17.231572  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:17.246433  4679 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:17.233126  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:17.235322  4601 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:26:17.252758  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.257459  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:17.258072  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:17.260210  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:17.263146  4647 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:17.237855  4672 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:17.236323  4674 raft_consensus.cc:2889] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:17.228585  5312 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:17.214778  8728 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:17.243800  4603 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:17.232827  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:17.236029  4720 leader_election.cc:400] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11.
I20260709 12:26:17.279683  4720 leader_election.cc:403] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:26:17.279654  4602 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:17.238970  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:17.233960  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:17.280541  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.239573  4752 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:17.269984  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:17.282192  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:17.282685  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:17.283484  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:17.283888  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:17.284225  5341 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:17.269837  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:17.279089  4571 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:17.297319  4673 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
W20260709 12:26:17.299415  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63531.6us
I20260709 12:26:17.300446  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:17.302836  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:17.281147  4747 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:17.309044  4747 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:17.312661  4679 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:17.314154  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:17.315415  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.317159  4747 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:17.319130  4752 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:17.319474  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.156064  8692 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:17.326493  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47491.7us
I20260709 12:26:17.204499  8627 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.330566  4571 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:17.335742  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.337260  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:17.339497  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:17.284607  8643 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:17.335421  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:17.340284  4674 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:17.350195  4752 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:17.350334  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.354707  4753 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:17.355104  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:17.355952  4674 raft_consensus.cc:2368] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12.
I20260709 12:26:17.344707  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:17.357101  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 282492us
I20260709 12:26:17.362504  4720 leader_election.cc:400] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11.
I20260709 12:26:17.299770  8036 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:17.306495  8720 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:17.377449  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:17.381549  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:17.382383  4602 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:17.387720  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:17.388208  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:17.388466  4676 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:17.227409  8615 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.109s
I20260709 12:26:17.397360  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:17.322114  8626 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:17.332212  8479 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:17.321947  8733 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11.
I20260709 12:26:17.331743  8714 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:17.414052  8714 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:17.414552  8714 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:17.415431  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 593) took 2256 ms. Trace:
I20260709 12:26:17.415799  4675 rpcz_store.cc:276] 0709 12:26:15.159226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.323015 (+163789us) service_pool.cc:224] Handling call
0709 12:26:15.323411 (+   396us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:15.389827 (+ 66416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:15.389877 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:16.895795 (+1505918us) spinlock_profiling.cc:243] Waited 6.6 ms on lock 0x7b2c00010c48. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:26:16.902259 (+  6464us) log.cc:844] Serialized 52 byte log entry
0709 12:26:17.050634 (+148375us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:17.050667 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.050746 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.415140 (+364394us) raft_consensus.cc:1685] finished
0709 12:26:17.415157 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.415371 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8083,"raft.queue_time_us":71787,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":18089,"spinlock_wait_cycles":15175040,"thread_start_us":122474,"threads_started":1,"wal-append.queue_time_us":419748}
I20260709 12:26:17.418108  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:17.418826  4675 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.331442  8593 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 12:26:17.349263  8633 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 12:26:17.241309  8485 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.341813  8500 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:17.425894  8485 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:17.425694  8642 consensus_queue.cc:1059] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:17.398379  8744 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:17.426811  8744 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:17.427201  8744 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:17.427412  8036 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:17.428385  4675 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:17.428606  8036 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:17.429095  8036 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:17.389598  8566 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.429945  8566 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.430462  8623 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.432360  8566 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:17.387161  8729 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:17.347968  8531 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 12:26:17.433454  8633 raft_consensus.cc:3055] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:17.385490  8649 consensus_queue.cc:1059] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:26:17.433842  8633 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:17.330317  8629 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:17.443382  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:17.444054  4672 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:17.408201  4752 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:26:17.409600  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:17.411875  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:17.331238  8422 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 12:26:17.433774  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:17.402197  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.436756  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:17.400722  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:17.429466  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:17.457854  4675 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:17.449450  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.376320  8519 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.480146  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:17.481153  4675 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:17.476895  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:17.483783  4673 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:17.478219  4569 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:17.486243  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:17.491101  4569 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:17.491931  4675 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:17.493436  4675 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:17.497560  4673 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:17.492262  4569 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:17.387563  8686 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 12:26:17.509205  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:17.510950  4675 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:26:17.512526  4751 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 12:26:17.512523  4675 pending_rounds.cc:85] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 5
I20260709 12:26:17.513374  4675 pending_rounds.cc:107] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:26:17.513725  4675 raft_consensus.cc:2889] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:17.519486  4672 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:17.522827  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:17.403625  8584 consensus_queue.cc:1059] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 12:26:17.451566  8479 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:17.528237  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:17.442269  8350 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:26:17.508155  8629 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:17.507928  8657 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:17.563158  4751 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:17.435122  8622 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.549350  4603 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:17.578830  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:17.580574  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:17.581211  4603 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:17.582015  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:17.582973  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:17.528878  8500 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:17.504217  8531 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:17.510123  8649 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.540036  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:17.557297  4602 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:17.604485  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:17.560058  4755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8e62ff81eb34853811314530131292b"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:42632
I20260709 12:26:17.574368  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:17.605965  4599 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:17.486047  8618 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:26:17.575788  4674 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:17.608322  4747 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 12:26:17.550034  4675 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: []
W20260709 12:26:17.589115  8747 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:26:17.614048  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:17.614812  4751 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:17.589926  8748 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:26:17.596501  8716 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:17.434903  8633 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.490199  8519 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.610985  4755 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:17.629465  4755 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:17.612058  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.560819  8693 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:17.608569  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 323162us
I20260709 12:26:17.589700  8481 raft_consensus.cc:2955] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Committing config change with OpId 7.7: config changed from index 3 to 7, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:17.617609  4675 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:17.616334  8593 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.623996  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:17.625100  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:17.631412  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:17.637059  5345 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:17.639130  4719 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:17.607124  8485 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.648345  4747 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:17.655038  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 291499us
I20260709 12:26:17.655745  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:17.653782  4605 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:17.618079  8623 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:17.617204  5341 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:17.661309  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:17.672453  4754 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:26:17.673735  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:17.679873  4751 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:17.614238  8649 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:17.718982  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 57624.7us
I20260709 12:26:17.576550  8036 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.682343  4755 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.620081  8531 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.659893  8650 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:17.668407  8633 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:17.733596  4747 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:17.733053  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:17.741990  4749 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:17.656284  8593 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.660151  8693 raft_consensus.cc:697] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:17.745162  8479 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.635156  8716 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.748272  8693 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:17.749627  8593 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:17.733814  8661 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:26:17.750553  8716 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:17.667198  8485 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.751099  8661 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:17.751523  8661 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:17.752681  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2798) took 1586 ms. Trace:
I20260709 12:26:17.752414  8661 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.752938  8485 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:17.753152  4680 rpcz_store.cc:276] 0709 12:26:16.165818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.707317 (+541499us) service_pool.cc:224] Handling call
0709 12:26:16.707664 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:16.716716 (+  9052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:16.716760 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:16.723506 (+  6746us) log.cc:844] Serialized 26 byte log entry
0709 12:26:16.974702 (+251196us) spinlock_profiling.cc:243] Waited 6.62 ms on lock 0x7b18002b3240. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:26:17.122516 (+147814us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:17.122559 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.122629 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.742981 (+620352us) raft_consensus.cc:1685] finished
0709 12:26:17.743008 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.752611 (+  9603us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8583,"raft.queue_time_us":10348,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":2940,"spinlock_wait_cycles":15231872,"thread_start_us":147594,"threads_started":1,"wal-append.queue_time_us":594312}
I20260709 12:26:17.747758  8036 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:17.752784  8531 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:17.758589  8751 log.cc:927] Time spent T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:26:17.759372  8531 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:17.760093  8753 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:26:17.760172  8531 consensus_queue.cc:1059] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:17.761320  8531 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:26:17.761658  8749 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:26:17.761969  8531 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:26:17.763365  8752 log.cc:927] Time spent T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:26:17.762984  8531 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.764611  4755 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:17.673364  8584 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:26:17.761394  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1858) took 1019 ms. Trace:
I20260709 12:26:17.777887  4678 rpcz_store.cc:276] 0709 12:26:16.741699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.068726 (+327027us) service_pool.cc:224] Handling call
0709 12:26:17.069085 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.069425 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.069456 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.083913 (+ 14457us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.262844 (+178931us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:17.262886 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.262952 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.761103 (+498151us) raft_consensus.cc:1685] finished
0709 12:26:17.761131 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.761325 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7603,"raft.queue_time_us":4368,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":177768,"threads_started":1,"wal-append.queue_time_us":423934}
I20260709 12:26:17.784298  4571 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:17.662751  8516 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 12:26:17.805670  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:17.795172  8661 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:17.818253  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.770524  8623 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:17.780534  8615 raft_consensus.cc:3055] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:17.822400  8626 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:17.877369  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 329441us
I20260709 12:26:17.823024  8650 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:17.837704  8615 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:17.833101  8629 raft_consensus.cc:697] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:17.850507  8623 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:17.851547  8627 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:26:17.919654  8629 consensus_queue.cc:237] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:17.932325  4596 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:17.942787  8519 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:17.944326  8519 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:26:17.881558  8763 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.133s
I20260709 12:26:17.898923  8744 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.883638  8626 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:17.911321  8686 raft_consensus.cc:3055] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:18.028941  4599 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
W20260709 12:26:17.997359  5314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.023s	sys 0.031s
W20260709 12:26:18.029783  8647 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.012s	sys 0.008s
I20260709 12:26:17.918511  8622 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:17.996762  8759 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 12:26:18.021041  8714 consensus_meta.cc:287] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.606s	user 0.008s	sys 0.003s
W20260709 12:26:18.042213  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 386512us
W20260709 12:26:17.996717  8756 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:26:17.889284  8650 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:18.028187  8757 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
W20260709 12:26:18.039505  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 404354us
I20260709 12:26:18.064460  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:18.037909  8447 consensus_meta.cc:287] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.809s	user 0.009s	sys 0.002s
I20260709 12:26:17.916105  8764 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.127s
I20260709 12:26:18.084719  4603 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:18.085515  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:18.086172  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:18.090569  4747 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:18.071713  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:18.073786  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:18.069706  5314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.026s	sys 0.042s
W20260709 12:26:18.060776  8647 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.012s	sys 0.008s
I20260709 12:26:18.033293  8500 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:18.093621  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 597606us
I20260709 12:26:18.100505  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:18.087514  5312 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:18.096267  4601 raft_consensus.cc:2410] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:18.065701  8633 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.104142  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:18.104911  4750 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:18.101241  4748 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:26:18.063794  8776 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:18.039160  8627 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
W20260709 12:26:18.098095  8601 consensus_meta.cc:287] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.895s	user 0.012s	sys 0.002s
I20260709 12:26:18.041239  8686 raft_consensus.cc:740] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:18.071157  8623 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:18.115084  8754 log.cc:927] Time spent T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.566s	user 0.007s	sys 0.000s
I20260709 12:26:18.116882  8650 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:18.105932  8643 consensus_meta.cc:287] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.716s	user 0.011s	sys 0.000s
I20260709 12:26:18.118021  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2814) took 1695 ms. Trace:
I20260709 12:26:18.119928  8745 raft_consensus.cc:3055] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:18.121493  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2810) took 1796 ms. Trace:
I20260709 12:26:18.077857  8765 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:26:18.147094  4677 rpcz_store.cc:276] 0709 12:26:16.325456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.968729 (+643273us) service_pool.cc:224] Handling call
0709 12:26:16.969148 (+   419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:17.063194 (+ 94046us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.063217 (+    23us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:17.072021 (+  8804us) log.cc:844] Serialized 52 byte log entry
0709 12:26:17.476608 (+404587us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:17.476648 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.476715 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.121242 (+644527us) raft_consensus.cc:1685] finished
0709 12:26:18.121268 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.121420 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97855,"raft.queue_time_us":90399,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":7357,"thread_start_us":330038,"threads_started":1,"wal-append.queue_time_us":683069}
I20260709 12:26:18.134624  4602 raft_consensus.cc:2410] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:18.138754  4750 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:18.106290  8649 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:18.129514  8447 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:18.141117  4681 rpcz_store.cc:276] 0709 12:26:16.422783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.988216 (+565433us) service_pool.cc:224] Handling call
0709 12:26:16.988641 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.139216 (+150575us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:17.157495 (+ 18279us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.240689 (+ 83194us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b180044f680. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:26:17.677560 (+436871us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.730039 (+ 52479us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.730234 (+   195us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:17.730507 (+   273us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.730707 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.730810 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.117774 (+386964us) raft_consensus.cc:1685] finished
0709 12:26:18.117802 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.117955 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":135180,"prepare.queue_time_us":20039,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":86,"raft.queue_time_us":203462,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":24529,"spinlock_wait_cycles":90945408,"thread_start_us":462352,"threads_started":1,"wal-append.queue_time_us":669597}
I20260709 12:26:18.157015  8422 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.157682  8422 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.159235  4672 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:18.161656  8784 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:18.162585  8645 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:18.162874  8784 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.164228  8729 raft_consensus.cc:3055] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:18.164659  8729 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:18.170361  8733 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:18.170929  8733 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:18.177641  8766 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:18.178138  8766 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.178458  8766 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:18.160614  8422 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:18.186179  8755 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.369s	user 0.000s	sys 0.008s
I20260709 12:26:18.138764  8441 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:18.071363  8615 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.211844  4571 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:18.151117  8643 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:18.146889  8745 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:18.141829  8772 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:18.150520  8773 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.111363  8633 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:18.225342  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.194690  8566 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:18.215306  8764 raft_consensus.cc:993] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
W20260709 12:26:18.256222  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.283397  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 71682.3us
I20260709 12:26:18.136373  8770 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 12:26:18.220572  8642 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.298194  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1856) took 1063 ms. Trace:
I20260709 12:26:18.299150  8435 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:18.299829  8435 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.300390  8744 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:18.298667  4600 rpcz_store.cc:276] 0709 12:26:17.234979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.241410 (+  6431us) service_pool.cc:224] Handling call
0709 12:26:17.241725 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.241923 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:17.242264 (+   341us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.216860 (+974596us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.297756 (+ 80896us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:18.297823 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:18.297896 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.297983 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.298123 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80610,"prepare.queue_time_us":22391,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":77,"raft.queue_time_us":114904,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":48943,"thread_start_us":973744,"threads_started":3,"wal-append.queue_time_us":503138}
I20260709 12:26:18.217773  8479 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:18.301044  8744 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:18.232259  8772 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.076812  8714 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.163479  8784 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.207401  8447 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.352613  4750 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:18.356355  4751 raft_consensus.cc:3055] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:18.356869  4751 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:18.292074  8773 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.359740  8784 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:18.360474  8773 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:18.361333  8661 consensus_meta.cc:287] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.535s	user 0.009s	sys 0.002s
W20260709 12:26:18.362391  8778 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:26:18.363162  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:18.364099  4749 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:18.365926  4600 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:18.366487  8441 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:18.367038  8441 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:18.372118  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2301) took 1127 ms. Trace:
I20260709 12:26:18.372701  4601 rpcz_store.cc:276] 0709 12:26:17.244824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.283654 (+ 38830us) service_pool.cc:224] Handling call
0709 12:26:18.371841 (+1088187us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b5400034654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:18.372035 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1866065664}
I20260709 12:26:18.374384  8643 raft_consensus.cc:3055] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:18.308156  8615 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:18.340767  8761 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:26:18.218986  8686 consensus_queue.cc:260] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.305346  8519 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.388792  4753 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:18.389104  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1858) took 1145 ms. Trace:
I20260709 12:26:18.206115  8733 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.165561  8729 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.302624  8782 raft_consensus.cc:3055] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:18.304076  8627 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:18.284308  8781 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:18.280236  8500 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:18.410027  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 310330us
I20260709 12:26:18.412050  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:18.408272  4751 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:18.463135  4751 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:18.412513  8643 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:18.414512  8441 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.415541  8762 raft_consensus.cc:1081] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:26:18.416680  8789 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:18.388494  8700 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.405517  8773 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.428399  4603 rpcz_store.cc:276] 0709 12:26:17.244181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.613075 (+368894us) service_pool.cc:224] Handling call
0709 12:26:17.613699 (+   624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.088609 (+474910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.388867 (+300258us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b540032c3d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:18.389045 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1082713472}
I20260709 12:26:18.398077  8772 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:18.306458  8642 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.341913  8479 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:18.466107  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 378699us
I20260709 12:26:18.354164  8745 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.456020  8627 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.452335  8782 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:18.488709  8784 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:18.481350  8729 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:18.455737  8435 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:18.531674  8790 log.cc:927] Time spent T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:26:18.502053  8627 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:18.474267  8769 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.441308  8519 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.489001  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:18.475564  8789 consensus_queue.cc:1243] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:26:18.532128  4747 raft_consensus.cc:3055] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:18.543277  4747 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:26:18.501674  8657 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.009s	sys 0.034s
I20260709 12:26:18.499325  8447 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:18.490414  8700 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.469033  8781 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.492153  8773 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.526932  8714 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:18.557825  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 68909.4us
W20260709 12:26:18.551067  8657 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.010s	sys 0.035s
W20260709 12:26:18.562223  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 217226us
I20260709 12:26:18.559729  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:18.495419  4753 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:18.499287  4751 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:18.540743  8633 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:18.527709  8784 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.570331  8649 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:18.554441  4747 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.590159  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:18.595680  4601 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:26:18.564011  8531 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.056s
W20260709 12:26:18.621592  8737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.322s	user 0.001s	sys 0.012s
W20260709 12:26:18.622066  8737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.337s	user 0.001s	sys 0.012s
I20260709 12:26:18.622834  4597 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:18.623512  4747 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:18.617969  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 124450us
I20260709 12:26:18.626361  4679 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:18.626756  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:18.626992  4673 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:26:18.627378  4681 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:18.562726  8657 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:18.550678  8769 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.636556  8808 consensus_queue.cc:1059] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:18.637835  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:18.552906  8798 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 12:26:18.632743  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:18.557890  8762 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 12:26:18.638444  4676 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 7.
I20260709 12:26:18.642055  8633 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:18.642762  8633 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:18.643848  8797 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:26:18.645213  8795 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:26:18.646831  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:18.640177  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:18.647394  4747 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:26:18.598176  8714 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:18.654820  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.638727  8785 log.cc:927] Time spent T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:26:18.634950  8531 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:18.664215  5342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53808 (request call id 544) took 1398 ms. Trace:
I20260709 12:26:18.664647  5342 rpcz_store.cc:276] 0709 12:26:17.259603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.259774 (+   171us) service_pool.cc:224] Handling call
0709 12:26:17.283049 (+ 23275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.662187 (+379138us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.662215 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.669469 (+  7254us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.226233 (+556764us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:18.226289 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.226384 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.657623 (+431239us) raft_consensus.cc:1685] finished
0709 12:26:18.657644 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.657776 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43462,"raft.queue_time_us":140101,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":66,"thread_start_us":816655,"threads_started":2,"wal-append.queue_time_us":773735}
W20260709 12:26:18.666266  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.645735  8633 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:18.651690  8652 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:26:18.533288  8643 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:18.653057  8728 consensus_meta.cc:287] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.346s	user 0.013s	sys 0.001s
I20260709 12:26:18.639837  8806 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:18.655646  8422 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:18.669328  8642 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:18.669869  8441 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:18.669776  8799 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:18.671078  8766 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.672317  8479 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:18.672638  8796 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 12:26:18.689360  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 222897us
W20260709 12:26:18.645516  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.561743  8800 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:18.653380  8714 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:18.682647  8794 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:26:18.674690  8744 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:18.700816  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.702798  8781 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:18.693804  8643 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:18.690320  8802 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:26:18.709076  8769 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:18.724489  8769 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:18.726171  8784 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:18.690599  8808 consensus_queue.cc:1243] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:26:18.716156  4647 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:18.708050  8714 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:18.723795  8766 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:18.727730  8728 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:18.728216  8728 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:18.698081  8799 raft_consensus.cc:697] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:18.729002  8728 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.731364  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2827) took 1371 ms. Trace:
I20260709 12:26:18.731804  4674 rpcz_store.cc:276] 0709 12:26:17.360104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.617691 (+257587us) service_pool.cc:224] Handling call
0709 12:26:17.618029 (+   338us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:17.631506 (+ 13477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.631556 (+    50us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:17.633030 (+  1474us) log.cc:844] Serialized 78 byte log entry
0709 12:26:18.133936 (+500906us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:18.133976 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.134048 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.731116 (+597068us) raft_consensus.cc:1685] finished
0709 12:26:18.731137 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.731300 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16219,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":5998,"thread_start_us":463810,"threads_started":1,"wal-append.queue_time_us":700084}
I20260709 12:26:18.732784  4647 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:18.733690  8721 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:18.734186  8721 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:18.734814  4598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2269) took 2641 ms (client timeout 3202 ms). Trace:
I20260709 12:26:18.735008  8721 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:18.735314  4598 rpcz_store.cc:269] 0709 12:26:16.093277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.093428 (+   151us) service_pool.cc:224] Handling call
0709 12:26:18.734548 (+2641120us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54002e4fd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:18.734727 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3522096000}
I20260709 12:26:18.687448  8806 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.693905  8422 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:18.692037  8657 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.631765  8649 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.706892  8479 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.760437  8422 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.739079  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:18.763382  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 116592us
W20260709 12:26:18.765679  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.770669  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 51624.7us
I20260709 12:26:18.771271  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:18.771929  4676 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:18.772725  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:18.704872  8782 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.774528  4674 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:18.775122  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:18.769167  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:18.729856  8799 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.784909  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:18.789518  4747 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 12:26:18.791185  4751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:42632
I20260709 12:26:18.793625  4751 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:18.794003  4751 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:18.792181  4672 raft_consensus.cc:2410] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 10 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:26:18.795861  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.799134  4679 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:18.802888  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:18.804096  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:18.770146  8794 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:18.766314  8519 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:18.813763  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.809798  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:18.717463  8744 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.819653  4596 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:18.820278  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.811529  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 184766us
I20260709 12:26:18.778678  8661 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:18.815847  4752 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:26:18.816469  4569 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:18.772123  8773 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:18.802271  8500 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:18.808257  8782 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:18.825073  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:18.833444  4752 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:18.834755  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:18.773144  8479 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:18.837509  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.835094  4751 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.801477  8816 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:26:18.818706  8447 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:18.818511  8762 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:18.824235  8661 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.823961  8800 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.827550  8728 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:18.846387  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:18.831655  8500 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:18.810977  8601 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:26:18.858038  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:18.850944  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:18.847488  4751 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:18.901288  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 132233us
I20260709 12:26:18.857156  4569 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:18.851743  8762 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.866137  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:18.905323  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:18.906006  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.866729  8793 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:18.867369  8447 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:18.868055  8479 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:18.868345  8792 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:26:18.871173  4752 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:18.871091  8720 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:18.903370  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 52563.7us
I20260709 12:26:18.861477  8661 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:18.874696  8728 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:18.899873  8737 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.877983  8700 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:18.877727  8775 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:18.909323  8435 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:18.845777  8615 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:26:18.900821  8500 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:18.919736  5312 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:26:18.946493  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.911652  4600 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:18.909416  8817 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:19.028493  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 108823us
I20260709 12:26:18.902886  8601 consensus_queue.cc:1243] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:26:18.964848  8728 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:18.995134  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 138044us
I20260709 12:26:19.038064  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.038673  4679 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:18.922823  8479 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:19.040047  8479 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:19.029311  5312 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:19.042976  8479 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:19.043527  8479 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:19.044063  8479 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:18.971499  8435 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.046069  8435 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:18.976322  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:19.050330  4751 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:19.051450  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:18.915580  8793 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:19.053121  4751 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:19.053658  4599 raft_consensus.cc:2410] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:18.964258  8720 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:19.054688  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:18.967024  8775 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:19.055536  8737 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:19.070160  4600 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:18.900216  8800 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:19.083966  8627 consensus_meta.cc:287] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.532s	user 0.008s	sys 0.003s
W20260709 12:26:19.084343  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 55118.9us
I20260709 12:26:19.112811  8720 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:19.113020  8447 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:19.070854  8721 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:19.086608  8772 consensus_meta.cc:287] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.590s	user 0.007s	sys 0.005s
I20260709 12:26:19.079830  8808 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:19.128100  8823 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.926681  8806 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:19.087201  8775 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:19.088471  8700 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.098944  8479 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:19.147460  8808 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:19.173764  8720 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.135600  8649 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:19.191331  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.115377  8627 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:19.183358  8808 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:19.175038  8744 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:19.154013  8721 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:19.212157  8821 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.309s	user 0.001s	sys 0.001s
I20260709 12:26:19.154213  8772 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:19.177929  8700 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:19.191831  8720 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:19.193660  8728 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:19.217341  8820 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 12:26:19.233227  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.236239  4680 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:19.214545  8762 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:19.234515  4675 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:19.194173  8813 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.326s
I20260709 12:26:19.240520  8818 consensus_queue.cc:1059] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 12:26:19.239082  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.241539  4675 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:19.234879  8649 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:19.242233  8649 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:19.242499  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.243147  4675 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:26:19.243932  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.244303  8649 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:19.244666  4675 raft_consensus.cc:2410] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:26:19.245709  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.246376  4645 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:19.247211  8481 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:19.248569  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:19.248670  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2302) took 2003 ms. Trace:
I20260709 12:26:19.249132  4596 rpcz_store.cc:276] 0709 12:26:17.245083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.933096 (+688013us) service_pool.cc:224] Handling call
0709 12:26:17.933446 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.820097 (+886651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.248430 (+428333us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b54002337d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:19.248609 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2037437696}
I20260709 12:26:19.250860  8481 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:19.252179  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:19.254989  8721 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:19.256763  8481 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:19.258234  4645 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:19.238473  8767 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:19.263670  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2807) took 1195 ms. Trace:
I20260709 12:26:19.263954  8737 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:19.264570  8481 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:19.265105  8481 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:19.265820  8772 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:19.265973  8481 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:19.168355  8823 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:19.264225  4753 rpcz_store.cc:276] 0709 12:26:18.068271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.073418 (+  5147us) service_pool.cc:224] Handling call
0709 12:26:19.263451 (+1190033us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54000a7654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:19.263592 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657038592}
I20260709 12:26:19.271397  8823 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:19.272020  8814 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:26:19.276226  8800 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:19.217367  8812 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.314s
I20260709 12:26:19.277911  8700 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:19.240724  4679 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:19.241386  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:19.283568  4678 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:19.281312  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.294452  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:19.291915  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.305696  4675 raft_consensus.cc:2393] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:19.315106  4645 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:19.317651  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.321342  4605 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:19.332083  8435 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:19.332402  4678 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:19.339324  4678 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 12:26:19.335812  8435 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:19.379391  4677 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.009s	sys 0.014s
W20260709 12:26:19.386426  4677 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.010s	sys 0.015s
W20260709 12:26:19.380481  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 65500.9us
I20260709 12:26:19.387171  4645 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:19.386047  4673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.006s	sys 0.018s
W20260709 12:26:19.390010  4673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.006s	sys 0.018s
W20260709 12:26:19.380399  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 267157us
I20260709 12:26:19.303061  8767 raft_consensus.cc:697] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:19.241485  8807 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.236s
I20260709 12:26:19.277987  8500 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:19.405144  8825 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.442166  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.400204  8831 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:19.446772  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.447783  4673 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:19.447922  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:19.448732  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.449450  4673 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:19.451155  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:19.451565  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.422230  8837 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.451802  4673 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:19.452332  4677 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:19.426033  8721 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:19.457017  4598 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:19.474956  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:19.456672  8852 log.cc:927] Time spent T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.004s
I20260709 12:26:19.441915  8435 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.472510  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:19.488786  4603 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:19.494047  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:19.460098  8769 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.475718  5342 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:19.461247  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.504959  4673 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:19.478772  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:19.479737  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:19.482540  4601 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:26:19.484985  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:19.462169  8627 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:19.485697  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:19.465207  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.507149  4598 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:26:19.499903  8643 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.760s	user 0.012s	sys 0.000s
I20260709 12:26:19.513813  4603 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
W20260709 12:26:19.514842  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:19.502741  8714 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.775s	user 0.010s	sys 0.001s
I20260709 12:26:19.515739  5348 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEARNER]: Advancing to term 12
I20260709 12:26:19.515900  8714 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:19.518288  8714 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:19.507797  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.475798  8519 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:19.523408  5342 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:19.525060  5347 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:19.525045  5348 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:26:19.514389  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:19.510703  8729 consensus_meta.cc:287] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.981s	user 0.010s	sys 0.003s
I20260709 12:26:19.450443  8825 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:19.445852  8767 consensus_queue.cc:237] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:19.462706  8837 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:19.498163  8435 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:19.507632  8728 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:19.970464  8531 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:19.605209  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 218107us
I20260709 12:26:19.973094  8531 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:19.973878  8825 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:19.558418  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49092.3us
I20260709 12:26:19.974323  4645 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:19.561838  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:19.543766  8500 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:19.893327  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.706593  4675 raft_consensus.cc:1217] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[4.3-9.5]   Dedup: 4.3->[7.4-9.5]
I20260709 12:26:19.525041  4673 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:19.525729  5348 pending_rounds.cc:85] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 5
I20260709 12:26:19.506987  8721 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.015596  5348 pending_rounds.cc:107] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:26:20.014422  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:20.016171  5348 raft_consensus.cc:2889] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:20.016525  5342 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:19.533722  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:20.016569  4673 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:19.973083  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.018503  4673 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:19.903288  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.017522  5344 raft_consensus.cc:3055] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:20.017001  5342 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:20.021205  5342 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.022625  4753 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:19.798862  4644 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.196:39067 timed out after 1.680s (SENT) [suppressed 6 similar messages]
I20260709 12:26:20.022890  5342 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:19.703898  4672 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:19.532117  4677 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:19.956364  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 670) took 1168 ms. Trace:
W20260709 12:26:20.018819  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.024614  4753 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 12:26:20.024484  4754 rpcz_store.cc:276] 0709 12:26:18.788048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.801931 (+ 13883us) service_pool.cc:224] Handling call
0709 12:26:18.802237 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:18.809415 (+  7178us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:18.823236 (+ 13821us) log.cc:844] Serialized 14 byte log entry
0709 12:26:19.885426 (+1062190us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b180021d420. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007efff238a58f 00007effeb3c9571 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd
0709 12:26:19.953420 (+ 67994us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:19.954044 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 12:26:19.954209 (+   165us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:19.954260 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.954356 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.956013 (+  1657us) raft_consensus.cc:1685] finished
0709 12:26:19.956125 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.956287 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1044560,"prepare.queue_time_us":14318,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":1038983,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":6385,"spinlock_wait_cycles":5025536,"thread_start_us":67938,"threads_started":1,"wal-append.queue_time_us":267628}
W20260709 12:26:20.026202  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.623785  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:19.894032  8788 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:19.544549  8842 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:26:19.975723  8531 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:19.580585  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:19.621361  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:20.017590  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:19.544386  4681 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
W20260709 12:26:19.873054  8808 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.002s
I20260709 12:26:20.030282  4679 raft_consensus.cc:1240] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:19.676728  8626 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:20.031467  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57216.3us
I20260709 12:26:20.035028  4749 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:19.665753  8627 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:26:19.558449  8661 consensus_meta.cc:287] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.588s	user 0.009s	sys 0.002s
W20260709 12:26:19.815802  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.950806  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 673) took 1162 ms. Trace:
I20260709 12:26:19.623440  8686 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:20.015618  4675 pending_rounds.cc:85] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 3
I20260709 12:26:20.059404  4675 pending_rounds.cc:107] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:26:20.059713  4675 raft_consensus.cc:2889] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:19.955438  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.025139  5344 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:20.060316  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:19.565340  8848 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:19.896234  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2308) took 2483 ms. Trace:
I20260709 12:26:19.604771  8720 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:19.954934  8770 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.035483  4756 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:19.870172  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:19.508001  8769 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.042757  4673 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:20.076712  4596 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:19.544533  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:19.593493  8775 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:19.520614  5341 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.044561  8784 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.565140  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:20.040912  8645 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:26:20.041409  8669 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.019373  8652 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.051986  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:20.052096  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:19.872033  8782 consensus_meta.cc:287] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 1.032s	user 0.011s	sys 0.001s
I20260709 12:26:20.056985  5312 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:20.050014  8815 consensus_queue.cc:1059] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:26:19.729199  8566 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.054307  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:20.130932  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:20.064492  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:20.068370  5342 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
W20260709 12:26:20.070467  4721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.962s	user 0.014s	sys 0.035s
W20260709 12:26:20.132017  4721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.023s	user 0.014s	sys 0.035s
I20260709 12:26:20.070866  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:20.071449  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:20.072032  4597 rpcz_store.cc:276] 0709 12:26:17.412613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.158381 (+745768us) service_pool.cc:224] Handling call
0709 12:26:18.158716 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.623213 (+464497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.895917 (+1272704us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b540013e7d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:19.896151 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1066873984}
I20260709 12:26:20.056427  4747 rpcz_store.cc:276] 0709 12:26:18.788797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.805411 (+ 16614us) service_pool.cc:224] Handling call
0709 12:26:18.823674 (+ 18263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:18.828575 (+  4901us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:19.081793 (+253218us) log.cc:844] Serialized 14 byte log entry
0709 12:26:19.895107 (+813314us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:19.928436 (+ 33329us) log.cc:844] Serialized 26 byte log entry
0709 12:26:19.928656 (+   220us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:19.928707 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.928808 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.950549 (+ 21741us) raft_consensus.cc:1685] finished
0709 12:26:19.950573 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.950741 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1771927,"prepare.queue_time_us":21091,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":736834,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":8020,"thread_start_us":251043,"threads_started":2,"wal-append.queue_time_us":78864}
I20260709 12:26:20.061012  5344 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.134876  5344 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:20.074836  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:19.703871  8793 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:19.533907  8519 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.155269  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:20.156229  4755 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:20.032101  8788 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.043573  4677 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:20.069866  8729 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.095736  5345 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:20.030280  8721 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:20.110009  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 53274us
I20260709 12:26:20.034981  8857 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:20.029085  4720 connection.cc:256] RPC call timeout handler was delayed by -1.1173531740000726s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
I20260709 12:26:20.110600  4600 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:20.107523  4752 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:20.085758  8720 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.043802  8808 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.059659  8627 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.114217  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:20.078325  8825 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:20.132797  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.093050  8763 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
W20260709 12:26:20.235178  4721 connection.cc:256] RPC call timeout handler was delayed by -1.3074403390000953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:26:20.060449  8661 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.153728  4719 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:20.237859  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 84225.4us
I20260709 12:26:20.154778  4674 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:20.042423  8531 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:20.078761  8848 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.186825  4749 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:20.111753  8645 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:20.113345  8838 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:20.223385  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.224020  8857 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:20.224152  5312 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:20.225814  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:20.226048  4747 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:20.258914  4680 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:20.228453  8788 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:20.260900  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 209084us
W20260709 12:26:20.229750  8860 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:26:20.231254  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:20.234733  4756 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:20.074699  8807 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:26:20.088924  8770 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.113655  8652 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.104847  8784 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:20.237141  8626 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.152261  8782 raft_consensus.cc:3055] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:20.249195  8531 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.155982  8566 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.202031  8831 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:26:20.247592  8642 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:20.264313  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.228688  8721 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:20.223479  8729 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.287602  5349 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:20.236133  8720 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.288846  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:20.267597  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.329177  8783 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.327399  8714 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.317724  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.367669  4601 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:20.239348  8825 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.253798  8822 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:20.258948  8857 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.318732  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:20.323071  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 53354.2us
W20260709 12:26:20.252571  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 190203us
I20260709 12:26:20.374636  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:20.285408  8837 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:20.286811  8773 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:20.318408  8788 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.322357  8435 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:20.260818  8838 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.361282  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.394227  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:20.267864  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 469002us
I20260709 12:26:20.327877  8626 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.323843  8782 raft_consensus.cc:740] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:20.263654  8645 raft_consensus.cc:697] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:20.371011  4747 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:20.338964  8799 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:20.417001  4568 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:20.379343  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 56419.1us
I20260709 12:26:20.394745  4598 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:20.396226  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:20.419842  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.398650  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.319792  8584 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:20.323591  8479 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.404048  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:20.415601  4644 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:20.421484  4601 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:26:20.420473  4598 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:26:20.418718  4568 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:20.430051  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.427234  5349 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:20.360942  8642 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.366029  8868 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:26:20.436026  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:20.417505  8793 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:20.376904  8714 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.394717  8773 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.444152  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:20.447553  5344 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:20.432416  8479 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.448822  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:20.423768  8865 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:20.394598  8788 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.438897  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:20.454293  4598 raft_consensus.cc:2368] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10.
I20260709 12:26:20.456517  8714 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:20.456988  8642 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:20.457187  8769 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:20.458220  8531 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:20.458521  8645 consensus_queue.cc:237] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.458590  8782 consensus_queue.cc:260] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.502511  8782 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:20.460072  8784 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:20.460449  8661 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:20.505626  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.432793  8519 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:20.373634  8783 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:20.507124  8819 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 1.402s	user 0.003s	sys 0.001s
I20260709 12:26:20.507922  8783 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:20.442912  8601 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:20.508656  8601 raft_consensus.cc:697] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:20.509617  8601 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.467350  4601 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:20.474231  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:20.422003  8799 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.453281  8627 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:20.474440  8729 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:20.498975  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:20.475885  8825 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:20.500821  8720 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:20.510237  8733 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:20.429999  8584 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:20.523901  8867 log.cc:927] Time spent T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:26:20.524243  8883 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:26:20.525702  8769 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:20.527840  8799 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:20.454002  8793 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.529830  8879 raft_consensus.cc:3055] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:20.529927  8652 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:20.529914  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:20.531241  4601 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:20.534013  8848 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:20.534364  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1877) took 1298 ms. Trace:
I20260709 12:26:20.585127  5344 raft_consensus.cc:2393] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:20.586056  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.597440  4680 rpcz_store.cc:276] 0709 12:26:19.235241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.236872 (+  1631us) service_pool.cc:224] Handling call
0709 12:26:20.533270 (+1296398us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b5400084654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:20.533500 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2162436992}
I20260709 12:26:20.535511  8773 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:20.618574  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:20.466722  8808 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:20.536194  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:20.537915  8479 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:20.539932  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.644192  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:20.545423  4598 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.552271  4571 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:20.572387  8627 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.575918  8788 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:20.576507  8566 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:20.578418  4681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2843) took 1808 ms (client timeout 2193 ms). Trace:
W20260709 12:26:20.583603  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 156360us
I20260709 12:26:20.584437  8879 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:20.662065  8879 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.663389  8879 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.666157  5349 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:20.653755  8627 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.667554  5349 raft_consensus.cc:3055] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:20.667905  5349 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:20.668397  8627 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:20.668609  5349 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:20.606735  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 232175us
W20260709 12:26:20.589527  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.670758  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:20.671821  8875 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:20.671897  5341 consensus_meta.cc:287] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.563s	user 0.007s	sys 0.003s
I20260709 12:26:20.671957  5312 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:20.672483  5341 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:20.637831  8626 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:20.673241  5341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 617) took 1195 ms. Trace:
W20260709 12:26:20.636332  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.673790  8875 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:20.673691  5341 rpcz_store.cc:276] 0709 12:26:19.477711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.481466 (+  3755us) service_pool.cc:224] Handling call
0709 12:26:20.673171 (+1191705us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:20.675096  8793 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:20.652964  8686 consensus_meta.cc:287] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.001s
I20260709 12:26:20.538740  8857 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:20.675473  5349 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:20.654311  4681 rpcz_store.cc:269] 0709 12:26:18.769823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.772490 (+  2667us) service_pool.cc:224] Handling call
0709 12:26:20.578241 (+1805751us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b540019fd54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:20.578354 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1773012224}
I20260709 12:26:20.654311  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:20.679301  5344 raft_consensus.cc:1240] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:20.568315  8868 consensus_queue.cc:1059] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:26:20.681216  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:20.679175  4680 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:26:20.598775  4601 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:20.656672  8584 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:20.657729  8447 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.654217  8769 consensus_queue.cc:1243] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:26:20.498096  8861 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:26:20.675778  8765 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.673255  8815 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:20.677155  8857 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:20.653615  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 101441us
I20260709 12:26:20.686144  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.687628  4752 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 12:26:20.698071  4601 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:20.688377  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:20.689689  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:20.689035  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 123013us
I20260709 12:26:20.694972  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2818) took 1878 ms. Trace:
I20260709 12:26:20.697814  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:20.699867  4681 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:20.700218  4756 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:20.688030  4598 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:20.702296  4680 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:20.699872  4751 rpcz_store.cc:276] 0709 12:26:18.817023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.094030 (+277007us) service_pool.cc:224] Handling call
0709 12:26:19.110610 (+ 16580us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:19.894490 (+783880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.894522 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:19.902879 (+  8357us) log.cc:844] Serialized 78 byte log entry
0709 12:26:20.086712 (+183833us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:20.295218 (+208506us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.328717 (+ 33499us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.544761 (+216044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.544843 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.694760 (+149917us) raft_consensus.cc:1685] finished
0709 12:26:20.694784 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.694909 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":870953,"prepare.queue_time_us":353689,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":4027,"raft.queue_time_us":858320,"raft.run_cpu_time_us":409,"raft.run_wall_time_us":20062,"spinlock_wait_cycles":18692480,"thread_start_us":522530,"threads_started":3,"wal-append.queue_time_us":221117}
I20260709 12:26:20.702463  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:20.711288  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.712427  4601 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:20.713797  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:20.715691  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.717276  4601 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:20.717770  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:20.718675  5312 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:20.718750  4680 raft_consensus.cc:2410] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:26:20.721258  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.721724  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1859) took 3477 ms. Trace:
I20260709 12:26:20.722064  4601 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:20.701167  8883 raft_consensus.cc:3055] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:20.722347  4605 rpcz_store.cc:276] 0709 12:26:17.244504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.668694 (+424190us) service_pool.cc:224] Handling call
0709 12:26:17.669081 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.321847 (+1652766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.721337 (+1399490us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b5400060754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:20.721636 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3799389568}
I20260709 12:26:20.724849  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.703689  8789 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.719583  8516 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:20.701287  8765 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.730813  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.795090  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.798751  4681 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
W20260709 12:26:20.826429  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 107847us
I20260709 12:26:20.783041  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.827297  4598 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:20.729465  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.828925  4596 raft_consensus.cc:2410] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:20.828886  5312 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:20.830340  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.831486  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.843192  4603 raft_consensus.cc:2410] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:20.852837  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:20.855506  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:20.856190  4681 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:20.758456  5343 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:20.857403  4681 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:26:20.855290  8789 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.858801  4681 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:26:20.859339  8884 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:20.859783  8884 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.860057  4681 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:20.860106  8884 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:20.861521  4605 raft_consensus.cc:1240] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 12:26:20.863399  8775 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:20.863490  4605 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:26:20.864142  8516 raft_consensus.cc:697] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:20.865106  8516 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:20.869853  4599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2314) took 2012 ms (client timeout 1907 ms). Trace:
I20260709 12:26:20.871093  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:20.872969  5340 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:20.875509  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 619) took 1396 ms. Trace:
I20260709 12:26:20.876091  4600 rpcz_store.cc:276] 0709 12:26:19.479395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.480177 (+   782us) service_pool.cc:224] Handling call
0709 12:26:19.497444 (+ 17267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.233879 (+736435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.875180 (+641301us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b5400352ad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:20.875424 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1409619328}
W20260709 12:26:20.880125  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 51325.2us
I20260709 12:26:20.882040  8782 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:20.882820  8782 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:20.883773  8782 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:20.885927  8769 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:20.897290  8350 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.897964  8350 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:20.870615  4599 rpcz_store.cc:269] 0709 12:26:18.857273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.857782 (+   509us) service_pool.cc:224] Handling call
0709 12:26:20.869471 (+2011689us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54003d0254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:20.869710 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350383872}
I20260709 12:26:20.900447  4674 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:20.881934  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2850) took 1814 ms. Trace:
I20260709 12:26:20.931383  5349 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:20.987583  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:20.988695  4598 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:20.768656  8883 raft_consensus.cc:740] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:20.754987  8669 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.000s
I20260709 12:26:20.905263  4676 rpcz_store.cc:276] 0709 12:26:19.067376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.067616 (+   240us) service_pool.cc:224] Handling call
0709 12:26:19.086436 (+ 18820us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.274284 (+187848us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.274316 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:19.274789 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 12:26:19.385805 (+111016us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:19.385854 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.881293 (+1495439us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400200654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:20.881366 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.881510 (+   144us) raft_consensus.cc:1685] finished
0709 12:26:20.881529 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.881865 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69146,"raft.queue_time_us":12130,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":16813,"spinlock_wait_cycles":431140608,"thread_start_us":110722,"threads_started":1,"wal-append.queue_time_us":107294,"wal-append.run_cpu_time_us":5413,"wal-append.run_wall_time_us":1115639}
I20260709 12:26:20.997320  4569 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:26:21.065274  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 68048.4us
W20260709 12:26:21.067183  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.066154  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:21.068923  8622 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:21.069471  8622 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:21.069977  8622 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:21.077663  4596 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:21.080029  4596 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 12:26:21.080468  4596 pending_rounds.cc:85] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 9
I20260709 12:26:21.080667  4596 pending_rounds.cc:107] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:26:21.080888  4596 raft_consensus.cc:2889] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:21.090409  8883 consensus_queue.cc:260] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:21.091936  8883 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:21.109666  8781 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:21.112654  8447 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:21.030045  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:21.175833  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:21.067159  8798 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:26:21.186492  4569 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:21.249712  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:21.250439  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:21.250943  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.117956  8669 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:20.998445  8792 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.299s
I20260709 12:26:20.998207  8837 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.225s
W20260709 12:26:21.223284  8618 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.045s	sys 0.097s
I20260709 12:26:21.271351  8519 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:21.295023  5340 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:21.222005  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:21.313730  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 91816.5us
W20260709 12:26:21.295840  8618 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.046s	sys 0.097s
I20260709 12:26:21.370252  4596 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:21.315119  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:21.383461  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 68445.3us
I20260709 12:26:21.302129  8519 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:21.344995  8781 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:21.428370  8792 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:21.456014  8792 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.456490  8792 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:21.437857  8781 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.461998  8781 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:21.437968  8519 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:21.406191  8622 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:21.384385  4569 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:21.345988  8884 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:21.542958  5344 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.020s	sys 0.122s
W20260709 12:26:21.676255  5344 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.027s	sys 0.126s
I20260709 12:26:21.677336  5344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 562) took 1230 ms. Trace:
W20260709 12:26:21.509217  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.482218  4596 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:21.683540  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2330) took 1022 ms. Trace:
I20260709 12:26:21.684034  4596 rpcz_store.cc:276] 0709 12:26:20.660739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.371486 (+710747us) service_pool.cc:224] Handling call
0709 12:26:21.481766 (+110280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.683135 (+201369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.683371 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:21.677927  5344 rpcz_store.cc:276] 0709 12:26:20.447223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.680195 (+232972us) service_pool.cc:224] Handling call
0709 12:26:20.680473 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.680666 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:20.910230 (+229564us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.676856 (+766626us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:21.676942 (+    86us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:21.677010 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.677093 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.677259 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":652778,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":9717,"thread_start_us":969855,"threads_started":2}
W20260709 12:26:21.685329  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 301031us
W20260709 12:26:21.694766  5345 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 554) took 2203 ms (client timeout 1680 ms). Trace:
I20260709 12:26:21.692971  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2332) took 1031 ms. Trace:
I20260709 12:26:21.696607  4596 rpcz_store.cc:276] 0709 12:26:20.661262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.692164 (+1030902us) service_pool.cc:224] Handling call
0709 12:26:21.692475 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.692651 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.692677 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:21.692719 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:21.692754 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.692804 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.692910 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:21.695299  5345 rpcz_store.cc:269] 0709 12:26:19.478435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.478545 (+   110us) service_pool.cc:224] Handling call
0709 12:26:21.681304 (+2202759us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b540055e6d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:21.681475 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1356379904}
I20260709 12:26:21.691262  8692 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:21.708364  4596 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:21.714766  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 626) took 1008 ms. Trace:
I20260709 12:26:21.715703  4596 rpcz_store.cc:276] 0709 12:26:20.706168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.702863 (+996695us) service_pool.cc:224] Handling call
0709 12:26:21.703144 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.714491 (+ 11347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.714704 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:21.717006  4596 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:21.722283  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 627) took 1013 ms. Trace:
I20260709 12:26:21.727522  4596 rpcz_store.cc:276] 0709 12:26:20.708791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.716533 (+1007742us) service_pool.cc:224] Handling call
0709 12:26:21.716834 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.721397 (+  4563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.722198 (+   801us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:21.721282  4569 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:21.723982  8622 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:21.712877  8692 raft_consensus.cc:697] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:21.720386  8879 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.725658  8350 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:21.728763  4596 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:21.733759  5348 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 619) took 1782 ms (client timeout 1602 ms). Trace:
I20260709 12:26:21.733497  8566 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:21.734369  5348 rpcz_store.cc:269] 0709 12:26:19.951296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.733542 (+1782246us) service_pool.cc:211] Skipping call since client already timed out
0709 12:26:21.733692 (+   150us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:26:21.734441  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:21.710479  8781 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:21.740309  8692 consensus_queue.cc:237] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:21.743204  8781 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:21.732371  8693 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:21.743965  8781 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:21.744174  8693 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:21.737279  8566 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.746054  8566 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:21.733736  8622 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:26:21.750254  8792 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:21.735620  8879 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:21.756770  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.756115  8879 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:21.758944  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 616) took 2239 ms. Trace:
I20260709 12:26:21.759653  4679 rpcz_store.cc:276] 0709 12:26:19.519263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.037591 (+518328us) service_pool.cc:224] Handling call
0709 12:26:20.043920 (+  6329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.044219 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:20.238008 (+193789us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.377633 (+139625us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.758249 (+1380616us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b180036aa20. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007efff238a58f 00007effeb3c9571 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd
0709 12:26:21.758570 (+   321us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:21.758645 (+    75us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:21.758692 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.758756 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.758886 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1382241,"prepare.queue_time_us":6092,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":56,"raft.queue_time_us":5740,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":2818,"spinlock_wait_cycles":41850624,"thread_start_us":311556,"threads_started":2,"wal-append.queue_time_us":159709,"wal-append.run_cpu_time_us":1672,"wal-append.run_wall_time_us":1052686}
I20260709 12:26:21.763253  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 617) took 2243 ms. Trace:
W20260709 12:26:21.765775  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 1.3695e+06us
I20260709 12:26:21.767467  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:21.763762  4673 rpcz_store.cc:276] 0709 12:26:19.519529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.076873 (+557344us) service_pool.cc:224] Handling call
0709 12:26:20.077220 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.253086 (+175866us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:20.263073 (+  9987us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.388916 (+125843us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.400188 (+ 11272us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.400415 (+   227us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:20.415567 (+ 15152us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.762786 (+1347219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.762900 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.762963 (+    63us) raft_consensus.cc:1685] finished
0709 12:26:21.762983 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.763176 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1357180,"prepare.queue_time_us":11704,"prepare.run_cpu_time_us":12030,"prepare.run_wall_time_us":29325,"raft.queue_time_us":78985,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":42150,"thread_start_us":277580,"threads_started":2,"wal-append.queue_time_us":163491,"wal-append.run_cpu_time_us":4552,"wal-append.run_wall_time_us":1138773}
W20260709 12:26:21.772711  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:21.759116  5343 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 557) took 1359 ms (client timeout 1631 ms). Trace:
I20260709 12:26:21.775698  4596 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:21.777926  4596 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 12:26:21.775446  5343 rpcz_store.cc:269] 0709 12:26:20.399762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.422170 (+ 22408us) service_pool.cc:224] Handling call
0709 12:26:21.758931 (+1336761us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54004c6b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:21.759054 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645769472}
I20260709 12:26:21.779621  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:21.781864  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 1.36622e+06us
I20260709 12:26:21.783730  4644 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:21.775530  8622 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:21.805796  8792 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:21.806529  8792 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.807001  8792 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:21.809721  8441 raft_consensus.cc:3055] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:21.810166  8441 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:21.811082  8441 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:21.781068  8769 consensus_queue.cc:1059] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:26:21.778755  8879 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:26:21.818799  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 620) took 1561 ms. Trace:
I20260709 12:26:21.801465  8831 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:21.825045  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 613) took 2319 ms. Trace:
I20260709 12:26:21.825703  4672 rpcz_store.cc:276] 0709 12:26:19.505422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.024182 (+518760us) service_pool.cc:224] Handling call
0709 12:26:20.037337 (+ 13155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.037545 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:20.037814 (+   269us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.541406 (+503592us) spinlock_profiling.cc:243] Waited 5.41 ms on lock 0x7b18003179e0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:26:21.824593 (+1283187us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:21.824672 (+    79us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:21.824733 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.824822 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.824969 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1116497,"prepare.queue_time_us":94,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"spinlock_wait_cycles":12431616,"thread_start_us":166338,"threads_started":1,"wal-append.queue_time_us":835804}
I20260709 12:26:21.831399  4673 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:21.818863  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 62483.9us
I20260709 12:26:21.790987  8615 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:26:21.842155  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:21.842799  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 59171.1us
I20260709 12:26:21.800720  8869 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:21.793207  8669 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:26:21.842828  4596 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:21.822564  4677 rpcz_store.cc:276] 0709 12:26:20.257110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.258335 (+  1225us) service_pool.cc:224] Handling call
0709 12:26:20.258690 (+   355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:20.259056 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.259084 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:20.264251 (+  5167us) log.cc:844] Serialized 52 byte log entry
0709 12:26:20.582608 (+318357us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:20.681039 (+ 98431us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.818437 (+1137398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.818551 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.818597 (+    46us) raft_consensus.cc:1685] finished
0709 12:26:21.818608 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.818746 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1137121,"prepare.queue_time_us":104896,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":1141025,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":2741,"thread_start_us":225414,"threads_started":1,"wal-append.queue_time_us":414360,"wal-append.run_cpu_time_us":2796,"wal-append.run_wall_time_us":1013409}
I20260709 12:26:21.860157  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1853) took 4612 ms. Trace:
I20260709 12:26:21.860075  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 621) took 1602 ms. Trace:
I20260709 12:26:21.860700  4680 rpcz_store.cc:276] 0709 12:26:20.257485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.853916 (+596431us) service_pool.cc:224] Handling call
0709 12:26:20.909162 (+ 55246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.909458 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:20.921995 (+ 12537us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.859729 (+937734us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:21.859781 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:21.859826 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.859888 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.860014 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":566496,"prepare.queue_time_us":154,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":3251,"thread_start_us":370710,"threads_started":1,"wal-append.queue_time_us":450307}
I20260709 12:26:21.816370  8622 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:21.863624  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 96240.8us
I20260709 12:26:21.868368  4672 raft_consensus.cc:1217] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Deduplicated request from leader. Original: 11.10->[13.11-15.12]   Dedup: 15.12->[]
I20260709 12:26:21.860630  4604 rpcz_store.cc:276] 0709 12:26:17.233425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.233542 (+   117us) service_pool.cc:224] Handling call
0709 12:26:17.234028 (+   486us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:18.227906 (+993878us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.227953 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:18.279828 (+ 51875us) log.cc:844] Serialized 52 byte log entry
0709 12:26:21.688670 (+3408842us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b180017fa40. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:26:21.730395 (+ 41725us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:21.730794 (+   399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.845598 (+114804us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b54003548d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:21.845674 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.845753 (+    79us) raft_consensus.cc:1685] finished
0709 12:26:21.845774 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.845984 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55663,"raft.queue_time_us":43819,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":6176,"spinlock_wait_cycles":2196077696,"thread_start_us":41503,"threads_started":1,"wal-append.queue_time_us":3437543}
I20260709 12:26:21.869176  4672 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 12:26:21.781471  8803 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:26:21.818185  8792 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:21.875878  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:21.879882  4645 connection.cc:256] RPC call timeout handler was delayed by -1.3044210509997356s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1907s for a response
I20260709 12:26:21.884047  4677 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:21.841796  8831 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.843137  8870 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:21.859692  8869 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.857084  8825 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.897897  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:21.906105  8825 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:21.899106  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 50659.4us
I20260709 12:26:21.865859  8584 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:21.910131  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 65589.6us
I20260709 12:26:21.863142  8729 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:21.910948  4644 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:21.865159  8857 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:21.870509  8479 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:21.912926  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:21.921773  4596 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:21.926229  4676 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:21.927254  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:21.894555  8831 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:21.952543  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:21.953061  4680 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:26:21.953377  4672 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:26:21.955762  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:21.956568  4604 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:21.907691  8869 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:21.960289  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:21.919114  8898 log.cc:927] Time spent T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 12:26:21.961443  4673 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:21.963589  4680 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:21.892696  8720 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
W20260709 12:26:21.968238  8894 log.cc:927] Time spent T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.234s	user 0.000s	sys 0.002s
I20260709 12:26:21.978094  8479 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.980420  8479 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:21.922246  8584 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:21.973491  8897 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:26:21.924485  8857 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:21.996018  8827 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 2.803s	user 0.011s	sys 0.018s
I20260709 12:26:21.996331  8857 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:21.922271  8729 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:21.984206  8908 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:21.984803  8519 consensus_meta.cc:287] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.519s	user 0.012s	sys 0.003s
I20260709 12:26:21.985373  8913 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:26:21.990396  8692 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:21.991072  8792 consensus_queue.cc:1059] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:21.993371  8584 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:21.994369  8825 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:21.983366  8907 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:21.952234  8769 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:26:21.988786  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:21.982996  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:22.056043  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:21.987424  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:21.974623  8884 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.063611  8800 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.064339  8800 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:21.975896  8803 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.096752  8803 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.099042  8803 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.053380  4672 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:22.099732  8745 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:22.100804  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:22.101392  4672 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:22.054899  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:22.106230  5343 raft_consensus.cc:2368] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9.
I20260709 12:26:22.107864  5341 raft_consensus.cc:3055] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:22.108566  5341 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:22.109768  8882 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 1.579s	user 0.000s	sys 0.022s
I20260709 12:26:22.111272  8763 raft_consensus.cc:3055] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:22.111824  8763 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:22.115859  8800 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:22.111840  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1869) took 1845 ms. Trace:
W20260709 12:26:22.069487  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.067209  8633 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.071955  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:22.073297  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:22.072654  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:22.122238  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1881) took 1790 ms. Trace:
I20260709 12:26:22.124830  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:22.126953  5348 raft_consensus.cc:2410] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:26:22.130177  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:22.127373  5344 raft_consensus.cc:2376] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11. Re-sending same reply.
I20260709 12:26:22.130782  4680 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:22.132490  5347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53808 (request call id 549) took 2608 ms. Trace:
I20260709 12:26:22.132650  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:22.133342  4680 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:22.133363  4645 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:22.133754  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:22.138664  4719 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:22.117139  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2294) took 2062 ms. Trace:
I20260709 12:26:22.139958  4748 rpcz_store.cc:276] 0709 12:26:20.054880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.057203 (+  2323us) service_pool.cc:224] Handling call
0709 12:26:20.057613 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.360083 (+302470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.360117 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:20.360159 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:20.360187 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.116889 (+1756702us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b5400184054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:22.116933 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.117075 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":695123200}
I20260709 12:26:22.065917  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:22.070284  8880 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:22.070099  8881 consensus_queue.cc:1059] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:22.086365  8036 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:22.087788  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:22.152344  4677 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:22.068751  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 619) took 1812 ms. Trace:
I20260709 12:26:22.081930  8799 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:22.067557  8802 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.076592  8783 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.997179  8622 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.121125  8907 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:22.128780  8857 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.134521  5346 raft_consensus.cc:2376] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10. Re-sending same reply.
I20260709 12:26:22.138756  4672 raft_consensus.cc:2410] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:26:22.145275  4645 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:22.128244  4605 rpcz_store.cc:276] 0709 12:26:20.332211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.864545 (+532334us) service_pool.cc:224] Handling call
0709 12:26:20.908010 (+ 43465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.908234 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:21.271511 (+363277us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.676608 (+405097us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:21.676689 (+    81us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:21.676759 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.676851 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.122027 (+445176us) spinlock_profiling.cc:243] Waited 43.1 ms on lock 0x7b54003d0250. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:22.122165 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":449843,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":7768,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":6585,"spinlock_wait_cycles":99123968,"thread_start_us":678954,"threads_started":2,"wal-append.queue_time_us":435682}
I20260709 12:26:22.124794  4597 rpcz_store.cc:276] 0709 12:26:20.266070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.266229 (+   159us) service_pool.cc:224] Handling call
0709 12:26:20.318210 (+ 51981us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:20.324275 (+  6065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.324308 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:20.334098 (+  9790us) log.cc:844] Serialized 52 byte log entry
0709 12:26:20.416215 (+ 82117us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:20.416254 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.416317 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.108997 (+1692680us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:22.111428 (+  2431us) raft_consensus.cc:1685] finished
0709 12:26:22.111462 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.111762 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1507,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":10717,"spinlock_wait_cycles":38174080,"thread_start_us":36413,"threads_started":1,"wal-append.queue_time_us":171250}
W20260709 12:26:22.153298  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.188951  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:22.189766  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:22.190433  4672 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:22.112692  8763 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.143805  8825 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:22.056819  8906 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:22.171926  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:22.121955  8721 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.044901  8908 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:22.179639  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 113777us
I20260709 12:26:22.180666  4719 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:22.044452  8519 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.149497  8910 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.153220  4674 rpcz_store.cc:276] 0709 12:26:20.256734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.256850 (+   116us) service_pool.cc:224] Handling call
0709 12:26:20.900138 (+643288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.901159 (+  1021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.068547 (+1167388us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b5400200650. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:22.068691 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463575424}
I20260709 12:26:22.243836  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:22.207109  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.215963  4677 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:22.151530  8813 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.207285  4645 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:22.208014  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 338510us
I20260709 12:26:22.253181  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:22.208904  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 48441.4us
I20260709 12:26:22.152696  8629 raft_consensus.cc:3055] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:22.160840  8036 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.162719  8720 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:22.224735  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 237403us
I20260709 12:26:22.132876  5347 rpcz_store.cc:276] 0709 12:26:19.515039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.515164 (+   125us) service_pool.cc:224] Handling call
0709 12:26:19.524905 (+  9741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.525676 (+   771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.123557 (+2597881us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x7b54006fbdd0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:22.123736 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14292864}
I20260709 12:26:22.177021  8881 consensus_queue.cc:1243] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:26:22.269403  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 62188.2us
I20260709 12:26:22.270732  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:22.174957  8799 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.174683  8642 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:22.144093  8633 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:22.238282  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 325268us
I20260709 12:26:22.188755  8714 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:22.204595  8773 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:22.186260  8769 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:22.186697  8907 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:22.204316  8350 consensus_queue.cc:1059] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:26:22.243290  8771 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:22.188572  8784 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:26:22.188683  8783 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.174404  8802 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.274938  8763 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:22.251313  8916 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:22.442458  8783 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:22.443768  8916 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:22.444397  8916 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:22.232300  8825 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.222048  8800 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 12:26:22.204828  8857 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:22.318068  8911 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.450011  8825 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.450608  8825 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:22.450347  8911 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.322698  8769 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.320356  8822 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:22.451952  8822 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.452598  8911 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:22.450027  8857 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.452623  8769 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:22.163365  8880 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.452666  8773 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:22.409897  8922 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:22.457818  8922 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:22.458362  8922 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:22.253795  8910 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.233994  8906 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.415081  8908 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.264797  8813 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:22.399158  8883 consensus_meta.cc:287] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 1.307s	user 0.022s	sys 0.022s
W20260709 12:26:22.410125  8912 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:26:22.322901  8823 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:22.244961  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:22.299556  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:22.506316  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 625) took 1605 ms. Trace:
I20260709 12:26:22.537941  4674 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:22.538370  4681 rpcz_store.cc:276] 0709 12:26:20.900794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.900978 (+   184us) service_pool.cc:224] Handling call
0709 12:26:20.901327 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.828248 (+926921us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.828296 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.828937 (+   641us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.994271 (+165334us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b480027e7d8. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff33dfaa1 00007efff33d6b37 00007efff33d69ca 00007efff34ce26f 00007efff34cdf37 00007efff34e255c 00007efff34e2508 00007efff34e2488 00007efff34e2443 00007efff34e16ec 00007efff240ba96
0709 12:26:22.134555 (+140284us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:22.134602 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.134661 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.505012 (+370351us) raft_consensus.cc:1685] finished
0709 12:26:22.505103 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.505826 (+   723us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":988630,"raft.queue_time_us":940687,"raft.run_cpu_time_us":1903,"raft.run_wall_time_us":136836,"spinlock_wait_cycles":3636096,"thread_start_us":241965,"threads_started":1,"wal-append.queue_time_us":444349}
W20260709 12:26:22.236249  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 55587.6us
I20260709 12:26:22.235163  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:22.247426  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:22.557178  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 286587us
I20260709 12:26:22.559108  4645 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:22.267930  4672 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:22.563839  4756 raft_consensus.cc:1217] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 7.4->[10.5-13.7]   Dedup: 10.5->[11.6-13.7]
I20260709 12:26:22.564602  4756 pending_rounds.cc:85] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 5
I20260709 12:26:22.564913  4756 pending_rounds.cc:107] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 12:26:22.515893  5312 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:22.271728  8036 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:22.517165  4644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.016s	sys 0.012s
I20260709 12:26:22.391011  8350 consensus_queue.cc:1243] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:26:22.522680  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 265230us
I20260709 12:26:22.258911  8479 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.440357  8519 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.250268  8566 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.576862  4677 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:26:22.580013  8566 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.258379  8629 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:22.412284  8633 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:22.316744  8642 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.318398  8714 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.275370  8720 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.598686  8714 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:22.599150  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:22.601061  8720 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.239382  8721 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.603514  8813 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.604285  8721 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:22.315754  8729 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:22.428644  8799 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:22.622792  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 107566us
I20260709 12:26:22.435932  8802 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:22.627082  4674 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:22.627960  8838 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:22.628906  8838 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:22.439603  8869 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.463894  8880 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:22.573446  8922 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.639056  4679 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:22.426846  8902 log.cc:927] Time spent T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.433s	user 0.002s	sys 0.000s
I20260709 12:26:22.640301  8922 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.468429  8906 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.439642  8907 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.642457  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1881) took 3404 ms. Trace:
I20260709 12:26:22.642936  4678 rpcz_store.cc:276] 0709 12:26:19.237519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.340637 (+103118us) service_pool.cc:224] Handling call
0709 12:26:19.341025 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.408907 (+ 67882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.408938 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.728173 (+2319235us) spinlock_profiling.cc:243] Waited 5.53 ms on lock 0x7b2c00010c48. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:26:21.728571 (+   398us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.824198 (+ 95627us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:21.824245 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.824343 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.641830 (+817487us) raft_consensus.cc:1685] finished
0709 12:26:22.641882 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.642239 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67508,"raft.queue_time_us":81720,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":8324,"spinlock_wait_cycles":12718080,"thread_start_us":83916,"threads_started":1,"wal-append.queue_time_us":216624}
I20260709 12:26:22.644915  8907 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:22.470564  8910 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.410218  8914 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:26:22.470794  8916 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.653012  4672 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:22.424162  8917 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:22.440462  8921 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:26:22.527271  8516 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 12:26:22.550195  8933 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.550619  8771 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:26:22.551074  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 315801us
I20260709 12:26:22.551661  8883 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.551883  8618 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:22.538277  8781 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.044s	sys 0.075s
I20260709 12:26:22.554134  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:22.554533  8800 raft_consensus.cc:3055] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:22.564289  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:22.224680  5341 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:22.512190  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 200383us
I20260709 12:26:22.567538  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:22.567076  8924 consensus_queue.cc:1059] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:26:22.572017  8915 raft_consensus.cc:2764] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 12:26:22.587291  4644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.017s	sys 0.012s
I20260709 12:26:22.577391  4677 pending_rounds.cc:85] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 4
I20260709 12:26:22.565786  4756 raft_consensus.cc:2889] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:22.596225  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:22.596784  4755 raft_consensus.cc:3055] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:22.597775  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:22.586764  8642 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:22.302079  8831 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:22.656435  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 54433.7us
W20260709 12:26:22.657406  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 98635us
I20260709 12:26:22.630400  8838 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:22.662539  8883 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.653342  8916 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:22.671149  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 377188us
I20260709 12:26:22.673976  4755 raft_consensus.cc:740] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:22.674167  8908 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:22.671473  5341 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:22.681066  8813 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.660130  8618 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.684381  4677 pending_rounds.cc:107] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:26:22.663851  8800 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:22.690392  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:22.685288  8422 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:22.695204  8831 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.695788  4676 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:22.697719  4677 raft_consensus.cc:2889] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:22.662886  8933 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:22.685604  8781 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.044s	sys 0.076s
I20260709 12:26:22.709003  4750 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:22.710676  8771 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.714069  8813 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.716238  4645 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:22.722759  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:22.724064  8869 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.726112  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:22.726271  8767 raft_consensus.cc:3055] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:22.726701  4753 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:22.727250  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:22.732786  5347 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEARNER]: Advancing to term 13
I20260709 12:26:22.734050  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:22.734232  8916 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.707157  8422 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:22.703276  8618 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:22.735735  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:22.730594  8770 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:22.681072  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:22.741885  8765 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:22.733938  4747 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:26:22.748871  8422 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.751915  5348 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:22.757462  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:22.758168  4747 pending_rounds.cc:85] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 6
W20260709 12:26:22.762142  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 59097.4us
I20260709 12:26:22.765941  4747 pending_rounds.cc:107] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:26:22.766397  4747 raft_consensus.cc:2889] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:22.733935  8928 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.770542  4755 consensus_queue.cc:260] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.771173  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:22.746086  8767 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:22.752671  8789 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:22.767269  4754 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:22.692874  8838 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.779681  4749 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:22.779932  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57522.8us
I20260709 12:26:22.779711  4752 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:22.784430  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:22.671790  8868 consensus_queue.cc:1175] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3a5aeedd11fe49938079fe86cb01232c" responder_term: 16 status { last_received { term: 16 index: 13 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 15. Current term is 16. Ops: [13.11-15.12]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:22.795768  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:22.796659  8903 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.962s	user 0.003s	sys 0.008s
I20260709 12:26:22.800203  8924 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:22.802670  4755 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:22.802915  8881 raft_consensus.cc:3055] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:22.805871  8629 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.809580  8422 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:22.809876  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 78458us
I20260709 12:26:22.810618  8928 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.783550  8908 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:22.850682  4677 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:22.799752  8813 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.814947  8916 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.789214  8899 raft_consensus.cc:3055] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:22.862285  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:22.864212  4601 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:22.868829  8933 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:22.855039  8714 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:22.869874  8629 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:22.870607  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:22.871101  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.869766  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:22.871762  5345 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:22.876420  8714 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:22.879004  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:22.879397  4754 raft_consensus.cc:2393] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:22.879815  5344 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:22.880755  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:22.824784  8924 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:22.825448  8881 raft_consensus.cc:740] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:22.900337  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:22.867895  8899 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:22.881409  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:22.828825  8036 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:22.890478  8813 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.921912  4753 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:22.894143  8924 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:22.928081  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:22.924651  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49525.4us
W20260709 12:26:22.930868  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 214940us
I20260709 12:26:22.938217  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:22.934157  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:22.952368  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 54653.1us
I20260709 12:26:22.919790  8928 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:22.938198  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:22.957509  5341 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:22.958676  4748 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:22.984311  5341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 560) took 2553 ms. Trace:
W20260709 12:26:22.984740  8817 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:26:22.985296  4568 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:22.984985  5341 rpcz_store.cc:276] 0709 12:26:20.430822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.674879 (+244057us) service_pool.cc:224] Handling call
0709 12:26:20.675200 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.107718 (+1432518us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b54005751a0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee42ddc0 00007effee4b81c0 00007effee4b9231 00007effee4b5d8f 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c
0709 12:26:22.958066 (+850348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.984213 (+ 26147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31136640}
W20260709 12:26:22.986769  8940 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:26:22.987540  4568 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:22.989892  8789 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:22.990028  8770 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:22.990374  8789 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.990641  8770 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.990931  8789 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:22.991143  8770 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:22.961627  8933 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:22.995378  8933 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.995846  8933 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:22.981921  4753 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:23.015297  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:23.016697  4676 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
W20260709 12:26:22.996505  8909 log.cc:927] Time spent T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.162s	user 0.001s	sys 0.000s
W20260709 12:26:23.018669  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.022485  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 88412.1us
W20260709 12:26:23.027179  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 45455.5us
I20260709 12:26:23.042603  8767 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.042761  8931 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.043992  8767 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:23.044071  8931 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:23.050426  8942 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.091s	user 0.003s	sys 0.000s
I20260709 12:26:23.052039  8899 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.060168  4749 raft_consensus.cc:2410] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:26:23.065490  5312 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:23.071211  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.071547  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2879) took 1084 ms. Trace:
W20260709 12:26:23.017479  8745 consensus_meta.cc:287] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.917s	user 0.016s	sys 0.009s
I20260709 12:26:23.017200  8036 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:23.073782  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2880) took 1077 ms. Trace:
I20260709 12:26:23.072290  4678 rpcz_store.cc:276] 0709 12:26:21.970064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.644741 (+674677us) service_pool.cc:224] Handling call
0709 12:26:22.645209 (+   468us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:22.671036 (+ 25827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.671077 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:22.672038 (+   961us) log.cc:844] Serialized 52 byte log entry
0709 12:26:22.861948 (+189910us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:22.862458 (+   510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.862530 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.054377 (+191847us) raft_consensus.cc:1685] finished
0709 12:26:23.054394 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.054534 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5454,"raft.queue_time_us":17354,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":21758,"thread_start_us":189376,"threads_started":1,"wal-append.queue_time_us":336624}
I20260709 12:26:23.053427  8899 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:23.082496  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.081355  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:23.025702  8883 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:23.088716  4720 leader_election.cc:400] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10.
I20260709 12:26:23.099444  4720 leader_election.cc:403] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:26:23.100113  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:23.025424  8765 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:23.004230  8813 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.101609  8831 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.953111  8908 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.135s
I20260709 12:26:23.085506  4681 rpcz_store.cc:276] 0709 12:26:21.978136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.697846 (+719710us) service_pool.cc:224] Handling call
0709 12:26:22.776910 (+ 79064us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.812785 (+ 35875us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.812819 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:22.813750 (+   931us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.846201 (+ 32451us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:22.846257 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.854996 (+  8739us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.054739 (+199743us) raft_consensus.cc:1685] finished
0709 12:26:23.054757 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.055076 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9774,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":38143,"thread_start_us":31860,"threads_started":1,"wal-append.queue_time_us":100765}
I20260709 12:26:23.095377  4748 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:26:23.091764  8868 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:23.124672  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.131580  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.096378  8928 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:23.131260  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:23.119690  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:23.149756  4749 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:23.150648  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:23.152210  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2862) took 1184 ms. Trace:
I20260709 12:26:23.153216  4673 rpcz_store.cc:276] 0709 12:26:21.924820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.924957 (+   137us) service_pool.cc:224] Handling call
0709 12:26:21.925249 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.965534 (+ 40285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.025390 (+1059856us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b540017dc54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:23.108789 (+ 83399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82276224}
I20260709 12:26:23.154341  8633 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10.
W20260709 12:26:23.155360  8941 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:26:23.156224  4673 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:23.157665  8823 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:23.157932  8799 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:23.158223  8823 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.158913  8799 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:23.161212  8823 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:23.128252  8868 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.165972  8880 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.166121  8642 raft_consensus.cc:3055] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:23.168922  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2873) took 1236 ms. Trace:
I20260709 12:26:23.170375  8916 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:23.170652  8943 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:26:23.138253  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:23.164359  8693 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:23.627095  4672 rpcz_store.cc:276] 0709 12:26:21.932136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.562298 (+630162us) service_pool.cc:224] Handling call
0709 12:26:22.562914 (+   616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.653553 (+ 90639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.168623 (+515070us) spinlock_profiling.cc:243] Waited 83.2 ms on lock 0x7b54001bd354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:23.168826 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191354752}
I20260709 12:26:23.119357  8831 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:23.207914  8799 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:23.186159  5347 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:23.209179  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:23.223292  4681 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:23.208420  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2867) took 1282 ms. Trace:
I20260709 12:26:23.231606  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:23.232365  8909 log.cc:927] Time spent T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:26:23.234764  4602 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:23.143186  8881 consensus_queue.cc:260] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.635032  4679 rpcz_store.cc:276] 0709 12:26:21.926425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.265796 (+339371us) service_pool.cc:224] Handling call
0709 12:26:23.172098 (+906302us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54001d29d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:23.208325 (+ 36227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230742912}
W20260709 12:26:23.236037  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 122934us
W20260709 12:26:23.245730  8949 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:26:23.247591  8931 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:23.092058  8800 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.148521  8765 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:23.250111  4678 raft_consensus.cc:2410] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 12:26:23.242625  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 286343us
I20260709 12:26:23.295060  4755 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
W20260709 12:26:23.227075  8950 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:26:23.211172  8868 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:23.258355  8947 log.cc:927] Time spent T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:26:23.320858  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:23.321825  4752 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:23.267853  8813 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:23.329877  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2885) took 1348 ms. Trace:
W20260709 12:26:23.334765  8618 consensus_meta.cc:287] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.000s
I20260709 12:26:23.342023  8865 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:23.345549  5348 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:23.236920  8857 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.251s
I20260709 12:26:23.270399  4496 catalog_manager.cc:5697] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 reported cstate change: config changed from index 2 to 4, term changed from 4 to 6, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 6 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:23.665335  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:23.666442  4752 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 13: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:26:23.438355  8953 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:26:23.439229  8770 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:23.608492  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.452373  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.589457  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.667838  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.601178  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:23.668481  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.221906  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49153.6us
I20260709 12:26:23.626845  8642 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:23.135455  8928 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.665760  8618 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:23.669490  8928 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.669891  8642 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.670565  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:23.671330  4747 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:23.672039  4750 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:23.672351  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:23.632427  4748 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
W20260709 12:26:23.594230  8924 consensus_meta.cc:287] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.654s	user 0.011s	sys 0.002s
I20260709 12:26:23.598111  8872 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:23.636439  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 630) took 1010 ms. Trace:
I20260709 12:26:23.660885  4680 rpcz_store.cc:276] 0709 12:26:21.981371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.881851 (+900480us) service_pool.cc:224] Handling call
0709 12:26:22.882343 (+   492us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:22.889006 (+  6663us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.889040 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:22.893145 (+  4105us) log.cc:844] Serialized 52 byte log entry
0709 12:26:23.095565 (+202420us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:23.095744 (+   179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.096016 (+   272us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.329547 (+233531us) raft_consensus.cc:1685] finished
0709 12:26:23.329594 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.329790 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6392,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":40829,"thread_start_us":201275,"threads_started":1,"wal-append.queue_time_us":407089}
I20260709 12:26:23.675833  4681 rpcz_store.cc:276] 0709 12:26:22.626129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.141557 (+515428us) service_pool.cc:224] Handling call
0709 12:26:23.177046 (+ 35489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.636211 (+459165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.636352 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.492923  8798 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.415660  8789 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.641952  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2826) took 1079 ms. Trace:
I20260709 12:26:23.645546  8848 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.645810  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:23.683944  4749 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:23.644492  4755 pending_rounds.cc:85] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 5
I20260709 12:26:23.646847  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2878) took 1679 ms. Trace:
I20260709 12:26:23.642000  8881 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:23.647981  8765 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:23.648926  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:23.650075  8831 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:23.656255  8939 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.694s	user 0.005s	sys 0.022s
I20260709 12:26:23.650720  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:23.662825  4753 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:23.454015  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1874) took 3186 ms. Trace:
I20260709 12:26:23.671411  8618 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:23.535640  8933 consensus_meta.cc:287] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.539s	user 0.013s	sys 0.000s
W20260709 12:26:23.667903  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.673245  4747 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:23.681443  4756 rpcz_store.cc:276] 0709 12:26:22.559932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.562206 (+  2274us) service_pool.cc:224] Handling call
0709 12:26:22.562933 (+   727us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:22.770596 (+207663us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:22.770630 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:22.771346 (+   716us) log.cc:844] Serialized 52 byte log entry
0709 12:26:22.861753 (+ 90407us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:22.861867 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.861957 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.638689 (+776732us) raft_consensus.cc:1685] finished
0709 12:26:23.638709 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.638852 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6607,"raft.queue_time_us":10940,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":10637,"thread_start_us":89880,"threads_started":1,"wal-append.queue_time_us":169677}
I20260709 12:26:23.671386  8642 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.685075  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:23.677995  5314 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:23.684710  4674 rpcz_store.cc:276] 0709 12:26:21.967965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.628367 (+660402us) service_pool.cc:224] Handling call
0709 12:26:22.628950 (+   583us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:22.654575 (+ 25625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.654644 (+    69us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:22.687335 (+ 32691us) log.cc:844] Serialized 78 byte log entry
0709 12:26:22.895065 (+207730us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:22.895115 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.895216 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.646600 (+751384us) raft_consensus.cc:1685] finished
0709 12:26:23.646625 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.646792 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":703,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":3700,"thread_start_us":128549,"threads_started":1,"wal-append.queue_time_us":213337}
I20260709 12:26:23.702308  8618 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:23.702749  8618 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.703125  8618 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.688992  4755 pending_rounds.cc:107] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:26:23.706023  4755 raft_consensus.cc:2889] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:23.706943  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2829) took 1122 ms. Trace:
I20260709 12:26:23.707257  8800 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:23.707523  8733 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:26:23.707706  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:23.699637  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2834) took 1089 ms. Trace:
I20260709 12:26:23.709704  8933 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.709453  4751 rpcz_store.cc:276] 0709 12:26:22.615703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.615852 (+   149us) service_pool.cc:224] Handling call
0709 12:26:22.665934 (+ 50082us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.725164 (+ 59230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.725199 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:22.852142 (+126943us) spinlock_profiling.cc:243] Waited 8.79 ms on lock 0x7b2c000d0108. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:26:22.852787 (+   645us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.865921 (+ 13134us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:22.865956 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.866020 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.693220 (+827200us) raft_consensus.cc:1685] finished
0709 12:26:23.693248 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.699567 (+  6319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28734,"raft.queue_time_us":29414,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":2257,"spinlock_wait_cycles":20228480,"thread_start_us":12613,"threads_started":1,"wal-append.queue_time_us":63794}
I20260709 12:26:23.711627  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:23.712107  8933 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:23.712476  4751 raft_consensus.cc:2410] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 11 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:23.688871  8848 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:23.689261  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:23.698261  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:23.714221  5314 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:23.714844  4645 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:23.715785  8831 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:23.716095  8848 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:23.716182  8831 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.717864  8831 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.718652  8770 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:23.720573  8789 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:23.720966  8931 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:23.721592  8931 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.722144  8931 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.679047  8924 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:23.713078  8813 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.727798  8813 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.691277  8872 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.730842  8813 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:23.731406  8872 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:23.694418  4601 rpcz_store.cc:276] 0709 12:26:20.267731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.865029 (+597298us) service_pool.cc:224] Handling call
0709 12:26:20.865340 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.902538 (+2037198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.453515 (+550977us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b54003520d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:23.453867 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4595751808}
I20260709 12:26:23.734474  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:23.735574  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:23.736370  5345 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
W20260709 12:26:23.737610  4675 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.006s	sys 0.000s
W20260709 12:26:23.738020  4675 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.007s	sys 0.000s
I20260709 12:26:23.739240  8661 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:23.739845  8661 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:23.743271  8661 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:23.743299  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:23.744817  5348 raft_consensus.cc:2368] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10.
W20260709 12:26:23.746493  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 95929.7us
I20260709 12:26:23.747260  8622 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:23.747473  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:23.748123  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:23.748301  4681 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:23.750561  4756 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:23.751036  4756 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:26:23.751466  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.713361  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:23.752328  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:23.752725  4751 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:23.752871  4749 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:23.699791  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:23.701058  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 52253.5us
I20260709 12:26:23.756042  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:23.707259  4755 rpcz_store.cc:276] 0709 12:26:22.584985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.595908 (+ 10923us) service_pool.cc:224] Handling call
0709 12:26:22.596378 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.706655 (+1110277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.706881 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:23.756668  4681 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:23.640517  8880 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.761391  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:23.762634  4753 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.728732  8924 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:23.766106  4754 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:26:23.751791  4756 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:23.781742  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2856) took 4341 ms. Trace:
I20260709 12:26:23.809814  4644 leader_election.cc:400] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9.
I20260709 12:26:23.810591  4644 leader_election.cc:403] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:23.811432  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 53904.2us
I20260709 12:26:23.760316  8693 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:23.788708  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:23.868468  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:23.858949  4754 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:23.857966  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:23.890091  8441 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.891058  8872 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.891824  8872 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.763291  8848 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:26:23.894745  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 140703us
I20260709 12:26:23.894971  8848 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:23.895840  4644 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:23.896592  8661 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9.
I20260709 12:26:23.897320  8661 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:23.897724  8661 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.898051  8661 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:23.901084  5349 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.161s	sys 0.190s
W20260709 12:26:23.901508  5349 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.161s	sys 0.190s
I20260709 12:26:23.890097  4756 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.687589  8798 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:23.974510  8848 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:23.997001  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:23.782352  4675 rpcz_store.cc:276] 0709 12:26:19.440458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.705868 (+265410us) service_pool.cc:224] Handling call
0709 12:26:19.706353 (+   485us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:20.086096 (+379743us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:20.097010 (+ 10914us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.618467 (+2521457us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b1800405880. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:26:22.785303 (+166836us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:22.939432 (+154129us) log.cc:844] Serialized 52 byte log entry
0709 12:26:22.954348 (+ 14916us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:23.738390 (+784042us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.738819 (+   429us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.738994 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.761318 (+ 22324us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.761363 (+    45us) raft_consensus.cc:1685] finished
0709 12:26:23.761375 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.781635 (+ 20260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1826,"prepare.queue_time_us":695541,"prepare.run_cpu_time_us":166,"prepare.run_wall_time_us":758,"raft.queue_time_us":115750,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":84407,"spinlock_wait_cycles":31120384,"thread_start_us":785113,"threads_started":2,"wal-append.queue_time_us":2706651}
I20260709 12:26:23.998030  4678 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:24.002715  5342 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
W20260709 12:26:24.008229  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 196908us
I20260709 12:26:23.975288  8933 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:23.964861  8789 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:23.975718  8693 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:23.998775  8924 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:24.017107  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:24.022581  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 259164us
I20260709 12:26:24.016584  4754 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:26:24.018847  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:24.029111  8693 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:24.030987  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.031810  8767 consensus_meta.cc:287] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.987s	user 0.012s	sys 0.000s
I20260709 12:26:24.032648  5344 raft_consensus.cc:2410] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:26:24.032692  5349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 567) took 1163 ms. Trace:
I20260709 12:26:24.023886  4754 pending_rounds.cc:85] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 7
I20260709 12:26:24.036108  4754 pending_rounds.cc:107] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:26:24.036477  4754 raft_consensus.cc:2889] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:24.036288  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:24.033160  5349 rpcz_store.cc:276] 0709 12:26:22.869334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.877131 (+  7797us) service_pool.cc:224] Handling call
0709 12:26:22.913859 (+ 36728us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.021882 (+108023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.021928 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:23.052949 (+ 31021us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.902018 (+849069us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:23.902080 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.902183 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:24.032447 (+130264us) raft_consensus.cc:1685] finished
0709 12:26:24.032466 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.032606 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8998,"raft.queue_time_us":9658,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":10211,"thread_start_us":848560,"threads_started":1,"wal-append.queue_time_us":974303}
I20260709 12:26:24.046957  4749 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:24.016187  8622 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.057843  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.059501  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:24.019502  8789 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.081348  4756 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:26:24.079404  8899 consensus_meta.cc:287] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.973s	user 0.004s	sys 0.007s
I20260709 12:26:24.086088  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:24.098148  4675 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 15: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:24.089115  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.089725  8500 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:24.091015  5314 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:24.022202  8928 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:24.099099  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.056411  8924 raft_consensus.cc:697] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:24.103540  4755 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:24.104029  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2848) took 1161 ms. Trace:
I20260709 12:26:24.106256  4756 rpcz_store.cc:276] 0709 12:26:22.935797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.749956 (+814159us) service_pool.cc:224] Handling call
0709 12:26:23.750352 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.097236 (+346884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.097449 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:24.107357  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.107993  4756 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:24.114012  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:24.114665  4675 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:24.120129  4749 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:24.121407  4681 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:24.127534  4753 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:24.132680  4753 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:24.124676  8813 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:24.137442  8813 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.137910  8813 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:24.138037  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.138962  4756 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:26:24.132520  8857 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:24.141767  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2310) took 6635 ms. Trace:
I20260709 12:26:24.142256  4602 rpcz_store.cc:276] 0709 12:26:17.506011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.207376 (+701365us) service_pool.cc:224] Handling call
0709 12:26:18.207685 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.637693 (+5430008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.141377 (+503684us) spinlock_profiling.cc:243] Waited 4.9 s on lock 0x7b54003552d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:24.141691 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11265568128}
I20260709 12:26:24.144085  8880 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:24.145670  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.146971  8868 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.152050  8618 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:24.154386  5347 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:24.155589  8879 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.156025  8928 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:24.142184  8857 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:24.162289  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 516978us
I20260709 12:26:24.167791  4755 raft_consensus.cc:2368] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11.
I20260709 12:26:24.110365  8500 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:24.168872  4681 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
W20260709 12:26:24.175166  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.180367  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:24.182241  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.181262  5349 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
W20260709 12:26:24.191195  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.201512  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.202289  4753 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
W20260709 12:26:24.205852  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 45325.7us
I20260709 12:26:24.206727  8872 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:24.206095  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.116989  8622 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.215843  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2843) took 1474 ms. Trace:
I20260709 12:26:24.218132  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:24.220599  5348 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:24.220311  4750 rpcz_store.cc:276] 0709 12:26:22.728662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.783965 (+ 55303us) service_pool.cc:224] Handling call
0709 12:26:24.202361 (+1418396us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b5400263c54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:24.202559 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2012621184}
I20260709 12:26:24.225003  8500 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:24.226265  5344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 639) took 1335 ms (client timeout 1602 ms). Trace:
W20260709 12:26:24.226887  5344 rpcz_store.cc:269] 0709 12:26:22.890402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.898690 (+  8288us) service_pool.cc:224] Handling call
0709 12:26:24.226016 (+1327326us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54007116d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:24.226173 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2535491968}
W20260709 12:26:24.228667  8931 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.506s	user 0.007s	sys 0.005s
I20260709 12:26:24.229641  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2888) took 2246 ms. Trace:
W20260709 12:26:24.211438  8881 consensus_meta.cc:287] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.000s
I20260709 12:26:24.232686  4676 rpcz_store.cc:276] 0709 12:26:21.983398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.054135 (+1070737us) service_pool.cc:224] Handling call
0709 12:26:23.108337 (+ 54202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.166718 (+ 58381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.166748 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.166808 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:23.166841 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.229307 (+1062466us) spinlock_profiling.cc:243] Waited 58 ms on lock 0x7b540017e154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:24.229354 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.229582 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133362560}
I20260709 12:26:24.156080  8879 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.218104  8868 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:24.193348  8831 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:24.242053  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.250097  4747 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:26:24.251227  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 43655.7us
I20260709 12:26:24.253875  4753 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:24.243237  8798 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:24.235129  8422 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:26:24.229127  8931 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.240548  8924 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.269652  8848 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:24.271173  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:24.271499  8629 consensus_queue.cc:1059] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:24.274971  8857 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:24.275326  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:24.276074  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:24.288837  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:24.290726  5342 raft_consensus.cc:2393] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:24.291496  5344 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
W20260709 12:26:24.294457  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 427224us
W20260709 12:26:24.269716  8959 log.cc:927] Time spent T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.123s	user 0.001s	sys 0.001s
W20260709 12:26:24.288885  8642 consensus_meta.cc:287] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.532s	user 0.012s	sys 0.000s
I20260709 12:26:24.316372  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:24.317297  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.317682  5345 raft_consensus.cc:2410] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:24.320901  4644 leader_election.cc:400] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10.
I20260709 12:26:24.321604  4644 leader_election.cc:403] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:26:24.327301  4750 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:26:24.247839  8900 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 12:26:24.333619  8693 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.334872  4748 raft_consensus.cc:2410] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:26:24.334698  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:24.340437  5346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 636) took 1449 ms. Trace:
W20260709 12:26:24.342895  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.324136  4571 connection.cc:256] RPC call timeout handler was delayed by -1.1729928830000063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2706s for a response
I20260709 12:26:24.357969  4644 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:24.360387  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.336185  8618 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:24.339351  8869 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10.
I20260709 12:26:24.360098  5341 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 11: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:24.344004  8900 consensus_queue.cc:1243] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:24.369414  8931 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:24.340989  5346 rpcz_store.cc:276] 0709 12:26:22.731062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.731371 (+   309us) service_pool.cc:224] Handling call
0709 12:26:22.731969 (+   598us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.761225 (+ 29256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.761257 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:22.789754 (+ 28497us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.930216 (+140462us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:23.132935 (+202719us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.133763 (+   828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.158310 (+1024547us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b54005e33d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:24.158381 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:24.158440 (+    59us) raft_consensus.cc:1685] finished
0709 12:26:24.179840 (+ 21400us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.179979 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2345,"prepare.queue_time_us":594476,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":8559,"raft.run_cpu_time_us":390,"raft.run_wall_time_us":8411,"spinlock_wait_cycles":25585920,"thread_start_us":274603,"threads_started":2,"wal-append.queue_time_us":257259,"wal-append.run_cpu_time_us":4091,"wal-append.run_wall_time_us":1121296}
I20260709 12:26:24.394385  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.324954  8868 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:26:24.365525  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:24.318576  8798 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:26:24.375336  8789 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:24.378162  8618 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.378299  8622 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:24.415671  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 50218.8us
I20260709 12:26:24.339097  8871 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:26:24.387658  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.369841  8933 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:24.408898  4673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.010s	sys 0.019s
W20260709 12:26:24.422255  4673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.010s	sys 0.019s
I20260709 12:26:24.423454  4754 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:24.384786  8661 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.418828  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:24.421124  4570 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.010s	sys 0.014s
I20260709 12:26:24.345907  8693 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:24.415274  8618 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:24.433118  4570 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.010s	sys 0.014s
W20260709 12:26:24.442730  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 132431us
I20260709 12:26:24.443174  8693 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:24.372717  8872 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:26:24.454299  4675 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:24.422389  8928 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:24.428978  8899 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:24.465087  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.409121  8813 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.465823  4753 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:24.463006  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.443652  8618 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:24.456286  8800 consensus_meta.cc:287] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.748s	user 0.013s	sys 0.000s
I20260709 12:26:24.479831  4752 raft_consensus.cc:2410] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:26:24.480387  8880 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.480738  8831 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:24.480978  8880 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.472108  4754 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:24.465337  8928 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.466722  8899 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:24.523818  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 53422.5us
I20260709 12:26:24.527022  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.528033  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.528635  4754 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
W20260709 12:26:24.531174  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 270457us
I20260709 12:26:24.265966  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:24.522442  8928 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:24.525730  8899 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:24.540345  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.547292  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:24.543195  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 185283us
I20260709 12:26:24.549317  4644 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:24.557655  4644 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:24.559089  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:24.561379  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.562057  4753 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:24.566223  4601 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:26:24.553973  8871 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:24.572039  4720 leader_election.cc:400] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12.
I20260709 12:26:24.572629  4720 leader_election.cc:403] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:26:24.579802  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:24.580544  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:24.581676  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 284325us
I20260709 12:26:24.586620  8798 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.606372  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:24.607579  4680 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:24.609584  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:24.629489  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:24.630344  4674 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:24.631654  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.632550  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.633191  4754 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:24.573139  8871 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:24.642175  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.642295  4753 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:24.646203  4756 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:24.644052  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:24.647733  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 65464.3us
W20260709 12:26:24.648542  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 57883.8us
I20260709 12:26:24.643419  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:24.650611  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:24.651675  4681 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:24.661561  4601 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:26:24.662350  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:24.568784  8622 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:26:24.650099  4672 raft_consensus.cc:2376] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11. Re-sending same reply.
I20260709 12:26:24.667809  4676 raft_consensus.cc:2368] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11.
I20260709 12:26:24.563696  8968 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 12:26:24.671741  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.587316  8798 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.628289  8618 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12.
I20260709 12:26:24.674309  4678 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:24.676498  4569 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:24.679527  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.644686  8871 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:24.684832  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:24.685606  4754 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:24.686096  4675 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:24.690676  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:24.691565  4598 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
W20260709 12:26:24.692821  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 326927us
W20260709 12:26:24.693672  8962 log.cc:927] Time spent T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 12:26:24.693866  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:24.694519  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:24.694584  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:26:24.698346  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.703212  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:24.654330  8837 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:26:24.705977  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 134115us
I20260709 12:26:24.639398  8789 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 12:26:24.710525  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 117179us
I20260709 12:26:24.713197  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:24.674774  8661 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:24.718590  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:24.719360  4754 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:24.664556  8500 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:26:24.686985  8813 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:24.723804  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:24.681636  8982 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:24.697495  8983 raft_consensus.cc:2764] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:26:24.728574  4675 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:24.706799  8838 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:24.720186  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:24.741447  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:24.742347  4601 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
W20260709 12:26:24.730264  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 53894.8us
I20260709 12:26:24.733345  4678 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:24.738260  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:24.752162  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:24.754738  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:24.762637  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:24.763680  5344 raft_consensus.cc:2393] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 12: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:24.765820  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:24.775774  4598 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:24.771476  4754 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:24.777678  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:24.732381  8789 consensus_queue.cc:1243] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:24.783653  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:24.784286  4672 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:24.785054  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:24.787575  4569 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:24.778409  4754 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:24.752837  5346 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
W20260709 12:26:24.781515  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.797736  4753 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:24.800197  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:24.800675  5348 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:24.749794  8966 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:24.813174  4601 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:24.766765  8838 raft_consensus.cc:697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:24.799120  4647 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.959s (SENT) [suppressed 36 similar messages]
I20260709 12:26:24.722743  8837 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:24.789698  8924 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:24.825917  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:24.827836  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.830340  4672 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:24.830636  4754 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:24.831526  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.832290  4754 raft_consensus.cc:2410] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:26:24.818151  8979 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.135s
W20260709 12:26:24.836588  8975 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:26:24.842083  5341 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:26:24.842231  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.843680  8806 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:26:24.843989  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 90715.2us
I20260709 12:26:24.844414  8806 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:24.845151  4569 leader_election.cc:400] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11.
I20260709 12:26:24.845290  8879 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:24.845481  4569 leader_election.cc:403] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:24.846218  8973 log.cc:927] Time spent T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 12:26:24.846410  8770 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:24.846869  8770 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:24.847306  8900 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:24.847748  8996 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:26:24.847813  8900 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:24.846218  4569 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:24.747032  8982 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:24.823513  8974 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:26:24.865115  4754 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:26:24.868348  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 173713us
I20260709 12:26:24.874837  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:24.875540  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:24.876286  4597 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:24.876684  8798 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:24.845491  8806 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:24.878111  4598 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:24.878152  4597 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:24.879354  8982 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:24.883247  8900 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.868561  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:24.875648  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 132505us
W20260709 12:26:24.864141  8965 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
W20260709 12:26:24.905154  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 59001.8us
I20260709 12:26:24.908367  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:24.918532  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.847785  8770 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:24.909361  8928 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.928802  4647 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:24.929402  8869 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:24.930063  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:24.930333  8869 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:24.930490  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:24.930766  8869 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:24.931207  8869 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:24.931542  8984 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:26:24.932536  8661 raft_consensus.cc:3055] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:24.932921  8661 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:24.899912  8982 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:24.933689  8982 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:24.905103  8838 consensus_queue.cc:237] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.935838  8982 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:24.937386  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 636) took 2053 ms. Trace:
I20260709 12:26:24.938151  5312 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:24.938585  8837 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:24.922508  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:24.921535  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47732.2us
I20260709 12:26:24.939406  4603 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
W20260709 12:26:24.930364  8960 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.305s	user 0.015s	sys 0.002s
I20260709 12:26:24.940119  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:24.939278  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:24.938627  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:24.947429  5345 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:24.926105  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:24.947618  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 265873us
I20260709 12:26:24.941663  4679 raft_consensus.cc:2393] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:24.933991  8661 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.942236  4647 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:24.946805  4678 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:24.949864  4571 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:24.950779  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:24.921733  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:24.953092  8982 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:24.921417  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:24.953649  8982 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.953215  4599 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:24.954102  8982 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:24.922039  5341 pending_rounds.cc:85] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 7
I20260709 12:26:24.954420  4647 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:24.954882  5341 pending_rounds.cc:107] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:26:24.955240  5341 raft_consensus.cc:2889] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:24.955286  8831 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:24.956099  8831 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.937848  4673 rpcz_store.cc:276] 0709 12:26:22.883615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.174984 (+291369us) service_pool.cc:224] Handling call
0709 12:26:23.623395 (+448411us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:23.663983 (+ 40588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.664003 (+    20us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:23.686700 (+ 22697us) log.cc:844] Serialized 52 byte log entry
0709 12:26:24.422887 (+736187us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:24.422932 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.423011 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:24.937160 (+514149us) raft_consensus.cc:1685] finished
0709 12:26:24.937180 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.937331 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5398,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":125,"thread_start_us":657237,"threads_started":1,"wal-append.queue_time_us":777595}
W20260709 12:26:24.956583  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 48133.7us
I20260709 12:26:24.957675  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:24.954097  4605 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:24.925974  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:24.958981  8998 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:24.959118  8872 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:24.959623  8872 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.959975  8872 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:24.961910  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:24.962639  4678 raft_consensus.cc:2393] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:24.963109  8982 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:24.964214  8661 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:24.959926  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:24.963443  8871 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.967458  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:24.971217  4753 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:24.980343  5345 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
W20260709 12:26:24.924068  8991 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 12:26:24.922660  8622 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:24.986593  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.005714  4753 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:24.987123  8629 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:25.007181  8629 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.007573  8629 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:25.007669  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.009929  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:25.011951  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 102741us
I20260709 12:26:25.012265  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.013885  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:25.014633  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 75443.7us
I20260709 12:26:25.015689  5312 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:24.952399  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:24.946871  8928 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:24.956650  8831 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:24.995314  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:24.997725  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47946.1us
I20260709 12:26:24.970218  8984 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.019865  4756 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:25.022363  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:25.032383  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 686) took 1869 ms. Trace:
I20260709 12:26:24.997267  8693 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:25.055348  8693 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.055884  8693 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:25.057173  8931 raft_consensus.cc:3055] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:25.057722  8931 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:25.058889  4568 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:25.059752  4644 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:25.060572  8857 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:25.061048  8857 raft_consensus.cc:697] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:25.061599  8500 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:25.062044  8500 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.062551  8500 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:25.061990  8857 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:25.063690  8982 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:25.064186  8964 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:25.065790  4571 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:25.066247  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2884) took 3085 ms. Trace:
I20260709 12:26:25.067632  8447 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:25.066614  4677 rpcz_store.cc:276] 0709 12:26:21.981117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.872507 (+891390us) service_pool.cc:224] Handling call
0709 12:26:22.915971 (+ 43464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.053708 (+137737us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.053740 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:23.092788 (+ 39048us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.440535 (+347747us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:23.440592 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.440691 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:24.784447 (+1343756us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:25.065944 (+281497us) raft_consensus.cc:1685] finished
0709 12:26:25.065962 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.066189 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31734,"raft.queue_time_us":101532,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":26180,"spinlock_wait_cycles":10029440,"thread_start_us":264407,"threads_started":1,"wal-append.queue_time_us":497867}
I20260709 12:26:25.068006  8447 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.068372  8447 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:25.068951  4677 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:26:25.072048  8992 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:25.072809  4681 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:25.051059  4496 catalog_manager.cc:5697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 5 to 7, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 7 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:25.075708  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2852) took 2073 ms. Trace:
I20260709 12:26:25.076109  4751 rpcz_store.cc:276] 0709 12:26:23.002193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.759951 (+757758us) service_pool.cc:224] Handling call
0709 12:26:23.856658 (+ 96707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.026020 (+169362us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:24.323428 (+297408us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.074724 (+751296us) spinlock_profiling.cc:243] Waited 5.48 ms on lock 0x7b18000e4580. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effee05e4bc 00007effeb3ac48b 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a
0709 12:26:25.075186 (+   462us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.075366 (+   180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:25.075411 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:25.075464 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.075532 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.075649 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":507013,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":2111,"raft.queue_time_us":2151,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"spinlock_wait_cycles":12612480,"thread_start_us":452836,"threads_started":2,"wal-append.queue_time_us":504578}
I20260709 12:26:25.036909  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.037209  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:25.001133  8798 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:25.042508  8933 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:25.043344  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.045094  4749 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:25.045693  8984 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:24.985495  8879 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.063s
W20260709 12:26:25.048101  4499 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52812 (request call id 291) took 27234 ms (client timeout 15000 ms). Trace:
I20260709 12:26:25.050012  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:25.050092  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.051028  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.051393  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.052177  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.053076  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:25.053953  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.054039  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:25.077404  4571 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:24.996929  8618 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:26:25.041340  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.117725  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.053805  8767 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:25.121970  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:25.122396  8767 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.122786  4604 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 15: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:25.122882  8767 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:25.092101  8899 consensus_meta.cc:287] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.538s	user 0.007s	sys 0.004s
I20260709 12:26:25.124881  8899 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:25.125283  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:25.125970  4754 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:26:25.126616  5347 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:26:25.127990  4679 raft_consensus.cc:2393] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:25.129170  4679 pending_rounds.cc:85] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 3
I20260709 12:26:25.129449  4679 pending_rounds.cc:107] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:26:25.129698  4679 raft_consensus.cc:2889] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:25.035921  4755 raft_consensus.cc:2393] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:25.035637  4747 rpcz_store.cc:276] 0709 12:26:23.162901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.965836 (+802935us) service_pool.cc:224] Handling call
0709 12:26:24.005160 (+ 39324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.252825 (+247665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.032138 (+779313us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b5400308c54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:25.032318 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":552101248}
I20260709 12:26:25.086248  4672 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:25.115655  4598 raft_consensus.cc:2368] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10.
I20260709 12:26:25.115882  4756 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:25.119338  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:25.137732  4600 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:25.121821  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:24.989144  8997 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
W20260709 12:26:25.144387  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.144706  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:25.150266  4603 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:25.144161  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.090830  8933 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:25.156412  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 65675.3us
I20260709 12:26:25.090394  8798 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:25.161841  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.157835  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.164412  4755 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
W20260709 12:26:25.165962  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 88634.6us
W20260709 12:26:25.166527  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.137130  8881 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:25.138056  8996 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11.
W20260709 12:26:25.177011  4748 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 689) took 2013 ms (client timeout 2249 ms). Trace:
I20260709 12:26:25.159806  8622 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:25.184785  4571 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:25.133975  8983 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:25.179554  4496 catalog_manager.cc:5697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 6 to 9, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 9 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:25.181854  4748 rpcz_store.cc:269] 0709 12:26:23.163691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.707162 (+543471us) service_pool.cc:224] Handling call
0709 12:26:25.176758 (+1469596us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b5400117954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:25.176931 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438910208}
I20260709 12:26:25.158963  8933 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:25.243525  4719 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:25.247189  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 62513.1us
I20260709 12:26:25.167377  8798 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:25.249419  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 212313us
I20260709 12:26:25.257153  4754 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:25.179181  8881 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:25.182431  8899 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:25.262596  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:25.111608  4499 rpcz_store.cc:269] 0709 12:25:57.818692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.818899 (+   207us) service_pool.cc:224] Handling call
0709 12:26:25.047840 (+27228941us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:14.214220 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:14.448681 (+234461us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:14.448746 (+    65us) write_op.cc:435] Acquired schema lock
0709 12:26:14.448762 (+    16us) tablet.cc:662] Decoding operations
0709 12:26:14.453429 (+  4667us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:14.453508 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:14.453553 (+    45us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:26:14.455368 (+  1815us) tablet.cc:701] Row locks acquired
0709 12:26:14.455412 (+    44us) write_op.cc:260] PREPARE: finished
0709 12:26:14.455588 (+   176us) write_op.cc:270] Start()
0709 12:26:14.455738 (+   150us) write_op.cc:276] Timestamp: P: 1783599974455506 usec, L: 0
0709 12:26:14.455760 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:26:14.621015 (+165255us) log.cc:844] Serialized 9033 byte log entry
0709 12:26:15.950297 (+1329282us) op_driver.cc:464] REPLICATION: finished
0709 12:26:16.122105 (+171808us) write_op.cc:301] APPLY: starting
0709 12:26:16.159427 (+ 37322us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:16.173697 (+ 14270us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:16.173851 (+   154us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:16.179120 (+  5269us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:16.179850 (+   730us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=23
0709 12:26:16.179907 (+    57us) write_op.cc:312] APPLY: finished
0709 12:26:16.180452 (+   545us) log.cc:844] Serialized 158 byte log entry
0709 12:26:16.180675 (+   223us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:16.180901 (+   226us) write_op.cc:454] Released schema lock
0709 12:26:16.180982 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:16.181429 (+   447us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:16.181460 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3874824,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":137884,"apply.run_cpu_time_us":6349,"apply.run_wall_time_us":78758,"num_ops":23,"prepare.queue_time_us":152634,"prepare.run_cpu_time_us":27655,"prepare.run_wall_time_us":789654,"raft.queue_time_us":474972,"raft.run_cpu_time_us":34500,"raft.run_wall_time_us":113439,"replication_time_us":1494366,"thread_start_us":668903,"threads_started":4,"wal-append.queue_time_us":329641,"wal-append.run_cpu_time_us":19971,"wal-append.run_wall_time_us":2168737}]]}
I20260709 12:26:25.265009  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.276326  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:25.276050  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:25.278384  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.281513  4571 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:25.284834  4754 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:26:25.290323  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 268038us
I20260709 12:26:25.280081  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:25.291855  4748 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:25.258360  8789 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:25.249089  8982 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:25.250393  8928 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:25.307646  5312 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:25.308813  5312 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:25.300591  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.301821  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.309880  4754 raft_consensus.cc:2393] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:25.309924  4755 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:25.306228  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:25.274653  8881 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:25.307176  5342 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:25.276074  8800 raft_consensus.cc:2764] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:26:25.310704  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.283413  4496 catalog_manager.cc:5697] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:25.314532  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.316057  4756 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:25.318305  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:25.319072  4748 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:25.320056  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:25.320503  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:25.323122  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:25.324406  4754 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:25.331609  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.335479  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:25.335216  4496 catalog_manager.cc:5697] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 3 to 4, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 4 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:25.333454  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:25.336392  4756 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:25.334925  4755 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:25.256880  8983 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:25.345889  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:25.312444  8964 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:25.348096  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.349763  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.350587  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:25.351320  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 45165.7us
I20260709 12:26:25.352488  8800 raft_consensus.cc:2764] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:25.354043  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:25.354323  8984 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:25.355309  9002 log.cc:927] Time spent T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:26:25.318064  8899 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:25.337222  8996 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:26:25.311477  8982 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:25.358995  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.392208  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:25.360881  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.393000  4756 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:25.374048  4604 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:25.355650  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:25.378618  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.380843  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.303928  8900 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:26:25.385265  4645 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:25.389353  4571 leader_election.cc:400] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11.
I20260709 12:26:25.365882  4755 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:25.393797  4599 raft_consensus.cc:2410] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 12:26:25.397439  4752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 699) took 1966 ms (client timeout 1653 ms). Trace:
I20260709 12:26:25.348644  8838 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:25.313668  8928 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:25.397979  4752 rpcz_store.cc:269] 0709 12:26:23.438312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.679617 (+241305us) service_pool.cc:224] Handling call
0709 12:26:25.397204 (+1717587us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b540030aa54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:25.397360 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1760519040}
I20260709 12:26:25.363126  8964 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:25.431624  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.426679  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.434924  4756 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:25.406404  4747 raft_consensus.cc:2410] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:26:25.391906  8618 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.446925  4600 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:25.450838  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:25.451568  4752 raft_consensus.cc:2376] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12. Re-sending same reply.
I20260709 12:26:25.453255  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.458379  4754 raft_consensus.cc:2410] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:26:25.459947  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:25.461462  4756 raft_consensus.cc:2376] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12. Re-sending same reply.
I20260709 12:26:25.432293  8984 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:25.439997  8964 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:25.467944  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:25.450088  9002 log.cc:927] Time spent T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:26:25.393453  8869 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:25.478664  4678 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:25.454640  8871 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:25.401474  8601 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.283s
I20260709 12:26:25.474820  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:25.495138  4755 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:25.495349  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:25.501758  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.501988  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:25.504865  4496 catalog_manager.cc:5697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 3 to 5, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 5 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:25.503963  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 159372us
I20260709 12:26:25.457231  8800 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:26:25.527235  4598 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:25.453814  8618 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:25.555951  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 129225us
I20260709 12:26:25.566500  4571 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:25.569240  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 184062us
I20260709 12:26:25.577099  8800 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:25.576656  4600 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:25.579597  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.580402  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:25.583227  4755 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
W20260709 12:26:25.588922  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 308894us
I20260709 12:26:25.590466  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.590788  4571 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:25.591265  4600 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:25.590456  4647 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:25.600304  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 244741us
I20260709 12:26:25.611126  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:25.587973  8789 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:25.619508  4489 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.660s	user 0.001s	sys 0.014s
I20260709 12:26:25.585116  9022 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:25.623087  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:25.624393  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:25.627316  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57196.9us
I20260709 12:26:25.621258  4496 catalog_manager.cc:5697] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 3 to 6, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 6 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:25.626775  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 634) took 1162 ms. Trace:
I20260709 12:26:25.634912  4602 rpcz_store.cc:276] 0709 12:26:24.523564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.539174 (+ 15610us) service_pool.cc:224] Handling call
0709 12:26:24.570960 (+ 31786us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:24.571510 (+   550us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:24.571844 (+   334us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.065040 (+493196us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:25.115328 (+ 50288us) log.cc:844] Serialized 78 byte log entry
0709 12:26:25.115501 (+   173us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:25.115535 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.115607 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:25.615749 (+500142us) raft_consensus.cc:1685] finished
0709 12:26:25.615774 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.619391 (+  3617us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87872,"prepare.queue_time_us":30646,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":95516,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":5964,"thread_start_us":377142,"threads_started":1,"wal-append.queue_time_us":571776}
W20260709 12:26:25.639673  9019 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:26:25.640740  9006 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
W20260709 12:26:25.642177  9005 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:26:25.623963  8983 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:25.644517  8982 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:25.645303  4679 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:25.646842  9029 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:25.646322  4647 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:25.646701  8618 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:25.613911  8800 raft_consensus.cc:3055] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:25.672843  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:25.658047  4679 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:26:25.658725  4603 consensus_meta.cc:287] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.003s
W20260709 12:26:25.655750  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 44732.3us
I20260709 12:26:25.674201  4603 raft_consensus.cc:2410] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:25.674891  4571 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:25.677053  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2913) took 1017 ms. Trace:
I20260709 12:26:25.682027  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:26:25.674361  8629 consensus_meta.cc:287] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.666s	user 0.011s	sys 0.000s
I20260709 12:26:25.741971  8629 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:25.744191  8629 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:25.744935  8831 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.629s	user 0.006s	sys 0.002s
W20260709 12:26:25.692786  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 64310us
W20260709 12:26:25.701324  9017 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
W20260709 12:26:25.703286  8872 consensus_meta.cc:287] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.743s	user 0.008s	sys 0.004s
I20260709 12:26:25.679215  8968 raft_consensus.cc:2764] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:26:25.707100  4674 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:25.687358  8997 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:25.629287  9031 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:25.710554  8693 consensus_meta.cc:287] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.654s	user 0.011s	sys 0.000s
I20260709 12:26:25.711071  9024 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:25.731760  9018 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 12:26:25.716161  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48198 (request call id 646) took 1017 ms. Trace:
I20260709 12:26:25.716517  8880 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:25.688498  9030 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:25.717381  9029 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:25.691277  8928 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:25.680064  4680 rpcz_store.cc:276] 0709 12:26:24.660202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.079612 (+419410us) service_pool.cc:224] Handling call
0709 12:26:25.134416 (+ 54804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.134619 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:25.152728 (+ 18109us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.676272 (+523544us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.676634 (+   362us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.676765 (+   131us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:25.676804 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:25.676850 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.676907 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.677000 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3755,"prepare.queue_time_us":21028,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":74,"raft.queue_time_us":194274,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":12875,"thread_start_us":501238,"threads_started":2,"wal-append.queue_time_us":307917}
I20260709 12:26:25.721375  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.722146  5347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 574) took 1083 ms. Trace:
I20260709 12:26:25.723212  9026 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:25.681675  8933 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:25.692090  8983 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 12:26:25.730159  8879 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:25.798817  8983 raft_consensus.cc:3055] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:25.731746  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:25.799317  8983 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:25.758747  9031 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:25.800154  9031 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:25.800439  8983 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:25.731945  9028 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.801992  9028 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:25.804747  9028 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:25.732363  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.808948  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:25.733765  9016 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:26:25.810057  4755 raft_consensus.cc:2410] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:26:25.737012  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 648) took 1010 ms. Trace:
I20260709 12:26:25.810911  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.811097  4672 rpcz_store.cc:276] 0709 12:26:24.726281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.151030 (+424749us) service_pool.cc:224] Handling call
0709 12:26:25.172135 (+ 21105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.397341 (+225206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.397390 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:25.397453 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:25.397503 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.736658 (+339155us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b54001d33d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:25.736729 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.736925 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":517294336}
I20260709 12:26:25.811877  4749 raft_consensus.cc:2410] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:26:25.739065  8931 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:25.812358  8447 consensus_meta.cc:287] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.744s	user 0.006s	sys 0.006s
I20260709 12:26:25.812716  8869 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:25.747028  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.677593  8767 consensus_meta.cc:287] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.554s	user 0.012s	sys 0.000s
W20260709 12:26:25.813802  8500 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.751s	user 0.013s	sys 0.000s
I20260709 12:26:25.790999  8984 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:25.747959  4748 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:25.808058  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:25.677678  8800 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:26:25.758872  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 143626us
I20260709 12:26:25.753026  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:25.679566  8900 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:25.622740  8838 raft_consensus.cc:697] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:25.760622  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.761547  9023 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:25.755218  8872 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:25.763330  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:25.990023  5341 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:25.764256  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2326) took 1068 ms. Trace:
I20260709 12:26:25.764448  5346 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:26:25.765861  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:25.712767  9004 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.258s	user 0.003s	sys 0.001s
I20260709 12:26:25.762385  5312 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:25.999511  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 237953us
I20260709 12:26:26.000756  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:25.772140  4681 rpcz_store.cc:276] 0709 12:26:24.698288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.714446 (+ 16158us) service_pool.cc:224] Handling call
0709 12:26:25.715957 (+1001511us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54001bc954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:25.716088 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":758855040}
I20260709 12:26:25.766059  8770 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:25.778455  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2915) took 1095 ms. Trace:
I20260709 12:26:26.002678  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:25.771437  9024 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:25.686627  8992 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 12:26:25.789949  5347 rpcz_store.cc:276] 0709 12:26:24.638283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.638655 (+   372us) service_pool.cc:224] Handling call
0709 12:26:24.666717 (+ 28062us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:25.127052 (+460335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.717445 (+590393us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b54005749d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:25.722066 (+  4621us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1056963840}
I20260709 12:26:25.767293  8693 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:25.814105  8767 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:25.757328  4645 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:25.680764  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:25.812855  8447 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:25.757858  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.039167  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 281895us
W20260709 12:26:25.777608  8661 consensus_meta.cc:287] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.813s	user 0.012s	sys 0.001s
I20260709 12:26:25.815828  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2321) took 1249 ms. Trace:
I20260709 12:26:25.810958  4756 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:26:25.942380  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 194220us
I20260709 12:26:25.943436  9010 raft_consensus.cc:2764] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:26:25.944160  4599 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:26.050311  9032 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:26.051107  8767 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:25.945021  8933 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:25.946029  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:25.947328  8881 consensus_meta.cc:287] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.616s	user 0.006s	sys 0.005s
I20260709 12:26:26.053788  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:26.054069  8881 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:26.054749  8996 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:26.055192  8996 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:26.056679  8881 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:26.046839  8872 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:25.947886  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:26.057771  4751 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:26.058104  8984 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:26.058601  8984 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:25.951695  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:25.953512  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:26.059783  4754 raft_consensus.cc:2376] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10. Re-sending same reply.
I20260709 12:26:26.059996  4755 raft_consensus.cc:2410] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:26:26.060815  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:26.061059  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:25.955282  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:25.956223  8798 consensus_meta.cc:287] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.676s	user 0.008s	sys 0.000s
I20260709 12:26:25.957777  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:25.745384  8831 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:25.962039  9018 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:26:25.964136  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:25.933844  4604 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:26:25.841888  8983 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:25.967885  8964 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:25.981119  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:26.115028  5345 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:25.981957  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:25.814309  8500 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:26.071652  8798 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:26.121766  8798 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:25.988072  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:26.122758  4678 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:25.990820  4750 rpcz_store.cc:276] 0709 12:26:24.696116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.014834 (+318718us) service_pool.cc:224] Handling call
0709 12:26:25.015128 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:25.079288 (+ 64160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.079317 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:25.111937 (+ 32620us) log.cc:844] Serialized 26 byte log entry
0709 12:26:25.295593 (+183656us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:25.295650 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.296671 (+  1021us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:25.764049 (+467378us) raft_consensus.cc:1685] finished
0709 12:26:25.764067 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.764199 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71727,"raft.queue_time_us":64252,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":34648,"thread_start_us":164144,"threads_started":1,"wal-append.queue_time_us":315433}
I20260709 12:26:26.022413  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.124996  4596 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:26.026118  4675 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:26.002847  4677 rpcz_store.cc:276] 0709 12:26:24.683291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.137515 (+454224us) service_pool.cc:224] Handling call
0709 12:26:25.155447 (+ 17932us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.155639 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:25.155981 (+   342us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.685378 (+529397us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.778074 (+ 92696us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:25.778157 (+    83us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:25.778207 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.778270 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.778394 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":277254,"prepare.queue_time_us":40947,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":26378,"raft.queue_time_us":219128,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":302305,"thread_start_us":607938,"threads_started":4,"wal-append.queue_time_us":341805}
W20260709 12:26:26.125559  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 103264us
I20260709 12:26:26.042421  4645 leader_election.cc:400] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10.
W20260709 12:26:26.127578  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 87648.5us
I20260709 12:26:26.025005  8879 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:26.131501  4673 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:26.017230  8857 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.137867  8857 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:26.046381  8838 consensus_queue.cc:237] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:26.145881  4756 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:26:26.146823  8693 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:26.147013  8964 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:26.122782  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:26.147186  8976 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:26.046962  8982 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:26.148327  8982 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:26.148808  8982 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:26.153888  8871 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.047506  9035 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:26.158247  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:26.159180  4750 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:26.056362  8996 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:25.949867  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:26.161355  4749 raft_consensus.cc:2410] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:26:26.055090  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:26.066023  4755 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:26.024824  9024 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:26.065676  4747 raft_consensus.cc:2376] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8. Re-sending same reply.
I20260709 12:26:26.042976  4753 rpcz_store.cc:276] 0709 12:26:24.566186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.013990 (+447804us) service_pool.cc:224] Handling call
0709 12:26:25.014339 (+   349us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:25.078056 (+ 63717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.078102 (+    46us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:25.079037 (+   935us) log.cc:844] Serialized 78 byte log entry
0709 12:26:25.298796 (+219759us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:25.298834 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.298917 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:25.815581 (+516664us) raft_consensus.cc:1685] finished
0709 12:26:25.815603 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.815773 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63218,"raft.queue_time_us":64301,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":71,"thread_start_us":219300,"threads_started":1,"wal-append.queue_time_us":391525}
I20260709 12:26:26.088053  4601 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:26:26.117491  4605 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:26.117462  5314 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.114765  8601 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:26.119347  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 155297us
I20260709 12:26:26.122169  8500 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:26.124439  5344 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:26.040545  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 359842us
I20260709 12:26:26.043726  9022 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:26.133718  8447 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:26.069185  8997 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:26.144126  8857 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:25.998603  9031 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:26.157367  9026 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.058648  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:26.161933  9035 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.167495  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.172252  5345 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:26.173635  4747 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:26:26.207173  4747 pending_rounds.cc:85] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 8
I20260709 12:26:26.207466  4747 pending_rounds.cc:107] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:26:26.207692  4747 raft_consensus.cc:2889] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:26.154508  8871 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:26.177850  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.142390  8879 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.212688  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:26.212848  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.213387  4749 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.072163  8800 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:26.181810  4571 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:26:26.166867  9004 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:26.215953  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2907) took 1615 ms. Trace:
I20260709 12:26:26.216444  4676 rpcz_store.cc:276] 0709 12:26:24.600895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.675389 (+ 74494us) service_pool.cc:224] Handling call
0709 12:26:24.697270 (+ 21881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:24.746366 (+ 49096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:24.746402 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:24.766152 (+ 19750us) log.cc:844] Serialized 26 byte log entry
0709 12:26:25.250448 (+484296us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:25.283404 (+ 32956us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.436246 (+152842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.436331 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:25.711982 (+275651us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:26.215531 (+503549us) raft_consensus.cc:1685] finished
0709 12:26:26.215681 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.215871 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44729,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":100583,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":9801,"spinlock_wait_cycles":25293952,"thread_start_us":620934,"threads_started":2,"wal-append.queue_time_us":571527}
I20260709 12:26:26.188184  4753 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:26:26.176815  9018 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:26:26.199044  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.195647  4751 raft_consensus.cc:2376] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11. Re-sending same reply.
I20260709 12:26:26.208798  4677 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:26.210536  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.176853  8813 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:26.227720  4597 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:26.227566  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:26.238922  8984 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:26.242309  5343 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:26.243451  8871 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:26.241304  8984 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:26.244690  5347 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:26:26.246152  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 141588us
I20260709 12:26:26.247795  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:26:26.254070  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 76400.8us
I20260709 12:26:26.255503  4647 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.258041  4749 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
W20260709 12:26:26.294054  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 46375.6us
I20260709 12:26:26.295949  4750 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:26.297735  4750 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:26.298336  4750 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:26.223270  9035 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:26.210615  8798 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:26.307320  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:26.303779  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:26.306300  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:26.377779  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2370) took 1024 ms. Trace:
I20260709 12:26:26.433072  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2360) took 1255 ms. Trace:
W20260709 12:26:26.317866  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 110738us
I20260709 12:26:26.318284  4672 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:26.452076  4601 rpcz_store.cc:276] 0709 12:26:25.178174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.179361 (+  1187us) service_pool.cc:224] Handling call
0709 12:26:25.183979 (+  4618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.174528 (+990549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.432804 (+258276us) spinlock_profiling.cc:243] Waited 888 ms on lock 0x7b5400300a54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:26.433007 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2043270528}
W20260709 12:26:26.311825  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 61279.5us
W20260709 12:26:26.314810  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 133105us
I20260709 12:26:26.362274  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:26.318691  5347 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:26.316493  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.395712  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48198 (request call id 645) took 1698 ms. Trace:
I20260709 12:26:26.361900  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:26.224393  8879 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.353955  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:26.306340  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 95887.5us
I20260709 12:26:26.312755  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:26.451844  4677 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:26.456821  4673 rpcz_store.cc:276] 0709 12:26:24.697118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.957369 (+260251us) service_pool.cc:224] Handling call
0709 12:26:26.395363 (+1437994us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540019fd54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:26.395578 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2697497472}
W20260709 12:26:26.310643  9051 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:26:26.448069  4678 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
W20260709 12:26:26.456914  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.459975  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 204590us
I20260709 12:26:26.460759  4645 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:26.246942  8661 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:26.454522  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.451906  4598 rpcz_store.cc:276] 0709 12:26:25.351257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.601614 (+250357us) service_pool.cc:224] Handling call
0709 12:26:25.602031 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:25.628817 (+ 26786us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.628864 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:25.674731 (+ 45867us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.067371 (+392640us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:26.113951 (+ 46580us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.114245 (+   294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.114364 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:26.341707 (+227343us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b240011a698. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb24f9c4 00007effeb24dacb 00007effeb408dc9 00007effeb408a55 00007effeb407fb0 00007effee07e488 00007effee07ebe8 00007effee056398 00007effee05176a 00007effee050bb6
0709 12:26:26.363317 (+ 21610us) raft_consensus.cc:1685] finished
0709 12:26:26.375232 (+ 11915us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.375407 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12522,"prepare.queue_time_us":46343,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":62103,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":17731,"spinlock_wait_cycles":8181632,"thread_start_us":290307,"threads_started":1,"wal-append.queue_time_us":408326}
I20260709 12:26:26.232313  8857 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:26.257215  8767 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:26.307046  9010 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:26.467458  5344 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:26.342012  9033 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:26:26.463166  4750 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:26.464691  8871 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:26.312199  8899 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.161s
I20260709 12:26:26.337649  8997 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.140s
I20260709 12:26:26.477519  8976 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.318269  8984 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:26.340601  9012 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.315186  9050 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:26.475378  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.300868  9040 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:26.231950  8881 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.235s
I20260709 12:26:26.476446  8813 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:26.500046  8813 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.500737  8813 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:26.503695  8982 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:26.293839  8838 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.179s
I20260709 12:26:26.505860  8871 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:26.353026  4644 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:26.462499  9031 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:26.510329  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 157367us
I20260709 12:26:26.485011  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:26.478832  8928 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:26.511993  8928 raft_consensus.cc:697] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:26.512635  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:26.469645  4647 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.473150  8848 consensus_queue.cc:1059] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:26:26.513175  8928 consensus_queue.cc:237] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:26.513976  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:26.487061  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:26.514654  4756 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:26.483008  4596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2357) took 1429 ms (client timeout 1887 ms). Trace:
W20260709 12:26:26.515205  4596 rpcz_store.cc:269] 0709 12:26:25.049292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.120339 (+ 71047us) service_pool.cc:224] Handling call
0709 12:26:26.478628 (+1358289us) spinlock_profiling.cc:243] Waited 971 ms on lock 0x7b5400140854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:26.478803 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2233536768}
I20260709 12:26:26.317960  9030 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.155s
I20260709 12:26:26.490447  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:26.486110  4750 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:26.517168  8813 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:26.517639  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 48096us
I20260709 12:26:26.518342  5343 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:26:26.518546  4647 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.519134  9035 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:26.519502  9022 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:26.520656  9022 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:26.520987  9022 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:26.483124  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:26.521356  9022 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.521844  4601 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.524010  9025 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:26.524755  9025 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:26.525070  9025 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:26.496515  8976 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:26.498878  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.525954  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.525851  9025 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:26.526472  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.510461  8871 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:26.527083  9025 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:26.527168  8871 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:26.527963  8976 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:26.528290  8871 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:26.514094  8848 raft_consensus.cc:3055] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:26.514726  4602 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:26.359295  9037 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.122s
I20260709 12:26:26.302708  4489 maintenance_manager.cc:643] P edba2778b94f4e5aa07d5f710879b375: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 10.156s	user 0.193s	sys 0.094s Metrics: {"bytes_written":29523,"cfile_init":1,"dirs.queue_time_us":387960,"dirs.run_cpu_time_us":597,"dirs.run_wall_time_us":50088,"drs_written":1,"lbm_read_time_us":8267,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3,"lbm_write_time_us":62532,"lbm_writes_10-100_ms":2,"lbm_writes_lt_1ms":28,"peak_mem_usage":0,"rows_written":63,"thread_start_us":380814,"threads_started":3,"wal-append.queue_time_us":597318}
I20260709 12:26:26.308998  9023 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.179s
I20260709 12:26:26.492164  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:26.531692  4598 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:26.538573  4750 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:26:26.230409  9026 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:26.552762  4601 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:26.554721  4750 pending_rounds.cc:85] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 7
I20260709 12:26:26.529722  8848 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:26.556456  4750 pending_rounds.cc:107] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:26:26.558216  5312 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:26.560261  4750 raft_consensus.cc:2889] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:26.560985  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.557919  4647 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.554425  4756 raft_consensus.cc:2410] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:26:26.556977  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:26.565228  4600 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:26.568161  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2364) took 1250 ms. Trace:
I20260709 12:26:26.568691  4605 rpcz_store.cc:276] 0709 12:26:25.317311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.317494 (+   183us) service_pool.cc:224] Handling call
0709 12:26:25.349752 (+ 32258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.176770 (+827018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.567836 (+391066us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b540013ecd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:26.568077 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1700376448}
I20260709 12:26:26.571664  4756 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:26.577061  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 85061.9us
I20260709 12:26:26.578153  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:26.580312  4571 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:26.581825  8968 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:26.582911  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.586113  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:26.577081  4756 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:26.576613  9037 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:26.590754  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.565379  9010 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:26.530495  9012 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:26.606650  9012 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:26.569852  8997 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:26.608100  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:26.608793  5348 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:26.608852  9024 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:26.611833  8983 consensus_meta.cc:287] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.500s	user 0.008s	sys 0.004s
I20260709 12:26:26.576243  8982 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:26.589558  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.567461  9042 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:26:26.602885  4490 maintenance_manager.cc:419] P edba2778b94f4e5aa07d5f710879b375: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9870 bytes on disk
I20260709 12:26:26.565734  9032 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:26:26.604353  9026 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:26.624342  8661 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:26.629933  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.645870  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 55228.9us
I20260709 12:26:26.651686  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.648844  4602 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:26.653952  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.620072  9010 raft_consensus.cc:697] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:26.649842  8997 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:26.650291  8661 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:26.653144  9031 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:26.708029  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 54140.7us
I20260709 12:26:26.711987  4647 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.714798  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:26.715448  4755 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:26.716639  5314 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:26.719818  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:26.700438  9024 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:26.742663  4489 maintenance_manager.cc:643] P edba2778b94f4e5aa07d5f710879b375: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.089s	user 0.002s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":471,"lbm_reads_lt_1ms":4}
I20260709 12:26:26.743103  4755 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:26.702003  9026 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:26.599537  8848 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:26.757601  5314 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:26.711459  8661 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.715308  9037 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:26.776429  4597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2352) took 1729 ms (client timeout 2067 ms). Trace:
W20260709 12:26:26.779851  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 193875us
I20260709 12:26:26.762584  4647 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:26.784178  4597 rpcz_store.cc:269] 0709 12:26:25.047562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.052782 (+  5220us) service_pool.cc:224] Handling call
0709 12:26:26.776177 (+1723395us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400302854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:26.776346 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2395259648}
I20260709 12:26:26.746546  8982 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:26.797113  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:26.755730  9026 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:26.846977  8992 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:26:26.848272  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 62602us
I20260709 12:26:26.883819  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:26.884917  4747 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:26.855710  8500 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.005s
W20260709 12:26:26.874053  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 77027.4us
I20260709 12:26:26.895710  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:26.781275  9037 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.787492  8813 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:26.794556  9024 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:26.897670  8933 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:26.802570  8848 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.879501  9026 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:26.898408  9059 log.cc:927] Time spent T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 12:26:26.909161  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 319722us
W20260709 12:26:26.930110  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.898908  8982 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:26.794165  8964 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
I20260709 12:26:26.905334  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.902665  9037 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:26.946586  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.906585  8933 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.909734  8928 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:26:26.936496  9057 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:26:26.967741  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:26.972424  4673 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.980718  4681 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:26.980829  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:26.982165  9063 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.201s	user 0.000s	sys 0.000s
I20260709 12:26:26.986681  9050 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:26.990051  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:26.990737  4674 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:26.991776  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:26.992420  4679 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:26.992936  9067 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:26:26.995709  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:26.996846  8931 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:26.999825  8879 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.003s
I20260709 12:26:27.000305  8879 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:27.001111  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 493756us
I20260709 12:26:27.001721  5346 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:27.001861  4645 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:27.003716  4645 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:27.003619  9037 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:27.004184  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 99238.1us
I20260709 12:26:27.006011  9037 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:27.004092  9022 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:27.013190  8447 raft_consensus.cc:1064] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:26:27.013556  8800 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:27.014192  9022 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:27.015385  8447 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 6, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:27.018167  9071 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.018723  9071 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:26.991776  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:27.019944  4751 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:27.020903  9071 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:27.021872  8984 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:27.029184  4647 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:26.912012  9089 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:27.031884  4673 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:27.033003  4679 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:27.035615  8983 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:26.902990  9010 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:27.052448  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2888) took 1082 ms. Trace:
W20260709 12:26:27.060431  9025 consensus_meta.cc:287] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.533s	user 0.007s	sys 0.003s
I20260709 12:26:27.062994  5342 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:27.067279  8857 raft_consensus.cc:3055] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:26:27.047289  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.961966  9072 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:26.968729  9024 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:26:27.075529  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 318062us
I20260709 12:26:27.078285  4751 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:27.072418  8879 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:26.974314  8933 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:27.081902  4754 rpcz_store.cc:276] 0709 12:26:25.971613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.061737 (+ 90124us) service_pool.cc:224] Handling call
0709 12:26:26.062066 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.062464 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:26.164642 (+102178us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.541093 (+376451us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.541679 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.541842 (+   163us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:26.570079 (+ 28237us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.705345 (+135266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.705422 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.052222 (+346800us) raft_consensus.cc:1685] finished
0709 12:26:27.052245 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.052383 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":225353,"prepare.queue_time_us":280534,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":191406,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":71048,"thread_start_us":568001,"threads_started":3,"wal-append.queue_time_us":496384}
I20260709 12:26:27.114715  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:27.114807  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:27.115933  4754 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:26.974797  8982 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:27.078701  9022 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:27.040516  9089 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.081398  8983 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.089150  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:27.085903  8857 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:27.134068  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 45008.9us
I20260709 12:26:27.135975  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:27.139874  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:27.139987  4749 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:27.073462  9026 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:27.129607  8983 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:27.129057  9089 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:27.133432  9022 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:27.159633  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 127992us
I20260709 12:26:27.198282  4675 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:27.162815  4751 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:27.127170  9025 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:26:27.141610  8879 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:27.129004  8984 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:27.146483  9079 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:27.260179  4749 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:27.353230  4751 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:26:27.355309  4752 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.010s	sys 0.015s
W20260709 12:26:27.355674  4752 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.010s	sys 0.015s
I20260709 12:26:27.365193  4644 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:27.383431  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 296703us
W20260709 12:26:27.358451  9078 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:26:27.352977  8879 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:27.364514  9058 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.007s	sys 0.018s
I20260709 12:26:27.366401  9079 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:27.453934  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 88820us
W20260709 12:26:27.470513  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.477203  9088 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.481686  9088 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:27.484426  9088 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:27.474668  8997 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:27.379245  8857 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:27.488595  8857 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:27.488336  8997 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:27.490537  8997 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:27.492745  9024 consensus_queue.cc:1175] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49277a7653524787a715232ab908bbc6" responder_term: 16 status { last_received { term: 16 index: 13 } last_committed_idx: 10 last_received_current_leader { term: 16 index: 13 } }
I20260709 12:26:27.493654  9098 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:27.494114  9098 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.494450  9098 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:27.498008  8996 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:27.506685  8661 consensus_meta.cc:287] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.709s	user 0.002s	sys 0.007s
W20260709 12:26:27.509310  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.512172  4747 consensus_meta.cc:287] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.615s	user 0.010s	sys 0.000s
I20260709 12:26:27.521355  4747 raft_consensus.cc:2410] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:26:27.498835  9026 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:27.453840  9099 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.374792  8982 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:27.512421  4645 connection.cc:256] RPC call timeout handler was delayed by -1.3741129980003279s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20670000000000002s for a response
W20260709 12:26:27.520349  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.468734  9101 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.507164  8661 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:27.378647  8928 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:27.541440  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:27.522365  9058 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.008s	sys 0.019s
I20260709 12:26:27.454150  9022 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:27.545030  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:27.546002  4673 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:27.547564  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.467839  8984 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:27.549850  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:27.551206  4679 raft_consensus.cc:2368] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9.
I20260709 12:26:27.497381  8933 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:27.553187  9094 log.cc:927] Time spent T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:26:27.553687  8933 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:27.555049  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 715) took 1081 ms. Trace:
I20260709 12:26:27.556137  9079 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:27.545938  8848 consensus_meta.cc:287] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.001s
W20260709 12:26:27.547345  9082 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:26:27.555433  4755 rpcz_store.cc:276] 0709 12:26:26.473769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.763960 (+290191us) service_pool.cc:224] Handling call
0709 12:26:26.764797 (+   837us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.080585 (+315788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:27.080635 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.086157 (+  5522us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.404821 (+318664us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:27.404876 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.404978 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.554844 (+149866us) raft_consensus.cc:1685] finished
0709 12:26:27.554865 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.554993 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22424,"raft.queue_time_us":220857,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":64,"thread_start_us":557495,"threads_started":2,"wal-append.queue_time_us":322147}
I20260709 12:26:27.540992  8693 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:26:27.570067  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:27.572170  5345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 655) took 1233 ms. Trace:
I20260709 12:26:27.512624  8879 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:27.574481  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:27.575660  4679 raft_consensus.cc:2410] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:26:27.577301  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:27.566088  9107 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:27.594133  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:27.595187  4679 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:27.544399  9101 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:27.572698  5345 rpcz_store.cc:276] 0709 12:26:26.338798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.338983 (+   185us) service_pool.cc:224] Handling call
0709 12:26:26.347699 (+  8716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.348147 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:26.564815 (+216668us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.566945 (+  2130us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.568181 (+  1236us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.056878 (+488697us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b180039d240. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:26:27.074139 (+ 17261us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b180039d240. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:26:27.471685 (+397546us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b180039d240. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c6455 00007effee4ae060 00007effee4c0716 00007effee4d1287 00007effee4d1208
0709 12:26:27.571773 (+100088us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:27.571821 (+    48us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:27.571870 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.571930 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.572099 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6940,"prepare.queue_time_us":324944,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":62,"raft.queue_time_us":6081,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":224,"spinlock_wait_cycles":98306944,"thread_start_us":304278,"threads_started":2,"wal-append.queue_time_us":9733}
I20260709 12:26:27.561015  8872 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:26:27.538563  9099 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:27.604547  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2908) took 1481 ms. Trace:
I20260709 12:26:27.623426  4756 rpcz_store.cc:276] 0709 12:26:26.123025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.614322 (+491297us) service_pool.cc:224] Handling call
0709 12:26:26.614908 (+   586us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.709813 (+ 94905us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:26.709857 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.717466 (+  7609us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.111783 (+394317us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:27.111837 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.113075 (+  1238us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.604335 (+491260us) raft_consensus.cc:1685] finished
0709 12:26:27.604353 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.604483 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55122,"raft.queue_time_us":272425,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":15278,"thread_start_us":403924,"threads_started":2,"wal-append.queue_time_us":632191}
I20260709 12:26:27.560923  8928 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:27.601689  9107 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:27.620108  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:27.629851  4674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:26:27.630640  4674 raft_consensus.cc:606] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:27.630973  4674 raft_consensus.cc:614] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:27.631922  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:27.632833  4681 raft_consensus.cc:2410] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:27.635697  9107 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:27.636983  4676 raft_consensus.cc:2393] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:27.637264  8831 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:27.630093  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:27.639110  4673 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:27.639022  9099 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:27.640111  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:27.640872  5345 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:27.642426  4644 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:27.645143  5343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 657) took 1136 ms. Trace:
I20260709 12:26:27.645539  5343 rpcz_store.cc:276] 0709 12:26:26.508926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.509211 (+   285us) service_pool.cc:224] Handling call
0709 12:26:26.509651 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.518821 (+  9170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.644800 (+1125979us) spinlock_profiling.cc:243] Waited 8.43 ms on lock 0x7b5400618354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:27.645066 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19393792}
W20260709 12:26:27.666268  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 129274us
I20260709 12:26:27.651643  9099 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:27.731511  4605 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:27.723711  9099 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.759943  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:27.885183  5343 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:26:27.727118  8996 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.141s
I20260709 12:26:27.675873  9098 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:27.727870  8983 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:27.832515  9099 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:27.988632  5345 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:27.867277  9079 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:28.020058  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.039016  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.873006  8984 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.983523  9111 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.920511  8933 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:28.078827  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 459467us
I20260709 12:26:28.246905  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.248023  4605 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:26:28.365415  9119 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 12:26:28.047297  9109 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.257s
W20260709 12:26:28.302350  9116 log.cc:927] Time spent T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.540s	user 0.000s	sys 0.001s
I20260709 12:26:28.517509  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:28.493083  5311 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.011s	sys 0.012s
I20260709 12:26:28.044543  8982 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.260s
I20260709 12:26:28.047763  9079 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:28.511566  9089 consensus_meta.cc:287] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 1.258s	user 0.013s	sys 0.000s
I20260709 12:26:28.201732  8933 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:28.386935  8928 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.193629  8984 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:28.561216  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.197278  9111 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:28.569892  4605 raft_consensus.cc:2393] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:26:28.531558  5311 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.012s	sys 0.012s
I20260709 12:26:28.586114  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:28.587033  4676 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:28.568092  8857 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:28.588790  4673 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:28.588780  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.594540  4605 raft_consensus.cc:2410] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:26:28.597024  9127 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.559825  9089 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:28.599421  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:28.565789  8928 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:28.588135  9042 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.619879  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.624434  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:28.632781  4676 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:28.603413  8871 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:28.637727  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 220800us
I20260709 12:26:28.640661  9109 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:28.641040  4674 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:26:28.640726  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:28.641410  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.641426  9109 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:28.600696  8857 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:28.638242  9115 log.cc:927] Time spent T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.639s	user 0.000s	sys 0.002s
W20260709 12:26:28.661125  9118 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:26:28.661401  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.662235  4600 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 5.
I20260709 12:26:28.645375  9110 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:28.631534  9042 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:28.651944  9098 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:28.619211  9127 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:28.668777  8933 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:28.657063  9129 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:28.687813  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 45770.8us
I20260709 12:26:28.649106  8871 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:28.684893  9111 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:28.677977  9128 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:28.704561  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 660) took 2228 ms. Trace:
I20260709 12:26:28.692004  8928 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:28.694255  8984 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:28.697894  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.703996  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.718371  4596 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:28.705137  4604 rpcz_store.cc:276] 0709 12:26:26.475915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.476064 (+   149us) service_pool.cc:224] Handling call
0709 12:26:26.476805 (+   741us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.477176 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:26.480950 (+  3774us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.636391 (+1155441us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b18002e4fe0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:26:28.245040 (+608649us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:28.301411 (+ 56371us) log.cc:844] Serialized 26 byte log entry
0709 12:26:28.301552 (+   141us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:28.301609 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.301678 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.696947 (+395269us) raft_consensus.cc:1685] finished
0709 12:26:28.704297 (+  7350us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.704477 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":146019,"prepare.queue_time_us":4088,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":159270,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":69568,"spinlock_wait_cycles":3829376,"thread_start_us":462246,"threads_started":1,"wal-append.queue_time_us":2116362}
I20260709 12:26:28.720844  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:28.703374  9042 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:28.723172  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.728421  4596 raft_consensus.cc:2410] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:26:28.732296  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.733458  9091 raft_consensus.cc:3055] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:28.733601  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.734026  9091 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:28.734862  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.734915  9091 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:28.736186  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.736363  9091 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:28.736999  4596 raft_consensus.cc:2393] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:28.708400  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.740521  4601 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:28.674484  9110 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:28.713210  9127 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:28.724052  8984 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.716879  9109 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:28.688489  9129 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:28.764431  8984 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:28.737609  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.739706  8871 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:28.740574  8857 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:28.708804  4605 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 11. Ops: []
I20260709 12:26:28.745322  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:28.770229  4601 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 8: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:28.770967  9110 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:28.772599  9129 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:28.774974  8984 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:28.745441  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:28.688655  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:28.749030  4604 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:26:28.709647  9128 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:29.078970  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 390382us
I20260709 12:26:29.079550  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.080425  9128 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:28.779457  9099 consensus_meta.cc:287] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.734s	user 0.010s	sys 0.000s
I20260709 12:26:28.787923  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.080571  8838 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:29.080683  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.081543  4604 raft_consensus.cc:2393] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:29.081755  8872 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:29.082139  4596 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:26:29.082496  4604 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:29.083681  4596 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:29.081059  9099 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.087124  9099 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:28.790464  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:28.806773  4569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.937s	user 0.016s	sys 0.030s
W20260709 12:26:29.091986  4569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.223s	user 0.017s	sys 0.030s
W20260709 12:26:29.092365  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 2.22314e+06us
I20260709 12:26:29.093977  8601 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:29.095728  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.096614  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1887) took 1341 ms. Trace:
I20260709 12:26:29.096949  4597 rpcz_store.cc:276] 0709 12:26:27.755095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.095442 (+1340347us) service_pool.cc:224] Handling call
0709 12:26:29.096539 (+  1097us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:28.813022  9117 log.cc:927] Time spent T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.198s	user 0.002s	sys 0.000s
I20260709 12:26:28.818843  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.817456  8997 raft_consensus.cc:3055] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:28.715840  9089 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:28.873456  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:28.902132  9104 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:29.330087  9104 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:29.331017  9104 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.332113  9104 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:29.333206  9079 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:28.908957  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:28.954846  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:29.338279  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.972952  9136 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.976589  8661 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:29.342018  8661 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.341830  9136 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.342500  8661 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.344147  9136 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:29.344516  8661 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:29.344681  9136 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.980608  9140 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:29.345686  9140 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.347609  9010 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:29.348052  9140 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:28.981122  9107 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:29.348838  9107 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.351120  9107 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:28.981899  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.351539  8983 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:29.351902  4678 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:29.353029  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.353760  4678 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:29.354576  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:29.355248  4678 raft_consensus.cc:2368] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14.
I20260709 12:26:29.355945  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:29.357086  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:28.986397  5342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 579) took 3622 ms (client timeout 2632 ms). Trace:
W20260709 12:26:29.358599  5342 rpcz_store.cc:269] 0709 12:26:25.364343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.364529 (+   186us) service_pool.cc:224] Handling call
0709 12:26:28.986117 (+3621588us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54004fb0d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:28.986268 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3840795392}
I20260709 12:26:29.359953  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.360576  5342 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:28.992771  4674 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:29.362497  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.363145  4674 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:29.363955  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.364705  4674 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:28.993458  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:29.007315  9124 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.640s	user 0.002s	sys 0.000s
I20260709 12:26:29.025918  9088 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:29.027966  5343 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:29.367089  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.367610  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:29.368225  5343 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:29.368302  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.369165  4672 raft_consensus.cc:2393] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:29.031002  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.037153  8984 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:29.370445  8984 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.372725  8984 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:29.039307  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.373893  8447 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.374482  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.374431  8447 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:29.376554  4571 connection.cc:256] RPC call timeout handler was delayed by -2.0076368250001906s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1754s for a response
I20260709 12:26:29.376822  8447 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:29.376948  4571 connection.cc:256] RPC call timeout handler was delayed by -1.6767488680002316s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2239s for a response
I20260709 12:26:29.047227  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.378063  4680 raft_consensus.cc:2393] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:29.378564  9036 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.378979  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:29.379122  9036 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.379630  4680 raft_consensus.cc:2376] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10. Re-sending same reply.
I20260709 12:26:29.380542  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.381099  4680 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:29.381714  8500 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.382253  8500 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:29.382918  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.384474  8500 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:29.384534  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.384965  8789 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.385134  8500 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.385704  8992 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.385618  8789 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.386270  8992 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.387895  8789 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:29.050354  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.388424  8789 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.388928  8789 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.068030  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.418542  5345 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:29.419416  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.071513  5347 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:29.420533  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.420724  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.422062  8789 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:29.102943  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:29.424662  8500 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.109680  8601 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:29.426342  8447 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:29.125665  8881 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.427441  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.427839  8881 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.425318  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.430469  8881 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:29.128518  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:29.431202  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.431380  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.432543  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.128975  9146 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:29.427778  8789 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:26:29.433897  9146 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.436424  9146 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:29.129113  9040 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:29.437870  9040 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.440122  9040 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:29.440614  9040 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:29.130856  9074 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.441883  9074 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.444380  9074 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:29.131035  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2897) took 3011 ms. Trace:
I20260709 12:26:29.444984  4753 rpcz_store.cc:276] 0709 12:26:26.119971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.294212 (+174241us) service_pool.cc:224] Handling call
0709 12:26:26.295100 (+   888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.295535 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:26.521828 (+226293us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b1800112e40. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007effeb3cba40 00007effeb3c64e4 00007effeb3c6455 00007efff1eae3c7 00007effee4c09a7 00007effee4d00bf 00007effee4d0038
0709 12:26:26.700334 (+178506us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.123675 (+423341us) log.cc:844] Serialized 14 byte log entry
0709 12:26:28.802372 (+1678697us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b1800112e40. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007effeb3cba40 00007effeb3c64e4 00007effeb3c6455 00007effee4ae060 00007effee4c0716 00007effee4d1287 00007effee4d1208
0709 12:26:28.832603 (+ 30231us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.125742 (+293139us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.126362 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.126564 (+   202us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:29.126848 (+   284us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.130702 (+  3854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.130777 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.130812 (+    35us) raft_consensus.cc:1685] finished
0709 12:26:29.130827 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.130955 (+   128us) inbound_call.cc:177] Queueing success response
0709 12:26:29.177543 (+ 46588us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b1800112e40. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
Metrics: {"mutex_wait_us":297777,"prepare.queue_time_us":308835,"prepare.run_cpu_time_us":142,"prepare.run_wall_time_us":146,"raft.queue_time_us":1983492,"raft.run_cpu_time_us":464,"raft.run_wall_time_us":19915,"spinlock_wait_cycles":91704320,"thread_start_us":592791,"threads_started":5,"wal-append.queue_time_us":720728,"wal-append.run_cpu_time_us":2508,"wal-append.run_wall_time_us":1159815}
I20260709 12:26:29.447921  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:29.448596  4753 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:29.132798  9155 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:29.450044  9155 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.451494  4676 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:26:29.452512  9155 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:29.452771  4676 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:29.133898  9030 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:29.453354  9030 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.134402  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 718) took 2659 ms. Trace:
I20260709 12:26:29.454742  4748 rpcz_store.cc:276] 0709 12:26:26.474724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.765663 (+290939us) service_pool.cc:224] Handling call
0709 12:26:26.766029 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.766291 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:26.790071 (+ 23780us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.133697 (+2343626us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:29.133746 (+    49us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:29.133790 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.134238 (+   448us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.134351 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1980634,"prepare.queue_time_us":24108,"prepare.run_cpu_time_us":12554,"prepare.run_wall_time_us":42281,"raft.queue_time_us":1996005,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":362520,"threads_started":2,"wal-append.queue_time_us":689023,"wal-append.run_cpu_time_us":1299,"wal-append.run_wall_time_us":1039507}
I20260709 12:26:29.456959  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:29.135036  9149 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.457922  9149 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.460287  9149 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:29.130818  9145 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.460939  9145 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.461699  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.463068  9145 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:29.151260  9151 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.153506  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2900) took 3032 ms. Trace:
I20260709 12:26:29.152245  9153 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:29.464202  9153 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.462306  4596 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:29.463804  4752 rpcz_store.cc:276] 0709 12:26:26.120833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.529790 (+408957us) service_pool.cc:224] Handling call
0709 12:26:26.530145 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.553234 (+ 23089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.553280 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.562927 (+  9647us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.356030 (+793103us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:27.403304 (+ 47274us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.152867 (+1749563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.152962 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.153245 (+   283us) raft_consensus.cc:1685] finished
0709 12:26:29.153259 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.153430 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1733467,"prepare.queue_time_us":53414,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":1901299,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":8835,"thread_start_us":793795,"threads_started":2,"wal-append.queue_time_us":928607,"wal-append.run_cpu_time_us":2030,"wal-append.run_wall_time_us":1008646}
I20260709 12:26:29.466465  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:29.467005  4752 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:29.468057  9153 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:29.160692  9128 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.155112  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:29.468752  9128 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.163300  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:29.172529  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.161185  8982 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.469549  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 314479us
I20260709 12:26:29.471029  4720 leader_election.cc:400] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9.
I20260709 12:26:29.471197  8500 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:29.181489  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:29.472012  5344 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:29.182554  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.225946  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 726) took 2117 ms. Trace:
I20260709 12:26:29.105206  4569 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:29.571282  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 466139us
I20260709 12:26:29.572271  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:29.572944  8789 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:29.573719  8789 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:29.574193  8789 raft_consensus.cc:697] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:29.575027  8789 consensus_queue.cc:237] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.233263  9101 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:29.327198  8997 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:29.578419  8997 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.579790  8997 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:28.980146  9121 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.382s	user 0.002s	sys 0.000s
W20260709 12:26:28.983147  9125 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.360s	user 0.001s	sys 0.000s
I20260709 12:26:29.390365  9079 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.583492  9079 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.369663  9088 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.388434  8992 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:29.399660  9031 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:29.586129  5345 raft_consensus.cc:2410] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:26:29.361526  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:29.587796  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.588873  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.373236  8984 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:26:29.405267  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:29.381477  9036 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:29.345155  9136 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:29.590449  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 185237us
I20260709 12:26:29.590698  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.591459  5341 raft_consensus.cc:2393] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:29.591935  9089 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:29.592446  9089 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:29.104243  9091 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:29.107869  9126 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.594019  9036 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:29.594242  9126 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.175392  9150 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.239529  9069 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:29.595669  9150 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.595767  9069 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.596496  9126 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:29.428323  4600 raft_consensus.cc:2410] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:26:29.700232  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.705389  9069 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:29.329818  4599 raft_consensus.cc:2393] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:29.712182  8998 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.712304  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1901) took 1008 ms. Trace:
I20260709 12:26:29.712685  4599 rpcz_store.cc:276] 0709 12:26:28.704044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.873094 (+169050us) service_pool.cc:224] Handling call
0709 12:26:29.712233 (+839139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:29.712738  8998 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:29.555465  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 157892us
I20260709 12:26:29.397809  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.223431  4598 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:29.831001  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1899) took 1135 ms. Trace:
I20260709 12:26:29.422785  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.833801  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.834673  4604 raft_consensus.cc:2410] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:26:29.835368  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.836301  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.837227  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.390173  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.872215  4603 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:29.393517  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.875524  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.876725  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.463763  9151 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.879563  9151 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:29.396905  4644 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:29.882519  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 485669us
I20260709 12:26:29.296581  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.403221  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.885777  8928 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:29.886400  4679 raft_consensus.cc:2410] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:29.887490  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.888309  4679 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:29.251243  4681 raft_consensus.cc:2393] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:29.417819  4678 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:29.889249  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.892673  4679 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:29.893626  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.894574  4679 raft_consensus.cc:2410] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:29.895567  4679 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:29.885521  4673 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:29.389427  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.401364  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.899507  4672 raft_consensus.cc:2410] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:29.402299  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:29.403117  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:29.907337  4680 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:26:29.471379  4720 leader_election.cc:403] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:29.910859  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 439928us
W20260709 12:26:29.511636  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.483481  4748 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:29.472800  4749 rpcz_store.cc:276] 0709 12:26:27.108348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.329006 (+220658us) service_pool.cc:224] Handling call
0709 12:26:27.374136 (+ 45130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:29.173706 (+1799570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.173749 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.174300 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.225508 (+ 51208us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:29.225555 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.225662 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.225708 (+    46us) raft_consensus.cc:1685] finished
0709 12:26:29.225720 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.225867 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1742904,"raft.queue_time_us":1755821,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"thread_start_us":88844,"threads_started":2,"wal-append.queue_time_us":45248}
I20260709 12:26:29.392096  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2901) took 3271 ms. Trace:
I20260709 12:26:29.481509  4752 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:29.266059  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:29.927656  4751 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:29.922348  4722 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42632 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2406)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2365)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2401)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2368)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2383)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2402)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2399)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2376)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2403)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2388)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2370)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2371)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2404)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2405)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2369)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2360)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2372)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2363)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2373)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2366)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2394)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2392)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2398)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2389)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2379)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2395)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 733)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 737)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 738)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2374)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2377)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2387)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2396)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2915)
I20260709 12:26:29.928634  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:29.498348  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:29.930357  4753 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:29.470096  4755 raft_consensus.cc:2410] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:26:29.308009  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:29.929656  4722 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:42632 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2365)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2401)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2368)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2383)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2402)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2399)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2376)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2403)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2388)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2370)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2371)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2404)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2405)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2369)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2360)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2372)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2363)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2373)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2366)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2394)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2392)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2398)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2389)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2379)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2395)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 733)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 737)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 738)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2374)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2377)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2387)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2396)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2915)
I20260709 12:26:29.306890  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:29.934463  4754 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:29.550696  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.938144  4596 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:29.939211  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.939818  4596 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:29.940629  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.941185  4596 raft_consensus.cc:2376] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10. Re-sending same reply.
W20260709 12:26:29.505044  4719 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43864 dropped due to backpressure: service queue is full with 50 items
I20260709 12:26:29.596519  9136 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:29.597805  9150 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:29.394047  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.619225  4570 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:53448 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2407)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2408)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 683)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2409)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2417)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2387)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 666)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 680)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2383)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2410)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 678)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 679)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2389)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2411)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2414)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2384)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2404)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2412)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2395)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 676)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 677)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2385)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2413)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 674)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2418)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2419)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 675)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2403)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2402)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2394)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 673)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2416)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2396)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2386)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2388)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2392)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2393)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2398)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2401)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2405)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2406)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 667)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 668)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 670)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 671)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 672)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 681)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 682)
I20260709 12:26:29.947818  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.948956  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.950001  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:29.950781  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.950984  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:29.664074  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:29.678406  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 252025us
W20260709 12:26:29.689464  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.499708  5344 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:29.956141  5344 raft_consensus.cc:606] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:29.956482  5344 raft_consensus.cc:614] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:29.956852  4645 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:29.552196  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.958923  5347 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:29.958834  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.506544  5340 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:29.961813  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:29.417593  5343 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:29.963713  9022 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:29.592171  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.494845  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:29.742403  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.470647  8982 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.968770  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.969472  8982 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:29.605249  8992 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.970863  8992 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.973052  8992 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:29.641773  9030 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:29.536999  8976 raft_consensus.cc:3055] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:29.976349  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:29.976807  8976 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:29.594621  9036 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:29.978345  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:29.979264  9150 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:29.979099  9036 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.979753  9150 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:29.980226  9036 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:29.441124  9040 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.599448  9079 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:29.746039  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.985293  4599 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:29.986268  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.987252  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.987752  4599 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:29.980247  9150 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:29.701380  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.990163  4600 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:29.990963  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:29.991412  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 96061.8us
I20260709 12:26:29.991882  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.992905  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:29.603904  9088 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:29.593304  9089 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.589989  9101 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:29.999749  9101 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:30.000202  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.776201  8998 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:29.777657  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:30.006166  9088 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:30.006326  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 228793us
I20260709 12:26:30.006588  9088 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:30.007016  9088 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:30.007434  5314 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:30.008981  5314 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:29.814214  5312 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.193:45077 timed out after 1.554s (SENT) [suppressed 39 similar messages]
W20260709 12:26:30.010639  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 196456us
I20260709 12:26:30.012336  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:29.828771  4602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1891) took 1262 ms (client timeout 1610 ms). Trace:
W20260709 12:26:30.013036  4602 rpcz_store.cc:269] 0709 12:26:28.566322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.397542 (+831220us) service_pool.cc:224] Handling call
0709 12:26:29.828700 (+431158us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:29.831619  4626 debug-util.cc:398] Leaking SignalData structure 0x7b0800106a80 after lost signal to thread 3434
I20260709 12:26:29.837656  9031 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.014154  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:30.014256  9031 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:30.015239  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:30.015851  4602 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:30.016664  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:30.017279  9031 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:29.474784  9153 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:29.831339  4598 rpcz_store.cc:276] 0709 12:26:28.696045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.222848 (+526803us) service_pool.cc:224] Handling call
0709 12:26:29.223222 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.830739 (+607517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.830941 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:29.839195  4604 raft_consensus.cc:2410] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:26:30.019901  8813 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:30.020335  8813 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:30.020701  8813 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:29.059347  4675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48198 (request call id 657) took 2094 ms (client timeout 2001 ms). Trace:
W20260709 12:26:29.902660  4681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48198 (request call id 667) took 1322 ms (client timeout 1690 ms). Trace:
W20260709 12:26:30.022747  4675 rpcz_store.cc:269] 0709 12:26:26.964553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.964741 (+   188us) service_pool.cc:224] Handling call
0709 12:26:29.059098 (+2094357us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54002dee54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:29.059230 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302146432}
W20260709 12:26:30.022859  4681 rpcz_store.cc:269] 0709 12:26:28.582491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.582713 (+   222us) service_pool.cc:224] Handling call
0709 12:26:29.902461 (+1319748us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b5400085554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:29.902602 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1489536256}
I20260709 12:26:29.904682  4678 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:29.917438  4676 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 13. Ops: []
I20260709 12:26:30.026533  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 1920) took 1009 ms. Trace:
I20260709 12:26:30.026867  4676 rpcz_store.cc:276] 0709 12:26:29.023672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.651707 (+628035us) service_pool.cc:224] Handling call
0709 12:26:29.917174 (+265467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.025620 (+108446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.026366 (+   746us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b5400085550. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:30.026479 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":609594112}
I20260709 12:26:29.873343  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.919942  9069 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:29.921492  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.028481  4603 raft_consensus.cc:2410] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:26:29.944146  4719 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43864 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2365)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2401)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2368)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2383)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2402)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2399)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2376)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2403)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2388)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2370)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2371)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2404)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2405)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2369)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2360)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2372)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2363)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2373)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2366)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2394)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2392)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2398)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2389)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2379)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2395)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 733)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 737)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 738)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2374)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2377)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2387)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2396)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2915)
I20260709 12:26:29.955282  8500 consensus_queue.cc:1059] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:30.034916  8500 raft_consensus.cc:3055] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:30.035255  8500 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:30.033039  4719 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43864 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2401)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2381)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2368)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2383)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2402)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2399)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2376)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2403)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2388)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2386)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2370)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2371)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2404)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2405)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2369)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2378)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2400)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2360)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2397)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2372)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2363)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 740)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2373)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2361)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2366)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2394)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2392)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2393)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 741)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2398)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2389)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2379)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2395)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 733)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 737)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 738)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2374)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2375)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2377)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2380)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2382)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2384)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2385)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2387)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2396)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2915)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2917)
I20260709 12:26:30.036051  8500 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.395215  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:30.038300  4647 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:30.039593  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:30.039987  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.040719  4752 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:30.040864  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:29.954133  8789 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:29.958099  9136 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:30.041944  9136 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.042330  9136 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:29.172771  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:30.044626  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.045686  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.045766  8982 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:30.046216  8982 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:30.046263  4756 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:30.046696  8982 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:30.048014  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.048646  4754 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:29.923975  4750 rpcz_store.cc:276] 0709 12:26:26.121070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.570643 (+449573us) service_pool.cc:224] Handling call
0709 12:26:26.572285 (+  1642us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.702426 (+130141us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:26.717817 (+ 15391us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.642945 (+925128us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b180020b840. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:26:29.006592 (+1363647us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:29.150954 (+144362us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.151491 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.151896 (+   405us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:29.151936 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.155593 (+  3657us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.391875 (+236282us) raft_consensus.cc:1685] finished
0709 12:26:29.391898 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.392032 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1024340,"prepare.queue_time_us":36828,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":1247399,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":84318,"spinlock_wait_cycles":13102848,"thread_start_us":612314,"threads_started":3,"wal-append.queue_time_us":1603151}
W20260709 12:26:29.964112  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.929278  4751 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:29.966331  5346 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:29.965446  5341 raft_consensus.cc:2393] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:29.966383  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:29.967103  5342 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 13. Ops: [11.10-11.10]
I20260709 12:26:29.967779  4597 raft_consensus.cc:2410] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:26:30.037259  8500 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:29.969776  4596 raft_consensus.cc:2376] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12. Re-sending same reply.
I20260709 12:26:29.977463  9030 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:30.041471  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:29.982811  4720 leader_election.cc:400] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14.
I20260709 12:26:29.984198  9040 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:29.993835  8976 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:30.000705  9101 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:29.481374  9128 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:30.004103  9150 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:30.068977  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:30.080426  8813 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:30.081233  4602 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
W20260709 12:26:30.014746  4626 debug-util.cc:398] Leaking SignalData structure 0x7b08002a2a80 after lost signal to thread 4536
W20260709 12:26:30.082556  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.886299  8928 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.028416  4748 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:30.089763  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.090339  4748 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:30.063087  9088 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:29.964588  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:30.049311  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.093502  4753 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:30.094421  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.100553  4748 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:30.050810  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.101317  4749 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:30.071585  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.051295  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.054517  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:30.058046  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 49141.8us
I20260709 12:26:30.103293  5314 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:30.050066  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 96131.8us
I20260709 12:26:30.062528  4752 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:30.059430  4599 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:30.109683  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.110453  4748 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.110654  4599 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:30.112819  4599 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:26:30.062813  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.114195  4755 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:30.068063  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:30.068236  8976 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.051075  9079 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:30.075939  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:30.122464  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 157936us
I20260709 12:26:30.122922  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:30.076460  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 84053.3us
I20260709 12:26:30.123634  4755 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:30.124166  4644 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:30.124555  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.068857  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:30.126888  4603 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:30.128314  4603 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:26:30.128695  4603 pending_rounds.cc:85] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 4
I20260709 12:26:30.128923  4603 pending_rounds.cc:107] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:26:30.129154  4603 raft_consensus.cc:2889] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:30.074949  9101 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:30.131639  4748 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
W20260709 12:26:30.083103  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.132315  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.132596  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.070026  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:30.125200  4755 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:30.070583  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:30.084131  4600 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:30.133279  4597 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
W20260709 12:26:30.083112  4626 debug-util.cc:398] Leaking SignalData structure 0x7b0800308720 after lost signal to thread 4630
I20260709 12:26:30.066238  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:30.134693  4596 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:30.067775  4720 leader_election.cc:403] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:26:30.049537  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.136162  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.136693  4754 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:30.139211  4748 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.105515  5314 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:30.120512  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.107520  4571 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:30.121356  4753 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:30.127314  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:30.125092  4645 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:30.130810  4605 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:26:30.132741  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.145212  4719 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:30.145735  4754 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:30.146628  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.147117  4754 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.069433  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:30.055509  4568 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:30.147840  4601 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:30.148180  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 92749.7us
W20260709 12:26:30.150069  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.135318  4626 debug-util.cc:398] Leaking SignalData structure 0x7b08001ae3e0 after lost signal to thread 4706
I20260709 12:26:30.054641  9010 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:30.151149  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.151710  4751 raft_consensus.cc:2376] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10. Re-sending same reply.
I20260709 12:26:30.062161  9030 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:30.151969  4626 debug-util.cc:398] Leaking SignalData structure 0x7b080005f5c0 after lost signal to thread 4780
I20260709 12:26:30.150841  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.153055  4750 raft_consensus.cc:2410] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:30.154335  4756 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 13: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:30.155217  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.155869  4756 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:26:30.155895  4626 debug-util.cc:398] Leaking SignalData structure 0x7b080037dce0 after lost signal to thread 5373
W20260709 12:26:30.157107  4626 debug-util.cc:398] Leaking SignalData structure 0x7b0800194940 after lost signal to thread 9011
I20260709 12:26:30.158670  4748 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
W20260709 12:26:30.163050  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.166814  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.167634  4748 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:30.092782  8928 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:30.169138  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.174222  4748 raft_consensus.cc:2410] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:30.174767  4676 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:30.175016  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.169533  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:30.170634  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 63268.5us
I20260709 12:26:30.171590  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.176875  4756 raft_consensus.cc:2376] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12. Re-sending same reply.
I20260709 12:26:30.178301  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.174108  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:30.180269  4752 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:30.181355  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.182089  4752 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:30.103348  9040 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.186713  5346 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:30.180222  5342 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:26:30.179083  4756 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:30.189203  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.190071  4756 raft_consensus.cc:2410] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 15 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:26:30.200699  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.200899  4749 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:30.201103  4681 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:30.200752  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.202138  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.202867  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.203260  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.203902  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:30.202503  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 130521us
I20260709 12:26:30.206769  4754 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:30.132994  9126 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:30.251159  9126 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:30.130514  9079 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:30.067530  9136 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:30.260833  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:30.265866  4681 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:30.202472  4571 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:30.057305  9012 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.112s
I20260709 12:26:30.150894  9128 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:30.133074  8984 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.213481  4676 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:30.251616  9126 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:30.224903  4599 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:26:30.226699  4749 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:30.296129  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.296810  4749 raft_consensus.cc:2376] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12. Re-sending same reply.
I20260709 12:26:30.132889  8789 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
W20260709 12:26:30.298820  4496 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52808 (request call id 276) took 31919 ms (client timeout 15000 ms). Trace:
I20260709 12:26:30.232997  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.236207  4604 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:26:30.237910  4603 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:26:30.168642  9030 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:30.301044  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:30.174177  9153 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:30.069233  8982 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:30.240005  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.240554  4747 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:26:30.242821  5312 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:30.253490  4678 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:30.175410  9091 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:30.210332  9040 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:30.260164  4675 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:30.261140  4602 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:26:30.268847  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 66455.1us
I20260709 12:26:30.324683  4571 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:30.187606  9169 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:30.232544  9089 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:30.209671  9150 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:30.286513  4601 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:30.212116  4756 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 12:26:30.175523  9151 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 12:26:30.334975  4571 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:30.298465  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.344211  4749 raft_consensus.cc:2376] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12. Re-sending same reply.
I20260709 12:26:30.346292  4749 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:30.347047  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 104299us
I20260709 12:26:30.347409  4749 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:26:30.348084  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:30.348649  4749 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 12:26:30.268065  9161 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:30.228655  4597 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:26:30.351850  4597 pending_rounds.cc:85] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 8
I20260709 12:26:30.352195  4597 pending_rounds.cc:107] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:26:30.352480  4597 raft_consensus.cc:2889] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:30.268378  9079 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:30.299985  4604 pending_rounds.cc:85] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 5
W20260709 12:26:30.311861  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 186831us
I20260709 12:26:30.357797  4604 pending_rounds.cc:107] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:26:30.358187  4604 raft_consensus.cc:2889] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:30.359493  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.360802  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.255774  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.361474  4754 raft_consensus.cc:2376] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11. Re-sending same reply.
I20260709 12:26:30.362461  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:30.363034  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.364207  4754 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:30.315937  4752 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:30.287938  8447 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:30.366864  8447 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:30.367914  8447 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:30.370545  9151 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14.
I20260709 12:26:30.371410  9151 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:30.371789  9151 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:30.372792  9022 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:30.335315  8997 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:30.373906  8997 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:30.375042  8997 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:30.349171  8992 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:30.313742  4747 pending_rounds.cc:85] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 7
I20260709 12:26:30.359413  4645 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:30.377835  4747 pending_rounds.cc:107] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:26:30.378114  4747 raft_consensus.cc:2889] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:30.342875  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.378855  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 733) took 1150 ms. Trace:
I20260709 12:26:30.379164  4754 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:30.379189  4747 rpcz_store.cc:276] 0709 12:26:29.228416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.215442 (+987026us) service_pool.cc:224] Handling call
0709 12:26:30.215723 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.378622 (+162899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.378804 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.372699  9151 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:30.382354  9151 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:30.322954  9010 raft_consensus.cc:697] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:30.383833  4749 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:30.361465  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:30.386219  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.386965  4751 raft_consensus.cc:2376] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11. Re-sending same reply.
I20260709 12:26:30.313580  9153 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.302695  9088 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:30.397711  4498 catalog_manager.cc:5697] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:30.398617  9010 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:30.399019  8982 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:30.399089  9173 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:26:30.299818  8984 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:30.314039  9091 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:30.407630  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.399883  4750 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
W20260709 12:26:30.415946  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.410216  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.417438  4747 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:30.384231  9128 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:30.384986  8983 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:30.409807  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 50469.2us
I20260709 12:26:30.421947  4647 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:30.395807  9169 raft_consensus.cc:697] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:30.405035  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.406451  9153 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.404949  9022 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:30.426383  9091 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.428725  4678 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:30.432678  4645 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:30.432226  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:30.435299  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.450698  4498 catalog_manager.cc:5697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 7 to 8, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 8 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:30.440770  4598 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
W20260709 12:26:30.299329  4496 rpcz_store.cc:269] 0709 12:25:58.379290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.379651 (+   361us) service_pool.cc:224] Handling call
0709 12:26:30.298742 (+31919091us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:26.051455 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:26.052207 (+   752us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:26.052254 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:26:26.052270 (+    16us) tablet.cc:662] Decoding operations
0709 12:26:26.053259 (+   989us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:26.053343 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:26.053373 (+    30us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:26:26.138293 (+ 84920us) tablet.cc:701] Row locks acquired
0709 12:26:26.138359 (+    66us) write_op.cc:260] PREPARE: finished
0709 12:26:26.138611 (+   252us) write_op.cc:270] Start()
0709 12:26:26.138789 (+   178us) write_op.cc:276] Timestamp: P: 1783599986138506 usec, L: 0
0709 12:26:26.138830 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:26:26.143917 (+  5087us) log.cc:844] Serialized 2901 byte log entry
0709 12:26:26.628197 (+484280us) op_driver.cc:464] REPLICATION: finished
0709 12:26:27.504118 (+875921us) write_op.cc:301] APPLY: starting
0709 12:26:27.504227 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:27.618669 (+114442us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:27.618789 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:28.692589 (+1073800us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:28.692935 (+   346us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 12:26:28.692994 (+    59us) write_op.cc:312] APPLY: finished
0709 12:26:28.693385 (+   391us) log.cc:844] Serialized 73 byte log entry
0709 12:26:28.757870 (+ 64485us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:28.758027 (+   157us) write_op.cc:454] Released schema lock
0709 12:26:28.758115 (+    88us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:28.816246 (+ 58131us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:28.816298 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8279887,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":875662,"apply.run_cpu_time_us":28745,"apply.run_wall_time_us":1280919,"cfile_cache_hit":26,"cfile_cache_hit_bytes":27131,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6214,"cfile_init":1,"lbm_read_time_us":13311,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":7,"num_ops":7,"prepare.queue_time_us":254546,"prepare.run_cpu_time_us":5629,"prepare.run_wall_time_us":92800,"replication_time_us":489194,"thread_start_us":1022320,"threads_started":3,"wal-append.queue_time_us":62852,"wal-append.run_cpu_time_us":1304,"wal-append.run_wall_time_us":1001307}]]}
I20260709 12:26:30.432240  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.452677  4675 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:30.479398  4747 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:30.427753  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 445273us
I20260709 12:26:30.464031  4749 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 13: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:26:30.473106  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 111746us
W20260709 12:26:30.475085  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.433445  8982 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:30.460657  9069 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9.
W20260709 12:26:30.500047  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 67450.1us
W20260709 12:26:30.492424  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 60278.9us
I20260709 12:26:30.514436  4647 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:30.515931  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:30.495397  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 147424us
I20260709 12:26:30.517789  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 738) took 1265 ms. Trace:
I20260709 12:26:30.521066  4753 rpcz_store.cc:276] 0709 12:26:29.252690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.215864 (+963174us) service_pool.cc:224] Handling call
0709 12:26:30.262456 (+ 46592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.262708 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:30.416010 (+153302us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.517430 (+101420us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:30.517488 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:30.517543 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.517623 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.517728 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":228458,"raft.queue_time_us":523,"thread_start_us":226896,"threads_started":2}
I20260709 12:26:30.518500  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:30.503057  9176 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:30.520153  4647 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:30.528610  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:30.494037  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:30.502933  4498 catalog_manager.cc:5697] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 6 to 9, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 9 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:30.504397  4675 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:30.461848  9088 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:30.507625  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.512135  4747 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:30.518306  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.534430  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.535008  4747 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:30.535151  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.535809  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.512979  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.494095  8813 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:30.533329  4756 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:30.508118  8982 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.491938  9189 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:30.535954  4751 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:30.537096  4749 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
W20260709 12:26:30.543478  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49724.2us
I20260709 12:26:30.516201  9192 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:30.549837  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2400) took 1115 ms. Trace:
W20260709 12:26:30.555105  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.536819  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.551007  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:30.538956  4571 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:30.531697  4647 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:30.579522  9088 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:30.539803  9178 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:30.573603  4675 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:30.578272  9186 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:30.542564  9191 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:30.550254  9128 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:30.550529  9079 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:30.550388  9169 consensus_queue.cc:237] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:30.550173  8982 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:30.545583  9136 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:30.519145  9182 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:30.571017  9150 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:30.551503  9189 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.517274  9190 consensus_queue.cc:1059] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 12:26:30.545840  9022 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:30.593030  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 676) took 1253 ms. Trace:
I20260709 12:26:30.581388  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:30.605770  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 680) took 1206 ms. Trace:
W20260709 12:26:30.709851  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 128594us
I20260709 12:26:30.710076  4674 rpcz_store.cc:276] 0709 12:26:29.399267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.903504 (+504237us) service_pool.cc:224] Handling call
0709 12:26:29.903828 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:29.904253 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:30.156812 (+252559us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.369101 (+212289us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.369858 (+   757us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.370043 (+   185us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:30.370090 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.372278 (+  2188us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.605471 (+233193us) raft_consensus.cc:1685] finished
0709 12:26:30.605501 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.605691 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5233,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":59035,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":5815,"thread_start_us":379878,"threads_started":3,"wal-append.queue_time_us":315354}
I20260709 12:26:30.610634  4598 rpcz_store.cc:276] 0709 12:26:29.434802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.018526 (+583724us) service_pool.cc:224] Handling call
0709 12:26:30.067905 (+ 49379us) spinlock_profiling.cc:243] Waited 47.3 ms on lock 0x7b5800002798. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:26:30.549588 (+481683us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b5400353ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:30.549749 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":858191104}
W20260709 12:26:30.611536  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.580681  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2366) took 1097 ms. Trace:
I20260709 12:26:30.713052  4750 rpcz_store.cc:276] 0709 12:26:29.482837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.153632 (+670795us) service_pool.cc:224] Handling call
0709 12:26:30.553641 (+400009us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b54000a7154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:30.580468 (+ 26827us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456498560}
I20260709 12:26:30.612013  4747 raft_consensus.cc:2393] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:30.588066  4754 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:26:30.647224  4755 raft_consensus.cc:2410] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 12:26:30.590581  9171 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
W20260709 12:26:30.593884  9172 log.cc:927] Time spent T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
I20260709 12:26:30.677983  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:30.709355  4673 rpcz_store.cc:276] 0709 12:26:29.340032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.897437 (+557405us) service_pool.cc:224] Handling call
0709 12:26:29.897794 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:29.902008 (+  4214us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:30.172434 (+270426us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.361642 (+189208us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.362266 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.362468 (+   202us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:30.362516 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.362603 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.592745 (+230142us) raft_consensus.cc:1685] finished
0709 12:26:30.592775 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.592951 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7253,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":32266,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":479,"thread_start_us":390659,"threads_started":3,"wal-append.queue_time_us":275457}
I20260709 12:26:30.629993  4749 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:30.630291  4681 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:30.632241  4678 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:30.582393  4748 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
W20260709 12:26:30.612824  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 81808.3us
I20260709 12:26:30.603363  9176 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:30.612263  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 73402.7us
I20260709 12:26:30.582967  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:30.632531  9040 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:30.587913  4498 catalog_manager.cc:5697] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:30.605958  9192 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:30.602658  8983 consensus_queue.cc:1059] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 12:26:30.625890  9031 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:30.612416  8789 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:30.602545  9196 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 12:26:30.631032  9178 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:30.626976  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:30.739598  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:30.739768  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:30.742977  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 160078us
I20260709 12:26:30.835186  4679 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:30.835858  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:30.622378  8984 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:30.742110  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:30.742650  8997 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.691088  9182 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.695721  9191 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.695513  9150 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:30.899927  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 118552us
I20260709 12:26:30.760536  5349 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:30.625459  9101 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.118s
I20260709 12:26:30.706595  9189 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.786849  4747 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:30.907094  4747 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:30.740332  9186 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.152s
I20260709 12:26:30.760380  4752 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 12:26:30.838114  9178 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:30.759058  4777 debug-util.cc:398] Leaking SignalData structure 0x7b0800196720 after lost signal to thread 3431
I20260709 12:26:30.800290  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:30.700937  9200 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:30.623611  9088 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:30.708598  9022 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:30.830348  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.794792  4498 catalog_manager.cc:5697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 3 to 7, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 7 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:30.753810  9174 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:30.838457  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 96457us
I20260709 12:26:30.663753  9126 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:30.758726  9198 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.839962  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:30.959559  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 274903us
I20260709 12:26:30.960078  4674 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:30.733353  8447 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.829691  8789 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.774205  9192 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:30.885005  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:30.754951  9176 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:30.867810  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:30.781847  9040 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:30.833905  9089 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:30.891429  8992 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:30.771219  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.909863  4747 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
W20260709 12:26:30.878058  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 138607us
I20260709 12:26:31.086436  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:30.936951  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.941354  4777 debug-util.cc:398] Leaking SignalData structure 0x7b08001e3240 after lost signal to thread 3434
I20260709 12:26:31.087635  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:31.089035  4777 debug-util.cc:398] Leaking SignalData structure 0x7b08001e3c80 after lost signal to thread 4536
I20260709 12:26:31.089253  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:31.089907  4777 debug-util.cc:398] Leaking SignalData structure 0x7b0800107a80 after lost signal to thread 4569
W20260709 12:26:31.091259  4777 debug-util.cc:398] Leaking SignalData structure 0x7b08000ed5c0 after lost signal to thread 4630
W20260709 12:26:31.091610  4777 debug-util.cc:398] Leaking SignalData structure 0x7b080018a880 after lost signal to thread 4631
W20260709 12:26:31.092935  4777 debug-util.cc:398] Leaking SignalData structure 0x7b08001974c0 after lost signal to thread 4706
W20260709 12:26:30.941496  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.093954  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.837860  9031 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:30.900228  9199 log.cc:927] Time spent T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:26:30.943414  4571 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:30.955288  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:31.075516  9195 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:26:30.945863  9022 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:30.851431  9181 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 12:26:30.962878  4498 catalog_manager.cc:5697] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 9 to 10, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 10 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:30.898154  9136 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:30.964244  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:30.909202  8928 consensus_meta.cc:287] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.700s	user 0.009s	sys 0.003s
I20260709 12:26:30.903880  9150 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:30.944573  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 158394us
I20260709 12:26:30.901302  9182 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:31.141774  8928 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:30.933910  5340 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:31.105074  9031 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:30.976035  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:30.976142  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 140355us
I20260709 12:26:30.941994  9174 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:31.002170  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.997483  5342 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:30.973809  9040 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:31.150511  4678 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:26:30.966370  9200 raft_consensus.cc:697] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:30.973244  9176 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:30.934541  9177 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:26:31.042761  4675 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:30.977380  8992 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.962785  9036 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.966754  8983 raft_consensus.cc:3055] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:30.908624  9191 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:30.972307  8447 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:31.109937  9211 consensus_queue.cc:1059] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:26:30.997331  9207 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.037014  9190 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:31.080580  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 110466us
I20260709 12:26:30.965696  9198 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:31.099197  4777 debug-util.cc:398] Leaking SignalData structure 0x7b08001095c0 after lost signal to thread 4780
I20260709 12:26:30.961318  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:31.101701  4681 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:31.103701  9216 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:26:31.104099  9188 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:26:31.104398  9186 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:31.104684  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:31.193857  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 89236.4us
I20260709 12:26:31.194824  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:31.105058  9210 log.cc:927] Time spent T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
W20260709 12:26:31.105759  9209 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
W20260709 12:26:31.107039  9204 log.cc:927] Time spent T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:26:31.082535  9153 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.592s	user 0.007s	sys 0.005s
I20260709 12:26:31.177843  8983 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:31.137265  4679 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
W20260709 12:26:31.144546  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 201192us
W20260709 12:26:31.139911  9151 consensus_meta.cc:287] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.756s	user 0.001s	sys 0.012s
I20260709 12:26:31.131110  9101 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:31.145987  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 145273us
I20260709 12:26:31.147389  4676 raft_consensus.cc:2368] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11.
I20260709 12:26:30.893421  8997 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:31.143559  4498 catalog_manager.cc:5697] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:30.965828  8789 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:31.163935  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:31.238070  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 74223us
I20260709 12:26:31.176505  5312 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:31.149875  9219 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:26:31.163143  9040 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:31.165460  9174 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:31.186764  4777 debug-util.cc:398] Leaking SignalData structure 0x7b0800362460 after lost signal to thread 5373
I20260709 12:26:31.186735  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:31.166348  9031 consensus_queue.cc:1059] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:26:31.174371  8992 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:31.112897  4645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.000s	sys 0.014s
W20260709 12:26:31.206593  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.267864  4645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.000s	sys 0.015s
W20260709 12:26:31.268191  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 694604us
I20260709 12:26:31.225265  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:31.249141  9022 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:31.237597  9126 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:31.186614  9222 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:31.192973  9186 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:31.300190  9186 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:31.222820  9153 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:31.303247  9153 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:31.260821  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:31.304970  4756 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 12: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:31.297052  9222 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.184887  9036 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:31.306049  9222 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:31.155539  9089 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:31.307790  9036 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:31.202776  9224 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:31.238780  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:31.237229  9101 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:31.292659  4498 catalog_manager.cc:5697] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 7 to 8, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 8 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:31.237020  9136 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:31.295720  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2945) took 1038 ms. Trace:
I20260709 12:26:31.301432  9022 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:31.254071  8928 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:31.313827  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:31.331390  4598 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:31.315668  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:31.324623  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 124290us
I20260709 12:26:31.312165  4677 rpcz_store.cc:276] 0709 12:26:30.257703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.257844 (+   141us) service_pool.cc:224] Handling call
0709 12:26:30.258243 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.329894 (+ 71651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.329940 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.330584 (+   644us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.489347 (+158763us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:30.489392 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.489467 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.295186 (+805719us) raft_consensus.cc:1685] finished
0709 12:26:31.295502 (+   316us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.295652 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":124898,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":229401,"threads_started":2,"wal-append.queue_time_us":401868}
I20260709 12:26:31.252903  9213 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:31.311393  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 72700.7us
W20260709 12:26:31.338683  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 112129us
I20260709 12:26:31.341275  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:31.341275  4605 pending_rounds.cc:85] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 10
I20260709 12:26:31.342286  4605 pending_rounds.cc:107] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 12:26:31.342517  4605 raft_consensus.cc:2889] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:31.345852  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:31.345744  9180 consensus_queue.cc:1059] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:31.346130  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:31.346730  9180 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:31.347103  9180 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:31.347309  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:31.347983  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:31.348381  4598 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:31.347967  9180 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:31.348848  4596 raft_consensus.cc:2393] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:31.349259  9180 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:31.354152  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.357147  4498 catalog_manager.cc:5697] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:31.352257  9153 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:31.359766  4602 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:31.360227  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:31.331404  9206 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.287s	user 0.000s	sys 0.001s
I20260709 12:26:31.349985  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:31.292009  9226 raft_consensus.cc:3055] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:31.361279  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:31.362666  9178 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:31.349514  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:31.365793  4720 connection.cc:256] RPC call timeout handler was delayed by -2.7302044260000002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1778s for a response
I20260709 12:26:31.307807  9201 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:26:31.341049  9203 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
W20260709 12:26:31.369189  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 192802us
I20260709 12:26:31.197824  9207 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:31.409698  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 184557us
I20260709 12:26:31.370781  9226 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:31.310745  9224 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:31.423260  9224 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:31.424162  9191 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:31.269486  9200 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:31.330246  9101 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:31.382462  4498 catalog_manager.cc:5697] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
W20260709 12:26:31.383841  4720 connection.cc:256] RPC call timeout handler was delayed by -2.7149044259999755s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
I20260709 12:26:31.383854  4599 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
W20260709 12:26:31.429673  4720 connection.cc:256] RPC call timeout handler was delayed by -2.6978044259999479s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
W20260709 12:26:31.430126  4720 connection.cc:256] RPC call timeout handler was delayed by -2.6708044259999042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
W20260709 12:26:31.381676  9184 log.cc:927] Time spent T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:26:31.341849  9213 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:31.403806  5312 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:31.374446  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:31.432901  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:31.405893  9238 consensus_queue.cc:1059] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:31.415659  9031 consensus_queue.cc:1059] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:31.422319  9226 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:31.430399  4720 connection.cc:256] RPC call timeout handler was delayed by -2.664504425999894s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
W20260709 12:26:31.441716  4720 connection.cc:256] RPC call timeout handler was delayed by -2.6582044259998838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
W20260709 12:26:31.442090  4720 connection.cc:256] RPC call timeout handler was delayed by -2.6582044259998838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
W20260709 12:26:31.442379  4720 connection.cc:256] RPC call timeout handler was delayed by -2.6429044259998591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1875s for a response
W20260709 12:26:31.442663  4720 connection.cc:256] RPC call timeout handler was delayed by -2.6429044259998591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1875s for a response
I20260709 12:26:31.373198  9153 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:31.341351  9036 consensus_queue.cc:1059] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:26:31.312999  8983 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:31.530720  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.444552  4498 catalog_manager.cc:5697] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 7 to 8, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 8 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:31.410595  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:31.468000  9221 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:26:31.468979  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 667) took 2026 ms. Trace:
W20260709 12:26:31.469691  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.541649  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 138815us
W20260709 12:26:31.542124  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.312500  9228 consensus_queue.cc:1059] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.113s
I20260709 12:26:31.442651  9213 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:31.400559  9022 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:26:31.471640  9177 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:31.442564  8928 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.412760  8998 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:31.477743  9226 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:31.478719  9198 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:31.480270  9153 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:31.487411  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:31.487713  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:31.488454  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:31.490948  9178 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:31.529232  9101 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:31.566630  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 156104us
I20260709 12:26:31.578950  4599 raft_consensus.cc:2393] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:31.586012  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:31.592111  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:31.594513  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:31.597281  4602 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:26:31.569962  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 301137us
I20260709 12:26:31.541252  4600 rpcz_store.cc:276] 0709 12:26:29.442677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.210607 (+767930us) service_pool.cc:224] Handling call
0709 12:26:30.211006 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.211435 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.211471 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.212034 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.781465 (+569431us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:30.781508 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.781603 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.468738 (+687135us) raft_consensus.cc:1685] finished
0709 12:26:31.468760 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.468903 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4541,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":21067,"thread_start_us":487034,"threads_started":1,"wal-append.queue_time_us":668486}
W20260709 12:26:31.604957  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 410203us
I20260709 12:26:31.607249  4677 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:31.557075  8447 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:31.616631  5312 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:31.624053  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:31.624454  4498 catalog_manager.cc:5697] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 7 to 10, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 10 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:31.618991  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.635615  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:31.590793  9091 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 1.081s	user 0.012s	sys 0.010s
W20260709 12:26:31.642716  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 276324us
I20260709 12:26:31.643251  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:31.646462  5312 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:31.570520  8998 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:31.647651  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 204417us
W20260709 12:26:31.659106  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.644383  4645 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:31.606256  9198 raft_consensus.cc:3055] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:31.589543  8983 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:31.642810  9203 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:26:31.590408  9238 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:31.692580  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48277.6us
I20260709 12:26:31.574533  8928 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:31.572211  8992 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:31.688083  4569 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:31.700557  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:31.707757  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 116222us
I20260709 12:26:31.677862  9200 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:31.718089  4498 catalog_manager.cc:5697] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:31.640794  9216 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:31.719024  4645 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:31.683305  8998 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:31.682471  9035 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:31.678033  9136 raft_consensus.cc:697] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:31.683661  9088 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:31.725765  4597 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:31.690408  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:31.686667  9198 raft_consensus.cc:740] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:31.750459  4679 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:31.718941  9183 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:31.693486  9187 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:31.743443  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:31.745638  4603 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:31.746022  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2946) took 1485 ms. Trace:
I20260709 12:26:31.685446  8997 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:31.746897  4498 catalog_manager.cc:5697] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 7 to 8, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 8 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:31.676347  9091 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:31.647754  9228 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:26:31.675859  8447 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
W20260709 12:26:31.749693  4676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 2947) took 1449 ms (client timeout 1749 ms). Trace:
W20260709 12:26:31.782828  4676 rpcz_store.cc:269] 0709 12:26:30.300263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.300813 (+   550us) service_pool.cc:224] Handling call
0709 12:26:31.749379 (+1448566us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b54002dee54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:31.749590 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1944800640}
I20260709 12:26:31.754726  4672 rpcz_store.cc:276] 0709 12:26:30.260515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.260700 (+   185us) service_pool.cc:224] Handling call
0709 12:26:30.261020 (+   320us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:30.480720 (+219700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.480767 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:30.481548 (+   781us) log.cc:844] Serialized 52 byte log entry
0709 12:26:30.703699 (+222151us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:30.732329 (+ 28630us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.732538 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.732610 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.745727 (+1013117us) raft_consensus.cc:1685] finished
0709 12:26:31.745758 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.745941 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36268,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":225100,"raft.run_cpu_time_us":4139,"raft.run_wall_time_us":51113,"thread_start_us":389781,"threads_started":2,"wal-append.queue_time_us":387911}
W20260709 12:26:31.792245  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 65051.5us
I20260709 12:26:31.754701  9128 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:31.797816  4596 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:31.727373  9249 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:31.735476  9178 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:26:31.742791  9191 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:26:31.739408  9238 consensus_queue.cc:1059] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
W20260709 12:26:31.804941  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 116937us
I20260709 12:26:31.759856  9244 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:31.879366  9030 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.564s	user 0.011s	sys 0.010s
I20260709 12:26:31.806905  9036 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:31.895123  4720 connection.cc:256] RPC call timeout handler was delayed by -2.4629044260000228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
W20260709 12:26:31.895562  4720 connection.cc:256] RPC call timeout handler was delayed by -2.4197044259999529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21230000000000004s for a response
I20260709 12:26:31.893800  9036 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.903748  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:31.804217  9249 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.904407  4676 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
W20260709 12:26:31.815688  9189 consensus_meta.cc:287] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.887s	user 0.016s	sys 0.000s
I20260709 12:26:31.897311  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:26:31.895926  4720 connection.cc:256] RPC call timeout handler was delayed by -2.4143044259999442s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2129s for a response
I20260709 12:26:31.790020  9219 consensus_queue.cc:1059] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:26:31.937515  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:31.938272  4676 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:31.927073  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:31.929219  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.796475  8997 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 12:26:31.946028  4681 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:26:31.938928  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.798264  9177 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.098s
I20260709 12:26:31.946273  4677 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
W20260709 12:26:31.894057  9182 consensus_meta.cc:287] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.752s	user 0.014s	sys 0.014s
I20260709 12:26:31.880748  8976 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.966197  4677 pending_rounds.cc:85] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 5
W20260709 12:26:31.901062  9222 consensus_meta.cc:287] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.595s	user 0.022s	sys 0.000s
W20260709 12:26:31.969163  4720 connection.cc:256] RPC call timeout handler was delayed by -2.37470442599988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
I20260709 12:26:31.965234  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:31.973502  4720 connection.cc:256] RPC call timeout handler was delayed by -2.3657044259998656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2183s for a response
W20260709 12:26:31.973835  4720 connection.cc:256] RPC call timeout handler was delayed by -2.3198044259997914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22340000000000002s for a response
W20260709 12:26:31.974087  4720 connection.cc:256] RPC call timeout handler was delayed by -2.3153044259997841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2239s for a response
W20260709 12:26:31.974367  4720 connection.cc:256] RPC call timeout handler was delayed by -2.3081044259997725s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2247s for a response
I20260709 12:26:31.974756  4676 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:31.975220  4720 connection.cc:256] RPC call timeout handler was delayed by -1.9346044260000781s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2662s for a response
W20260709 12:26:31.975555  4720 connection.cc:256] RPC call timeout handler was delayed by -1.4234044259997063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.323s for a response
W20260709 12:26:31.975821  4720 connection.cc:256] RPC call timeout handler was delayed by -1.4234044259997063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.323s for a response
I20260709 12:26:31.965797  4678 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:31.976397  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 252074us
I20260709 12:26:31.969257  4755 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:31.901185  9036 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:31.974598  4677 pending_rounds.cc:107] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 10.6
W20260709 12:26:31.939532  9040 consensus_meta.cc:287] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.000s
W20260709 12:26:31.926017  8982 consensus_meta.cc:287] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.225s	user 0.013s	sys 0.000s
I20260709 12:26:31.978241  5314 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:31.809293  9187 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.133s
I20260709 12:26:31.983166  4647 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:31.896198  9128 raft_consensus.cc:697] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:31.984088  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 240757us
I20260709 12:26:31.985163  4677 raft_consensus.cc:2889] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:31.987735  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:32.009179  4597 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:31.964901  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:31.902148  9207 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:31.966140  9244 raft_consensus.cc:2764] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:26:32.021486  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.019109  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:32.023701  4602 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:31.928238  9249 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:31.898566  9224 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:31.978468  8789 consensus_meta.cc:287] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.725s	user 0.004s	sys 0.015s
I20260709 12:26:32.063321  8992 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.062943  8789 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:31.984380  8976 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:32.069664  9240 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.380s	user 0.000s	sys 0.001s
W20260709 12:26:32.069913  9251 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:26:31.962172  9216 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:31.812868  9198 consensus_queue.cc:260] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:31.895769  9250 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.224s
I20260709 12:26:32.071559  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:32.072127  9101 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:32.073220  9248 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
W20260709 12:26:32.073948  9150 consensus_meta.cc:287] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.920s	user 0.013s	sys 0.005s
I20260709 12:26:32.068430  8976 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:32.074368  9150 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.075591  9079 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:32.026299  9191 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:31.903946  9030 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:31.964167  9189 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.006434  9040 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:32.071512  9246 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:26:32.036505  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:31.982226  9222 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:31.965236  9241 consensus_queue.cc:1059] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 12:26:32.087276  4681 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:32.089725  4674 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:32.092271  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 53613.9us
I20260709 12:26:32.006660  8982 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.018004  9253 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 12:26:32.040961  9177 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:32.099450  9245 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:32.105753  9177 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.106112  9245 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.075347  8976 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:32.106670  9245 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:31.985389  9136 consensus_queue.cc:237] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.109963  9069 raft_consensus.cc:3055] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:32.110399  9069 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:32.110674  9211 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:32.110347  9128 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.111130  9211 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:32.111323  9069 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.112432  9069 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:32.112033  9211 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.112815  8789 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.113216  9211 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:32.103952  4754 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:32.101389  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:32.131330  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:32.105481  9257 consensus_queue.cc:1175] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49277a7653524787a715232ab908bbc6" responder_term: 11 status { last_received { term: 10 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:32.113822  9260 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.115291  9224 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:32.115548  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 693) took 1117 ms. Trace:
I20260709 12:26:32.076867  9244 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:26:32.117529  9254 raft_consensus.cc:3055] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:32.117930  9089 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:32.146719  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:32.119863  9189 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:32.121301  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 2922) took 1613 ms. Trace:
I20260709 12:26:32.150017  4756 rpcz_store.cc:276] 0709 12:26:30.508347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.550742 (+ 42395us) service_pool.cc:224] Handling call
0709 12:26:32.121082 (+1570340us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54000730d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:32.121239 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1593402240}
W20260709 12:26:32.150806  9258 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:26:32.145642  8976 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:32.145965  9254 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:32.152141  9254 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.133138  9091 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:32.153367  9254 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:32.154059  9222 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.157883  9264 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.158463  9264 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.160810  9264 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.121527  9030 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:32.122188  9220 raft_consensus.cc:3055] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:32.122970  9040 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:32.101881  9191 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:32.102104  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 113791us
W20260709 12:26:32.097314  9192 consensus_meta.cc:287] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.095s	user 0.000s	sys 0.012s
I20260709 12:26:32.042493  9178 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:26:32.129331  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:32.129709  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 151536us
W20260709 12:26:32.129878  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.172425  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:32.106258  9177 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:31.980702  9182 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.094981  9200 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:32.094859  8928 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.135313  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:32.139020  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 340690us
I20260709 12:26:32.142254  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:32.144695  4679 rpcz_store.cc:276] 0709 12:26:30.998610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.226532 (+227922us) service_pool.cc:224] Handling call
0709 12:26:31.750221 (+523689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.750977 (+   756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.115354 (+364377us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b54002dee50. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:32.115489 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1141935872}
I20260709 12:26:32.147954  4681 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:32.183936  4681 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:32.099681  9150 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:32.149048  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 47699.9us
I20260709 12:26:32.146618  9260 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.166615  4677 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:32.169684  4647 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:32.170652  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:32.174651  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 689) took 1231 ms. Trace:
I20260709 12:26:32.177309  4756 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:26:32.185621  9191 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.203188  4756 pending_rounds.cc:85] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 8
I20260709 12:26:32.203639  4756 pending_rounds.cc:107] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:26:32.186408  9200 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.188117  9182 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.167196  9220 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:32.185624  9186 consensus_meta.cc:287] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.885s	user 0.015s	sys 0.007s
W20260709 12:26:32.189965  9176 consensus_meta.cc:287] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 1.008s	user 0.007s	sys 0.014s
I20260709 12:26:32.196801  4605 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:32.197271  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.199239  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:32.204304  4756 raft_consensus.cc:2889] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:32.203756  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:32.162575  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2931) took 1651 ms. Trace:
I20260709 12:26:32.203014  4675 rpcz_store.cc:276] 0709 12:26:30.943429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.168390 (+224961us) service_pool.cc:224] Handling call
0709 12:26:31.184702 (+ 16312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:31.185063 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.185092 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:31.185537 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 12:26:31.641383 (+455846us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:31.641436 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.641533 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.174435 (+532902us) raft_consensus.cc:1685] finished
0709 12:26:32.174454 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.174591 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":412,"raft.queue_time_us":745,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":10772,"thread_start_us":404888,"threads_started":1,"wal-append.queue_time_us":577236}
I20260709 12:26:32.211141  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:32.210013  4748 rpcz_store.cc:276] 0709 12:26:30.511076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.782536 (+271460us) service_pool.cc:224] Handling call
0709 12:26:30.782811 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.783010 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:30.879090 (+ 96080us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.060772 (+181682us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.061066 (+   294us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.302636 (+241570us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b1800473d40. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:26:32.114669 (+812033us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b1800473d40. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c6455 00007effee4ae060 00007effee4c0716 00007effee4d1287 00007effee4d1208
0709 12:26:32.162073 (+ 47404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.162220 (+   147us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:32.162286 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.162357 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.162490 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11428,"prepare.queue_time_us":103895,"prepare.run_cpu_time_us":155,"prepare.run_wall_time_us":11342,"raft.queue_time_us":390386,"raft.run_cpu_time_us":422,"raft.run_wall_time_us":26976,"spinlock_wait_cycles":120129152,"thread_start_us":212431,"threads_started":3,"wal-append.queue_time_us":220883}
I20260709 12:26:32.212088  9186 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.212411  8928 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:32.212769  9241 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:32.214280  9241 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:32.214560  9186 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.217789  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:32.204458  9174 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.937s	user 0.014s	sys 0.005s
I20260709 12:26:32.215271  9176 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.228547  4750 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:32.237418  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:32.238058  4750 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:32.241408  4647 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:32.241513  4756 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:32.242543  4755 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:32.243355  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:32.244016  4756 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:32.233882  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:32.244783  4749 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:32.246644  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:32.259879  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 60749.6us
I20260709 12:26:32.260921  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:32.257411  4753 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:32.174177  9192 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:32.292922  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.215791  9261 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:32.227945  9268 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:32.229756  9182 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:32.309268  9040 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:32.244270  9174 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:32.312196  8998 consensus_meta.cc:287] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.002s
I20260709 12:26:32.271492  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:32.318156  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57333us
I20260709 12:26:32.317396  5349 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:32.256763  9224 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:32.332952  4680 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:32.236531  9150 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:32.346366  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:32.368290  4749 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:32.257104  8928 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:32.316479  9192 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.366690  5346 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:32.263470  9241 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:32.320744  8983 consensus_meta.cc:287] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.627s	user 0.002s	sys 0.006s
I20260709 12:26:32.372076  4750 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:32.372783  4720 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:32.381372  4756 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:32.248164  9189 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.117s
I20260709 12:26:32.398782  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:32.398933  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:32.399511  4748 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:32.400099  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:32.400334  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:32.400946  4756 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:32.401441  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:32.401762  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:32.402194  4750 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:32.402321  4756 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:32.403014  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:32.404265  4749 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:32.405158  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:32.405922  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:32.348085  9268 raft_consensus.cc:697] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:32.423134  4749 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:32.406339  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:32.406929  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 269174us
I20260709 12:26:32.365995  9182 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:32.425887  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:32.420697  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.378422  9224 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:32.428548  9180 consensus_meta.cc:287] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.059s	user 0.014s	sys 0.000s
I20260709 12:26:32.423532  9277 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:26:32.431974  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 59508.6us
I20260709 12:26:32.423019  4748 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:32.436296  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:32.433049  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:32.378788  8982 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:32.423245  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:32.445446  4750 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 12: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:32.445667  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:32.447324  4605 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:32.428369  9205 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.095s	user 0.006s	sys 0.011s
I20260709 12:26:32.447650  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:32.448000  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.448304  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:32.448426  4750 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:32.449097  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:32.449823  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:32.450171  4748 raft_consensus.cc:2410] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:26:32.450420  4749 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.371417  8998 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.453922  5340 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:32.455667  8998 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.458101  9224 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.369270  9220 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.462424  9220 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.466738  9176 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:32.399507  9244 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:26:32.468029  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:32.447849  9228 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:26:32.479171  4756 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:32.424568  9256 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:32.481168  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 45010us
I20260709 12:26:32.482123  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:32.467777  9036 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.486229  4605 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
W20260709 12:26:32.488448  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.485997  9256 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.467839  9244 raft_consensus.cc:3055] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:32.490233  9244 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:32.452837  9205 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.352s	user 0.008s	sys 0.011s
I20260709 12:26:32.494626  9256 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.491019  9244 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.444423  9241 raft_consensus.cc:697] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:32.444998  9277 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:32.515489  5342 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:26:32.517374  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 69125.8us
I20260709 12:26:32.472041  9177 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.529059  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:32.514907  9277 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:32.536175  5340 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:26:32.536768  9150 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.491714  9091 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:32.537184  8983 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:26:32.538190  9226 consensus_meta.cc:287] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.949s	user 0.012s	sys 0.000s
I20260709 12:26:32.550983  9150 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.552058  4604 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:32.553180  9150 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:32.540906  9249 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.435137  9266 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:26:32.494136  9180 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:32.537091  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:32.543154  8789 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:32.515167  9275 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:26:32.544974  9040 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.539095  9260 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.554463  4644 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:32.563973  9040 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.565697  9249 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.549319  9268 consensus_queue.cc:237] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.566322  9249 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:32.566632  9040 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:32.567034  9249 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:32.567899  9249 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.576460  4749 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:32.577363  4756 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:32.557127  9244 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.581039  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:32.493319  9182 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:32.586851  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 104820us
I20260709 12:26:32.587731  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2915) took 3083 ms. Trace:
I20260709 12:26:32.588181  4752 rpcz_store.cc:276] 0709 12:26:29.504204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.366133 (+861929us) service_pool.cc:224] Handling call
0709 12:26:30.760135 (+394002us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:30.909181 (+149046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.587468 (+1678287us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54001f1650. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:32.587662 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":905328000}
I20260709 12:26:32.590848  4569 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:32.586189  9069 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:32.591823  9213 consensus_meta.cc:287] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.017s	user 0.009s	sys 0.003s
I20260709 12:26:32.604676  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:32.578269  9180 consensus_queue.cc:1059] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:26:32.615015  9177 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.616755  9036 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:32.617389  9273 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.211s	user 0.005s	sys 0.010s
I20260709 12:26:32.608916  9254 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:32.621677  9254 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:32.622124  9254 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:32.609153  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:32.623862  4569 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:32.623956  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:32.624833  4605 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.610023  9260 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:32.634800  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:32.636720  4675 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.634070  9150 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:32.639343  9030 raft_consensus.cc:3055] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:32.639972  9030 raft_consensus.cc:740] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:32.640928  9030 consensus_queue.cc:260] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.642172  9030 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:32.634390  8789 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:32.646574  9153 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:32.649191  4605 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:32.649825  4720 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:32.654376  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.654363  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:32.656433  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:32.657850  4677 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:32.661121  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:32.663386  8928 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:32.605261  9241 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:32.624034  9213 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:32.667819  9208 log.cc:927] Time spent T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 1.592s	user 0.000s	sys 0.003s
I20260709 12:26:32.668944  9213 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:32.669166  9153 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:32.669559  9153 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.669925  9153 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:32.659060  8976 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:32.656353  9245 consensus_meta.cc:287] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.549s	user 0.006s	sys 0.005s
I20260709 12:26:32.674104  9245 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.675222  9236 raft_consensus.cc:3055] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:32.646620  9239 raft_consensus.cc:3055] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:32.676177  9239 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:32.677022  9239 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.677994  9239 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.678042  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2935) took 2144 ms. Trace:
I20260709 12:26:32.678500  4754 rpcz_store.cc:276] 0709 12:26:30.533718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.779082 (+245364us) service_pool.cc:224] Handling call
0709 12:26:30.779370 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.130794 (+1351424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.677793 (+546999us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54000555d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:32.677970 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3045821824}
I20260709 12:26:32.683104  8992 raft_consensus.cc:3055] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:32.662074  8789 consensus_queue.cc:1059] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:26:32.683983  9277 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.684976  9266 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:32.675783  9236 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:32.683576  8992 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:32.695170  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 2923) took 2186 ms. Trace:
I20260709 12:26:32.699112  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:32.654094  9150 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:32.701927  9278 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:26:32.702229  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:32.700045  4605 raft_consensus.cc:2376] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8. Re-sending same reply.
I20260709 12:26:32.699703  4751 rpcz_store.cc:276] 0709 12:26:30.508639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.552967 (+ 44328us) service_pool.cc:224] Handling call
0709 12:26:30.582091 (+ 29124us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.582431 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.582464 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.587464 (+  5000us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.675065 (+ 87601us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:30.696376 (+ 21311us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.696649 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.696739 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.265529 (+568790us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b18003d7440. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:26:32.667199 (+1401670us) spinlock_profiling.cc:243] Waited 58.5 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:32.694969 (+ 27770us) raft_consensus.cc:1685] finished
0709 12:26:32.694989 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.695111 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26249,"prepare.run_cpu_time_us":1833,"prepare.run_wall_time_us":16687,"raft.queue_time_us":16565,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":22189,"spinlock_wait_cycles":146506368,"thread_start_us":87159,"threads_started":1,"wal-append.queue_time_us":410466}
I20260709 12:26:32.704314  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:32.705022  4751 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.699714  4675 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:32.710933  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:32.711133  9196 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:32.712002  9040 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:32.713191  4752 raft_consensus.cc:1240] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:32.713531  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:32.714751  4750 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 12: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:32.719444  9277 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:32.720682  9236 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.722100  9245 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.740338  4751 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:32.746690  8992 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.749716  8992 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.758980  5346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53808 (request call id 583) took 1179 ms. Trace:
I20260709 12:26:32.759462  5346 rpcz_store.cc:276] 0709 12:26:31.579867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.587747 (+  7880us) service_pool.cc:224] Handling call
0709 12:26:32.758722 (+1170975us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b54004c5ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:32.758902 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1671975680}
W20260709 12:26:32.763623  9281 log.cc:927] Time spent T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.334s	user 0.001s	sys 0.000s
I20260709 12:26:32.767680  9169 raft_consensus.cc:3055] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:32.768337  9169 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:32.785929  8984 raft_consensus.cc:3055] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:32.878469  8984 raft_consensus.cc:740] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:32.852912  4597 raft_consensus.cc:2410] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:26:32.882041  9187 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:32.853933  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:32.851058  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 153054us
I20260709 12:26:32.883330  9187 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.887451  9187 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:32.888259  8500 raft_consensus.cc:3055] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:32.889020  8500 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:32.743888  9153 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.890225  4600 raft_consensus.cc:2410] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:26:32.892234  9207 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:32.892831  9207 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:32.894310  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:32.769443  9169 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:32.892784  9217 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 1.786s	user 0.003s	sys 0.013s
I20260709 12:26:32.896327  5340 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:32.882071  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:32.851828  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 693) took 1296 ms. Trace:
I20260709 12:26:32.901424  9239 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:26:32.904037  9239 raft_consensus.cc:3055] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:32.904599  9239 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:32.908211  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:32.908488  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1922) took 1563 ms. Trace:
I20260709 12:26:32.853223  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:32.911569  9207 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.901511  4599 rpcz_store.cc:276] 0709 12:26:31.555382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.596814 (+ 41432us) service_pool.cc:224] Handling call
0709 12:26:31.597200 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:31.980291 (+383091us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:32.146066 (+165775us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.348664 (+202598us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.364241 (+ 15577us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.364893 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.365085 (+   192us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:32.365312 (+   227us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.369959 (+  4647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.370036 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.572660 (+202624us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b180005b9c0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c9522 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:26:32.669025 (+ 96365us) spinlock_profiling.cc:243] Waited 5.5 ms on lock 0x7b180005b9c0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c9522 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:26:32.762361 (+ 93336us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:32.766893 (+  4532us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b180005b9c0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:26:32.850878 (+ 83985us) raft_consensus.cc:1685] finished
0709 12:26:32.850981 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.851494 (+   513us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37500,"prepare.queue_time_us":120598,"prepare.run_cpu_time_us":155,"prepare.run_wall_time_us":159,"raft.queue_time_us":211814,"raft.run_cpu_time_us":472,"raft.run_wall_time_us":46528,"spinlock_wait_cycles":75411840,"thread_start_us":701651,"threads_started":3,"wal-append.queue_time_us":259930}
I20260709 12:26:32.920792  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 692) took 1976 ms. Trace:
I20260709 12:26:32.786389  9254 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.921262  4674 rpcz_store.cc:276] 0709 12:26:30.944254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.201014 (+256760us) service_pool.cc:224] Handling call
0709 12:26:31.201275 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.100383 (+899108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.920050 (+819667us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b54001d33d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:32.920444 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2037811968}
I20260709 12:26:32.924825  9254 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.909066  4603 rpcz_store.cc:276] 0709 12:26:31.344825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.347093 (+  2268us) service_pool.cc:224] Handling call
0709 12:26:32.897415 (+1550322us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b5400141ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:32.908251 (+ 10836us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":914256256}
I20260709 12:26:32.928308  9153 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:32.930122  9239 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.931211  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:32.933439  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:32.933894  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:32.935380  9239 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.936232  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:32.936852  9174 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:32.938731  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:32.941277  4675 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 9: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:32.941795  4747 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:32.942904  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:32.909302  4598 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:32.944622  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:32.945941  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:32.946308  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:32.947052  4673 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:32.947376  4598 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:32.947769  9215 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:32.948490  9215 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:32.949055  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:32.951743  8813 consensus_queue.cc:1059] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:32.955130  8500 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:32.958797  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:32.880722  8984 consensus_queue.cc:260] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.951790  4598 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:32.962986  8984 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:32.964283  9263 raft_consensus.cc:3055] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:32.964826  9263 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:32.966655  9264 raft_consensus.cc:3055] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:32.967293  9264 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:32.969260  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.863075  4751 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:32.971328  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:32.807397  9293 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:26:32.973877  9174 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.002s
W20260709 12:26:32.802451  9297 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:26:32.975646  9169 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:33.000695  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:32.969019  9264 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:32.889441  9187 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:33.024466  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.036676  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:33.038663  4598 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:33.040778  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:33.042586  8500 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:33.043011  4598 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:33.048768  9215 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:33.050678  9215 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:33.063251  4598 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:33.064603  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:33.064844  9187 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:33.070420  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.076352  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:33.101732  4597 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 12: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:33.077157  9153 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.105553  9181 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:33.105819  9153 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:33.106446  9264 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:33.106596  9181 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:33.108691  9153 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:33.109714  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 63313.8us
W20260709 12:26:33.111351  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.104347  9187 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:33.118093  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:33.119171  4672 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 16: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:33.119516  4605 raft_consensus.cc:2410] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:33.120275  9277 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.121153  9181 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:33.123246  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.126847  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.126997  9239 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:33.128078  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:33.130153  8992 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:33.130764  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:33.130918  8992 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:33.131928  4597 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:33.138088  9239 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:33.120982  9277 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:33.042495  9263 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:33.172042  9224 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.020s	sys 0.045s
W20260709 12:26:33.172588  9224 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.020s	sys 0.045s
W20260709 12:26:33.173822  4604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1928) took 1799 ms (client timeout 2201 ms). Trace:
W20260709 12:26:33.174386  4604 rpcz_store.cc:269] 0709 12:26:31.373854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.374144 (+   290us) service_pool.cc:224] Handling call
0709 12:26:33.173292 (+1799148us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540032b9d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:33.173597 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2574958976}
W20260709 12:26:33.175343  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.176379  4604 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:33.190920  4597 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:33.218743  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:33.192772  9254 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.224648  4598 raft_consensus.cc:2410] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:26:33.234483  4596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1927) took 1884 ms (client timeout 1917 ms). Trace:
W20260709 12:26:33.234941  4596 rpcz_store.cc:269] 0709 12:26:31.349652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.349749 (+    97us) service_pool.cc:224] Handling call
0709 12:26:33.234043 (+1884294us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b540004f454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:33.234284 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":985485184}
I20260709 12:26:33.240170  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 2959) took 2613 ms. Trace:
I20260709 12:26:33.240662  4680 rpcz_store.cc:276] 0709 12:26:30.627038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.627210 (+   172us) service_pool.cc:224] Handling call
0709 12:26:30.627509 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.367174 (+1739665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.239563 (+872389us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b540017f554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:33.239959 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3901018368}
I20260709 12:26:33.243350  9263 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:33.214084  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 405320us
I20260709 12:26:33.213961  9236 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:33.215293  8992 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:33.247375  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:33.274521  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.262108  9040 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.275822  9040 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:33.253345  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.286607  4753 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:33.298223  9243 raft_consensus.cc:3055] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:33.299031  9243 raft_consensus.cc:740] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:33.300395  9243 consensus_queue.cc:260] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:33.281450  5349 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:33.320266  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.321069  4751 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:33.273540  4680 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:33.319036  9207 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:33.330749  9010 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:26:33.331178  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:33.331732  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:33.332830  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.333168  9198 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:33.337873  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:33.340543  4605 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:26:33.342427  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.349022  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 700) took 1220 ms. Trace:
I20260709 12:26:33.349658  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:33.344543  4599 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:33.350188  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:33.250623  9254 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:33.357277  4598 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:33.364418  9277 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:33.365094  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:33.365362  4674 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:33.365880  4678 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:33.357161  4681 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:33.390347  5349 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:33.357494  9239 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:33.349529  4602 rpcz_store.cc:276] 0709 12:26:32.125451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.125724 (+   273us) service_pool.cc:224] Handling call
0709 12:26:32.142294 (+ 16570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.200291 (+ 57997us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.200323 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.207598 (+  7275us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.900296 (+692698us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b18003920e0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:26:33.190438 (+290142us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:33.190559 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.207119 (+ 16560us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:33.342217 (+135098us) raft_consensus.cc:1685] finished
0709 12:26:33.342234 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.346014 (+  3780us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57485,"raft.queue_time_us":58871,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":7796,"spinlock_wait_cycles":26737408,"thread_start_us":279699,"threads_started":1,"wal-append.queue_time_us":1004631}
W20260709 12:26:33.402395  4720 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.023s
I20260709 12:26:33.407131  9243 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:33.410738  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:33.411221  9040 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:33.411664  9224 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.412460  9224 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:33.414191  4720 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.000s	sys 0.024s
W20260709 12:26:33.415527  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.420454  4596 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:33.420027  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.424597  4678 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
W20260709 12:26:33.426091  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.427836  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:33.428459  4599 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:33.428547  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.426887  9224 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:33.421586  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 715279us
I20260709 12:26:33.438862  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:33.439466  4605 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:33.443233  4676 raft_consensus.cc:2368] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11.
I20260709 12:26:33.435787  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:33.447420  9030 consensus_meta.cc:287] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.805s	user 0.012s	sys 0.014s
I20260709 12:26:33.454895  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:33.461153  9241 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:33.468353  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:33.469542  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.469903  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.471300  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.472252  4678 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:33.475586  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.492985  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:33.498441  4681 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:33.502494  9254 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:33.503211  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 92687us
I20260709 12:26:33.504699  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:33.508656  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:33.519109  4604 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.6-12.6]
W20260709 12:26:33.520172  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.521828  4597 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:26:33.522328  4605 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:33.521391  4676 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:33.527920  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.505795  9241 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:33.501319  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:33.529771  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:33.532272  4596 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:26:33.531332  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.533965  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.538700  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.539945  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.532646  9241 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.534466  9010 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.553002  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:33.557157  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.561340  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:33.564724  4596 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:33.550252  9241 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:33.549613  9254 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:33.560202  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:33.569764  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.572201  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:33.578604  4678 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:33.581920  4680 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:33.584918  4597 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:33.585098  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:33.603740  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:33.610962  5349 raft_consensus.cc:2393] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 13: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:33.616536  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:33.619089  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.558975  9010 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:33.620198  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:33.580777  9254 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.623926  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.624751  9010 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:33.626005  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.621678  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.626910  4681 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:33.559197  9239 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:33.635089  4676 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:33.619168  4680 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 12. Ops: [10.7-10.7]
I20260709 12:26:33.637494  4602 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:33.645475  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:33.645529  4678 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:33.648492  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 147442us
I20260709 12:26:33.648550  4672 raft_consensus.cc:2393] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:33.652901  4601 raft_consensus.cc:2410] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:26:33.654110  4597 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:33.657307  4605 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:33.636807  9254 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:33.666515  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.669137  4596 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:26:33.665162  9239 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:33.670840  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:33.675839  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:33.676702  5342 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:33.678074  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:33.680676  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.680923  4647 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:33.681551  4675 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:33.682559  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:33.667797  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:33.684868  4681 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:33.686093  4599 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:33.687830  4678 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:33.694485  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.708353  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.712435  4569 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:33.713686  9236 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:33.714272  9236 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.714712  9236 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:33.716099  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.719630  9264 consensus_queue.cc:1059] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:33.721169  9264 raft_consensus.cc:3055] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:33.722671  9264 raft_consensus.cc:740] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:33.723105  9169 consensus_queue.cc:1059] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:33.723760  9264 consensus_queue.cc:260] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:33.724778  9031 raft_consensus.cc:3055] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:33.722427  4569 leader_election.cc:400] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11.
I20260709 12:26:33.725550  9031 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:33.725788  4569 leader_election.cc:403] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:33.726847  9031 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:33.727888  5346 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:33.728345  9031 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:33.729359  9215 consensus_meta.cc:287] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.678s	user 0.014s	sys 0.000s
I20260709 12:26:33.721230  9036 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:33.721911  4753 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:33.731598  5340 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:33.734635  4571 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:33.736892  9215 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.737190  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:33.738739  9175 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.738626  9215 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:33.739463  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:33.740049  9239 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:33.739331  9175 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:33.741312  4678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:26:33.742134  4678 raft_consensus.cc:606] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:33.742300  9175 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:33.742524  4678 raft_consensus.cc:614] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:33.742978  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:33.743703  4599 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:33.744334  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:33.743860  4674 raft_consensus.cc:2410] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 15 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:33.745225  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:33.745716  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:33.745815  4675 raft_consensus.cc:2376] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14. Re-sending same reply.
I20260709 12:26:33.746430  4605 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:33.742441  4602 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:33.717002  4719 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:33.741132  9215 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:33.755447  4680 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:33.749993  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:33.761888  9036 raft_consensus.cc:697] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:33.765676  4602 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:26:33.748966  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:33.703401  9224 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:33.754375  9040 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:33.766883  4602 pending_rounds.cc:85] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 7
I20260709 12:26:33.778410  8997 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:33.780243  4605 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:26:33.780974  4605 pending_rounds.cc:85] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 5
W20260709 12:26:33.781220  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.779309  8997 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:33.783733  8997 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:33.779304  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:33.786254  9315 log.cc:927] Time spent T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:26:33.786289  4749 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:33.790697  4599 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:33.791534  9277 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:33.793962  4647 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:33.798501  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 451428us
I20260709 12:26:33.798617  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:33.781279  4605 pending_rounds.cc:107] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:26:33.802206  4605 raft_consensus.cc:2889] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:33.800554  5314 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:33.774084  9224 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:33.776364  4602 pending_rounds.cc:107] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 13.8
W20260709 12:26:33.821247  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 99038.4us
I20260709 12:26:33.801705  9215 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:33.831674  4602 raft_consensus.cc:2917] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Aborting config change with OpId 13.8: Aborted: Op aborted by new leader
I20260709 12:26:33.832228  4602 raft_consensus.cc:2889] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:33.762820  9169 consensus_queue.cc:1175] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49277a7653524787a715232ab908bbc6" responder_term: 10 status { last_received { term: 10 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 10 index: 6 } }
W20260709 12:26:33.833833  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.805835  9277 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:33.819789  9224 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:33.819617  9175 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:33.834919  9264 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:33.848691  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 114218us
W20260709 12:26:33.850706  9316 log.cc:927] Time spent T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:26:33.851038  9270 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11.
I20260709 12:26:33.852968  4753 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:33.829804  9215 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.856601  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:33.855036  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 138215us
I20260709 12:26:33.818357  9040 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:33.868899  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 119383us
W20260709 12:26:33.868587  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 46307.1us
I20260709 12:26:33.874186  9186 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:26:33.879205  9198 consensus_meta.cc:287] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.519s	user 0.009s	sys 0.001s
I20260709 12:26:33.880000  9036 consensus_queue.cc:237] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:33.884953  9040 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:33.886526  8500 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.892114  8500 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:33.870426  4569 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:33.894629  9239 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:33.895529  9239 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:26:33.897518  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.855511  9175 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:33.904011  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.904588  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:33.866905  9215 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:33.924402  9198 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:33.939689  4571 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:33.941186  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:33.942991  9319 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:26:33.943631  9239 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:33.944036  9239 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:33.944540  9239 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:33.946029  9198 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:33.948309  9198 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:33.949501  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:33.950512  5314 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:33.952435  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:33.942170  8992 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:33.942780  9270 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:33.941973  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:33.934916  9316 log.cc:927] Time spent T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:26:33.927657  9175 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:33.954979  9270 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.908520  9181 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:26:34.010458  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:33.953467  8992 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:34.008526  4747 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:34.019164  4751 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:34.009119  9198 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:34.011452  9270 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:34.027117  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:34.026432  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:34.032827  4748 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:34.035121  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:34.036896  9031 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.050678  9031 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.052441  9254 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.037856  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:34.042268  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:34.045800  9277 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.046751  4751 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:34.058571  4755 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:26:34.060467  4747 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:34.066984  4752 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:34.066974  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.079176  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 48917.1us
I20260709 12:26:34.091279  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:34.084139  9263 consensus_meta.cc:287] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.838s	user 0.017s	sys 0.000s
I20260709 12:26:34.075969  9031 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:34.093811  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.091596  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:34.095973  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:34.099195  4756 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:34.088289  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:34.053496  9241 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.083806  9198 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.108255  4749 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:26:34.115267  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:34.116050  4753 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:34.086032  9277 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.100781  9031 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.136607  4672 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:34.119505  9324 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:26:34.120043  8992 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.127492  9239 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.129223  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:34.130059  4751 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:26:34.133559  4600 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:34.133839  4603 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:34.148495  9031 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.149219  9241 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:34.150759  9031 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:34.141809  4755 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
W20260709 12:26:34.151377  9243 consensus_meta.cc:287] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.741s	user 0.012s	sys 0.000s
I20260709 12:26:34.152006  9243 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:34.152642  9243 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.153074  9243 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:34.102296  9224 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.181254  4747 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:26:34.182089  4747 pending_rounds.cc:85] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 9
I20260709 12:26:34.209232  4747 pending_rounds.cc:107] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:26:34.209534  4747 raft_consensus.cc:2889] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:34.142490  8984 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:34.215493  4753 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:26:34.216887  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.117662  9198 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:34.234177  5340 consensus_meta.cc:287] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.005s
I20260709 12:26:34.235296  5340 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:26:34.235792  5340 pending_rounds.cc:85] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 7
I20260709 12:26:34.236163  5340 pending_rounds.cc:107] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 13.8
I20260709 12:26:34.236444  5340 raft_consensus.cc:2917] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Aborting config change with OpId 13.8: Aborted: Op aborted by new leader
I20260709 12:26:34.236972  5340 raft_consensus.cc:2889] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:34.238842  9236 consensus_meta.cc:287] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.524s	user 0.013s	sys 0.000s
I20260709 12:26:34.239501  9236 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.243578  9236 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:34.247409  4751 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:34.219600  9263 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.230005  9031 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:34.248409  8984 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:34.270287  4753 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:34.247792  9186 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:34.258329  9254 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:34.273955  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 525099us
I20260709 12:26:34.280400  4644 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:34.280593  9022 consensus_queue.cc:1059] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:26:34.286703  8500 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:34.290042  5342 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 6.
I20260709 12:26:34.271191  9031 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.293128  9031 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:34.297879  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 408479us
I20260709 12:26:34.299841  4747 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:34.304147  4756 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
W20260709 12:26:34.309772  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 95484.8us
I20260709 12:26:34.313943  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2423) took 1343 ms. Trace:
I20260709 12:26:34.314476  4600 rpcz_store.cc:276] 0709 12:26:32.970778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.970930 (+   152us) service_pool.cc:224] Handling call
0709 12:26:34.313753 (+1342823us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b54003539d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:34.313868 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2129875712}
I20260709 12:26:34.322765  4604 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:26:34.327773  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.318783  9030 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.279762  9263 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.325542  8500 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:34.339668  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 389229us
I20260709 12:26:34.349200  4719 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:34.352735  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:34.358304  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:34.367295  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 46140.9us
I20260709 12:26:34.345011  9239 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:34.382429  9241 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:34.385342  9330 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:34.386003  9198 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:34.344945  9030 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.347195  8984 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.342643  8500 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.378312  9277 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:34.428993  9205 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:26:34.431128  9330 raft_consensus.cc:697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:34.432406  9330 consensus_queue.cc:237] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.436276  9177 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:34.436877  9177 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:34.398070  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 77723.9us
W20260709 12:26:34.409896  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 48606.9us
W20260709 12:26:34.391558  9264 consensus_meta.cc:287] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.000s
I20260709 12:26:34.426225  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:34.351307  9224 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:34.427381  4749 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:26:34.428541  4748 raft_consensus.cc:2410] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:26:34.381606  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:34.395777  9336 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.395941  9335 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:26:34.446735  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 136031us
I20260709 12:26:34.390723  9263 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:34.442700  4753 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:34.410707  9175 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.389485  9215 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.452035  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:34.442893  9331 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.459265  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:34.460355  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:34.449971  9335 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.467551  5342 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 12: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:34.479609  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:34.483011  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:34.483078  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:34.483572  5346 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:34.482918  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:34.484629  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:34.486192  9264 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:34.486793  9264 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:34.487342  9215 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:34.487818  9264 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.481930  9335 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:34.494345  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 173079us
I20260709 12:26:34.460108  9224 consensus_queue.cc:1059] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:26:34.450603  9270 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.495503  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:34.501067  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 548741us
I20260709 12:26:34.507467  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:34.441285  9186 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.510509  4681 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:34.457387  9336 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.520931  4681 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:34.510078  9277 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:34.532063  4571 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:26:34.533936  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 75576.6us
I20260709 12:26:34.540082  4752 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:26:34.520578  9010 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:34.549867  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 93206us
I20260709 12:26:34.545285  4645 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:34.491670  9331 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.554222  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:34.557202  4753 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:26:34.558961  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 79551.9us
I20260709 12:26:34.551388  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:34.561688  4679 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:34.564940  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.541513  8992 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.544366  9239 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:34.560851  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:34.582686  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:34.572443  9239 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:34.582007  4569 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:34.554061  4681 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:34.590590  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 82102.8us
I20260709 12:26:34.591724  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:34.573314  9327 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.113s	user 0.003s	sys 0.000s
I20260709 12:26:34.550843  9198 consensus_queue.cc:1059] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:26:34.542177  9243 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:34.598320  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 58412.7us
I20260709 12:26:34.598029  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:34.600334  4752 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
W20260709 12:26:34.602245  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 91690.8us
I20260709 12:26:34.603335  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:34.586930  9239 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:34.588147  9277 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.589771  9236 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:34.588384  9177 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:34.590487  9030 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:34.593004  9254 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:34.635305  4571 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:34.560776  9344 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:26:34.634007  9353 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:26:34.638823  9030 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:34.639060  9342 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:26:34.641674  9345 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.610173  8500 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:34.643728  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 56129.4us
I20260709 12:26:34.637784  5314 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:34.590258  8992 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:34.709105  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 100799us
I20260709 12:26:34.658443  9207 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:26:34.715271  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 77597.5us
I20260709 12:26:34.718660  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:34.645239  9224 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:26:34.731953  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:34.741927  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 106713us
I20260709 12:26:34.610682  9277 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.725133  9177 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:26:34.753417  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.757820  9035 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.758313  9010 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:34.758685  9035 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.759397  9331 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:34.762091  9354 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:34.763034  9358 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:34.764561  9030 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:34.732805  9270 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:34.739490  9254 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:34.770488  4679 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:34.779888  9359 consensus_queue.cc:1059] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:26:34.781026  9357 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:34.743881  9207 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:34.780217  9355 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:26:34.724481  9344 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
W20260709 12:26:34.784427  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 239196us
W20260709 12:26:34.788352  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 79319.6us
I20260709 12:26:34.756609  9177 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:34.769989  9035 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:34.720361  9345 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:34.798888  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 75920.7us
I20260709 12:26:34.800457  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:34.806262  9365 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:26:34.806782  9277 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:34.807391  9364 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.794397  9177 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:34.807956  9364 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.813099  9343 raft_consensus.cc:3055] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:34.813755  9343 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:34.814996  9343 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:34.816764  9350 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:34.817513  9350 raft_consensus.cc:697] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:34.818554  9350 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.820379  9207 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.791379  9254 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:34.848105  4645 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:34.794329  9336 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:34.850941  8992 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:34.809366  9270 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.853869  9243 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:34.854167  9270 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.855759  9175 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:34.856261  9270 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:34.857683  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:34.858342  4752 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:34.866029  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:34.874045  4752 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:34.874378  4753 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:34.875741  5314 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:34.880982  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:34.881801  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 160633us
I20260709 12:26:34.884332  9243 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:34.892479  9352 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:34.892585  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:34.893482  9177 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:34.894363  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:34.894920  4750 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:26:34.900286  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:34.901786  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:34.813097  9364 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:34.884333  9343 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:34.908078  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47804us
I20260709 12:26:34.908842  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:34.781880  9357 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:34.905766  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57821.6us
I20260709 12:26:34.885398  9336 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:34.910605  4676 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:34.891321  9224 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:34.913093  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:34.905810  4673 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:34.903373  9243 raft_consensus.cc:697] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:34.864233  9236 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:26:34.913005  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:34.916672  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 69828.3us
I20260709 12:26:34.917076  4681 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:34.918053  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:34.919106  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:34.921795  4752 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:34.915125  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:34.912292  9368 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
W20260709 12:26:34.923595  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 123917us
I20260709 12:26:34.906261  9177 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:34.927227  9244 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:34.930353  9236 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:34.902405  8997 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:34.932231  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:34.932848  9270 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:34.973363  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 68335.8us
W20260709 12:26:34.974404  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 74204.7us
I20260709 12:26:34.907384  9352 raft_consensus.cc:697] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:34.897986  9354 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:26:35.007213  4752 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:34.974288  9236 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:35.016426  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 84270us
I20260709 12:26:35.018407  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:35.027658  4568 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:35.012635  9368 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:35.009954  9224 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:35.014590  8984 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:35.050654  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 140490us
I20260709 12:26:35.009029  9371 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:35.053457  9352 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.059242  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:35.064239  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:35.024452  9198 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:35.069329  4753 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:35.036456  9186 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.074405  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:35.075209  4752 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:26:35.079429  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.081106  9341 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.552s	user 0.005s	sys 0.000s
I20260709 12:26:35.081408  9367 consensus_queue.cc:1059] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.128s
I20260709 12:26:35.009069  9254 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:35.082170  4756 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:35.082613  9224 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:35.085913  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:35.055757  9336 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:35.088833  9335 consensus_meta.cc:287] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.002s
I20260709 12:26:34.997315  9244 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.037173  9031 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:26:35.089771  4753 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:35.056360  9263 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:35.087142  9357 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:35.096520  9356 log.cc:927] Time spent T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
I20260709 12:26:35.096246  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:35.097227  9366 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 12:26:35.097720  9326 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:26:35.044267  8992 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:26:35.072639  9371 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:35.102368  4748 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:35.012951  8997 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:35.109977  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 110280us
I20260709 12:26:35.068696  9368 raft_consensus.cc:697] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:35.094506  8984 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:35.028234  9333 consensus_queue.cc:1059] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:26:35.155759  4755 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:35.111433  4749 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:26:35.179131  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 80485.7us
I20260709 12:26:35.099290  9354 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:35.103696  9169 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:35.084525  9239 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:35.183282  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:35.270641  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:35.060740  9243 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.080184  9236 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:35.110098  9331 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 12:26:35.178858  9336 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:35.110025  9358 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:26:35.286059  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:35.229748  8500 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.258827  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:35.123409  9186 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:35.261885  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 227912us
W20260709 12:26:35.276021  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.280171  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.282161  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:35.151705  9335 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.287302  4747 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:35.309900  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:35.316417  4752 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:35.292644  4753 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:35.317104  4748 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 12: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:35.317700  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:35.289633  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:35.318316  5312 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:35.318667  4753 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:35.288784  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:35.320513  4749 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:35.317066  4756 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:35.336367  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:35.339663  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:35.340662  4600 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:35.342068  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:35.346408  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 58860.2us
W20260709 12:26:35.287900  9362 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.407s	user 0.001s	sys 0.000s
W20260709 12:26:35.312201  9376 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.336s	user 0.001s	sys 0.000s
I20260709 12:26:35.280717  9169 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:35.356518  9186 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:35.298125  8992 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.358040  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:35.361320  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:35.362717  4596 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:35.362663  4756 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:35.307585  8997 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:35.369807  9383 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEARNER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:35.370478  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:35.371253  4604 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:35.369702  9334 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
W20260709 12:26:35.372076  9372 log.cc:927] Time spent T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.452s	user 0.001s	sys 0.000s
I20260709 12:26:35.311865  9239 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:35.363626  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:35.378764  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.365280  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:35.366698  4601 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:26:35.309266  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:35.369110  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:35.369668  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:35.323638  9198 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.367354  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:35.348685  8500 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:35.437182  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:35.447796  4600 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:35.409260  9370 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.426790  4603 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:35.402361  4748 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:26:35.396616  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:35.451566  4756 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:35.274137  9354 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.295624  9368 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:35.418908  4601 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:35.442447  4596 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:35.294487  9371 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:35.380406  9030 consensus_meta.cc:287] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.615s	user 0.006s	sys 0.007s
W20260709 12:26:35.444657  4750 consensus_meta.cc:287] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.549s	user 0.008s	sys 0.003s
I20260709 12:26:35.409714  9263 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.446938  4599 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:35.407943  4571 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:35.457875  4750 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:35.458190  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:35.458427  9030 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:35.408207  8997 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.460001  9354 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:35.445518  4681 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
W20260709 12:26:35.463717  9373 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.288s	user 0.002s	sys 0.000s
I20260709 12:26:35.411401  9335 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:35.466554  5314 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:35.466583  4754 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:26:35.405867  9350 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.470126  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:35.380761  8992 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:35.494771  5314 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:35.473650  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 65819.4us
I20260709 12:26:35.449227  9330 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:35.476671  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 107670us
I20260709 12:26:35.491770  4752 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:35.477208  4681 pending_rounds.cc:85] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 8
I20260709 12:26:35.507035  4681 pending_rounds.cc:107] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:26:35.507306  4681 raft_consensus.cc:2889] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:35.507876  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:35.507951  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3013) took 1042 ms. Trace:
I20260709 12:26:35.508380  4681 rpcz_store.cc:276] 0709 12:26:34.465813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.465945 (+   132us) service_pool.cc:224] Handling call
0709 12:26:34.495498 (+ 29553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.911256 (+415758us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b5800927bd0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4120aa 00007effee494447 00007effee4acfe7 00007effee4ae7de 00007effee4b8b49 00007effee4b9416 00007effee4b5d8f 00007effee4b583e 00007efff347a419 00007effec9f1bd4
0709 12:26:35.507707 (+596451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.507881 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7173504}
I20260709 12:26:35.494059  4756 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:35.412571  9169 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:35.498571  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:35.500962  4600 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:35.481102  9263 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.503175  4747 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:35.503888  9333 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:35.504765  9379 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 12:26:35.532627  4751 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:35.410023  9336 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:35.541684  4750 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:35.456676  9198 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:35.509119  9334 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:35.543830  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 45354.5us
I20260709 12:26:35.580660  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:35.547127  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:35.553893  4754 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:35.554324  9337 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:26:35.555539  4747 raft_consensus.cc:3055] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:35.555913  8992 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:35.583770  4747 raft_consensus.cc:740] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:35.556907  9394 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:35.566515  9390 log.cc:927] Time spent T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:26:35.493237  8997 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.576452  9263 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:35.457495  9370 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:35.587291  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 79461.4us
I20260709 12:26:35.587924  9347 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:35.593289  4571 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:35.596305  8992 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:35.597607  8500 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:35.542618  9207 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:35.608042  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:35.598152  9381 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:35.496346  9350 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.616994  4754 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:26:35.584689  4747 consensus_queue.cc:260] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:35.608981  8992 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.611320  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:35.641904  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:35.643555  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:35.648185  4571 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:35.612818  9036 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:35.651042  4599 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:26:35.647248  9394 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:35.609637  9334 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:35.612775  9030 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:35.649490  4571 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:35.614136  9254 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:35.680857  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:35.610144  9347 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:35.683385  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:35.686578  4747 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:35.687125  4754 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:26:35.644476  8992 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:35.687507  4754 pending_rounds.cc:85] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 6
I20260709 12:26:35.685091  4672 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:35.636170  9186 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:35.651113  8500 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:35.690270  4754 pending_rounds.cc:107] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:26:35.694790  4754 raft_consensus.cc:2889] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:35.692320  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 161836us
I20260709 12:26:35.694723  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:35.693823  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:35.637069  9207 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:35.664984  9263 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:35.676330  9169 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:35.673118  9036 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:35.659520  9333 raft_consensus.cc:697] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:35.674383  9337 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:35.697014  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:35.690382  9239 raft_consensus.cc:2764] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:26:35.697964  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 48548.5us
I20260709 12:26:35.696336  4674 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:35.703212  4678 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:26:35.710119  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.684370  9254 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.723914  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:35.729387  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:35.730150  4680 raft_consensus.cc:2393] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:35.724238  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:35.726248  4676 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:35.725901  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:35.697124  5341 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:26:35.730836  4571 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:35.732326  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:35.694607  8500 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.746950  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:35.710042  9169 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.725868  9337 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.762673  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:35.726523  9186 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:35.760968  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:35.761790  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.764464  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 125619us
I20260709 12:26:35.734917  9254 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:35.742846  8500 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:35.770112  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:35.769369  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 87226.9us
I20260709 12:26:35.774147  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:35.775605  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:35.761725  9169 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:35.727083  9394 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:26:35.765273  9337 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:35.770215  9239 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:35.797863  9407 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 12:26:35.799568  9396 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 12:26:35.803900  9345 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:35.805604  9394 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:35.805593  4749 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:35.806888  9394 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:35.805734  9336 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:35.808292  9336 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:35.811182  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.812414  9336 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:35.778102  9347 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:35.815894  9347 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:35.814178  9405 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:35.821640  4677 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:35.828697  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 140224us
I20260709 12:26:35.829358  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:35.830659  9030 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:35.856693  4569 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:35.832419  9368 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.858006  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:35.861790  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:35.863057  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:35.867771  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 139521us
I20260709 12:26:35.781775  9382 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.226s
I20260709 12:26:35.859437  4647 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:35.870493  4679 raft_consensus.cc:2410] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 12:26:35.853370  9403 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:26:35.764678  9334 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.903739  4756 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:26:35.880693  9186 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.841724  9345 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:26:35.877995  9198 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:35.934149  9239 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:35.934696  9239 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.935101  9338 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.878223  9368 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.935637  9338 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.937260  9368 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:35.927924  9397 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
W20260709 12:26:35.939823  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 164289us
I20260709 12:26:35.927649  9409 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:35.928318  9350 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:35.930614  4680 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:26:35.932858  9406 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 12:26:35.933736  9408 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:26:35.883808  9365 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:35.854358  9036 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:35.840142  9207 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.843304  9333 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:35.867585  9344 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:26:35.935142  9239 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:35.953222  4571 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:26:35.960744  9367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.029s	sys 0.044s
I20260709 12:26:35.938827  9335 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:26:35.958746  9394 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:35.952488  9338 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:35.968772  9036 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:35.976958  9367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.029s	sys 0.045s
I20260709 12:26:35.967475  9365 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:35.992089  9244 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:35.969144  9353 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:35.960551  9350 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:26:35.978899  9394 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:36.003886  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:36.007684  9378 log.cc:927] Time spent T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.788s	user 0.002s	sys 0.000s
I20260709 12:26:36.009687  9010 raft_consensus.cc:3055] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:36.010192  9010 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:36.011191  9010 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:36.012326  9010 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:35.998719  9175 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:36.017889  9175 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:36.018069  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:36.018769  9175 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:36.019760  4681 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:26:36.020171  4681 pending_rounds.cc:85] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 6
I20260709 12:26:36.020390  4681 pending_rounds.cc:107] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:26:36.020633  4681 raft_consensus.cc:2889] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:36.030246  9401 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:26:35.994055  9345 raft_consensus.cc:3055] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:36.031487  9345 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:36.032383  9345 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:36.035079  9416 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:26:36.016961  8992 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.049310  5341 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 12:26:36.047978  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 763933us
I20260709 12:26:36.019788  9175 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:36.052865  4598 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:26:36.055210  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:36.044606  9030 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:36.051570  9345 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:36.059749  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:36.060010  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:36.059665  9415 raft_consensus.cc:3055] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:36.003507  9353 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.074765  9415 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:36.074616  9425 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:36.051997  9419 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 12:26:36.096082  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 44317.6us
I20260709 12:26:36.085892  9224 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:36.075067  9421 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:36.103471  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 52043us
W20260709 12:26:36.103998  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.075340  9331 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:26:36.112722  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:36.120837  9377 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 1.143s	user 0.001s	sys 0.002s
I20260709 12:26:36.121603  9421 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:36.123328  5342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 620) took 1254 ms. Trace:
I20260709 12:26:36.122980  9421 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.123766  5342 rpcz_store.cc:276] 0709 12:26:34.869359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.869611 (+   252us) service_pool.cc:224] Handling call
0709 12:26:34.885387 (+ 15776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:34.894629 (+  9242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:34.894669 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:34.895185 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 12:26:34.978159 (+ 82974us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:34.978197 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.978262 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:36.120159 (+1141897us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:36.122989 (+  2830us) raft_consensus.cc:1685] finished
0709 12:26:36.123014 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:36.123240 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9403,"raft.queue_time_us":12307,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":4910,"spinlock_wait_cycles":6063360,"thread_start_us":82301,"threads_started":1,"wal-append.queue_time_us":81507}
I20260709 12:26:36.126663  4676 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:36.130045  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 176913us
I20260709 12:26:36.139456  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:36.142318  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:36.145918  4569 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:36.155692  9263 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:36.156124  9430 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:26:36.156670  9263 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:36.157692  9036 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:36.158093  9036 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:36.158922  9036 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.160221  9036 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:36.160710  9239 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.162714  9353 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:36.143621  9416 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:36.131296  9415 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:36.163560  9416 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:36.163959  9416 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:36.164883  9367 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:36.166349  9254 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.167366  9411 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:36.168020  9344 raft_consensus.cc:1064] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
W20260709 12:26:36.168468  9344 raft_consensus.cc:1068] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: Unable to promote non-voter dbe960c35c84451faf4e5b5c9233fa39: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:26:36.168748  9254 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:36.133873  8992 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:36.085582  9365 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:36.181720  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 46472.5us
I20260709 12:26:36.108894  9169 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:36.187142  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.212772  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:36.186442  9427 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:26:36.222996  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:36.228858  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:36.229338  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 89965.9us
I20260709 12:26:36.186429  8992 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:36.232748  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 46099.5us
I20260709 12:26:36.225615  9425 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:36.225677  9353 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:36.241371  8992 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.242075  9409 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.242621  9409 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.227993  8500 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.247253  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:36.249198  8500 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:36.258183  9425 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:36.259467  9369 log.cc:927] Time spent T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.459s	user 0.001s	sys 0.002s
I20260709 12:26:36.250214  9030 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:26:36.261080  9030 raft_consensus.cc:3055] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:36.261161  9239 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:36.261487  9370 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:36.263290  4681 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 12:26:36.266121  4681 pending_rounds.cc:85] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 5
I20260709 12:26:36.266445  4681 pending_rounds.cc:107] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 12:26:36.266738  4681 raft_consensus.cc:2889] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:36.269328  8992 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:36.261554  9030 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:36.270659  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:36.271394  4675 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:36.257297  9337 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:36.274573  9030 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.275264  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:36.278278  9353 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:36.280346  9244 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:36.280818  9337 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:36.281755  9425 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:36.278134  9345 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:36.282581  9345 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.281317  9353 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.298069  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:36.308495  9352 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.007s	sys 0.007s
W20260709 12:26:36.308962  9352 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.007s	sys 0.007s
I20260709 12:26:36.310101  4645 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:36.281277  9244 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:36.313046  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:36.320799  4679 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:36.322094  8992 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:26:36.331277  9418 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:36.312726  9244 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:36.332260  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:36.310410  9370 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.310932  9030 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:36.341066  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:36.347359  4645 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:36.352569  9244 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.356241  9244 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:36.364748  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:36.371230  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:36.394068  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.346378  9425 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:36.399539  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 52313.1us
I20260709 12:26:36.404510  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:36.339843  9418 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:26:36.350852  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:36.343087  9370 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.363580  9347 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:36.431298  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:36.414054  9365 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.016s
W20260709 12:26:36.436558  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:36.437484  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.416074  9224 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:36.442495  9357 raft_consensus.cc:2764] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:36.467345  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:36.470046  9425 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:36.452116  5340 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 6.
I20260709 12:26:36.412166  9355 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:26:36.430910  9344 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:26:36.429445  9335 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:26:36.470567  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:36.460319  9347 raft_consensus.cc:697] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:36.464172  9224 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:36.488952  5343 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:36.559325  9370 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:36.567317  4675 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
W20260709 12:26:36.620464  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.489984  9425 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:36.575251  9224 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:36.454166  9186 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
W20260709 12:26:36.565667  9445 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:26:36.462357  9416 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:36.683804  9357 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:36.684316  9446 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
W20260709 12:26:36.685511  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 215068us
W20260709 12:26:36.661042  9436 log.cc:927] Time spent T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:26:36.693286  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.693112  4569 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:36.694533  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.697362  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:36.698082  4675 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:36.700253  4569 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:36.733482  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:36.733999  4675 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:26:36.678959  9330 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:36.736164  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 278698us
I20260709 12:26:36.751046  4645 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:36.702342  9431 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:36.742667  9443 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:36.740399  8500 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:36.766263  9447 log.cc:927] Time spent T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:26:36.767441  9448 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:26:36.769836  9416 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:36.693930  9357 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:36.738855  9330 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:36.772938  8992 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:36.774061  9031 raft_consensus.cc:3055] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:36.756151  9347 consensus_queue.cc:237] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.764981  9030 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:36.689057  9425 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:36.742882  9421 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.047s
W20260709 12:26:36.796156  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 56925.2us
W20260709 12:26:36.805801  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 71920.6us
I20260709 12:26:36.786916  8500 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.809232  9453 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:36.818775  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.797116  9031 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:36.797435  9330 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:36.831684  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:36.807776  9416 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.810539  8500 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:36.830462  9453 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:36.849033  9357 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:36.849445  4755 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:36.849647  9357 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.850121  9357 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:36.852871  9357 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:36.854916  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:36.855544  4597 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:36.839411  9416 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.859392  9416 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:36.839919  9030 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.860090  9030 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:36.862260  4597 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
W20260709 12:26:36.869765  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 97174.4us
W20260709 12:26:36.871181  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 65055.7us
I20260709 12:26:36.870699  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:36.874605  4753 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:26:36.883549  4605 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:36.888268  4598 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:36.901857  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:36.903406  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:36.908598  4602 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:36.914026  9030 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:36.845324  9031 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:36.915306  9244 consensus_meta.cc:287] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.558s	user 0.000s	sys 0.013s
I20260709 12:26:36.916095  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:36.923765  4674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:26:36.924427  4674 raft_consensus.cc:606] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:36.924767  4674 raft_consensus.cc:614] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:36.853499  9357 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:36.928980  4674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:26:36.929593  4674 raft_consensus.cc:606] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:36.929951  4674 raft_consensus.cc:614] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:36.863363  9453 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:36.935050  9425 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:36.941252  4674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:26:36.942031  4674 raft_consensus.cc:606] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:36.942513  4674 raft_consensus.cc:614] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:36.944033  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:36.949149  4602 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:36.954532  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:36.962174  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:36.963176  4672 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
W20260709 12:26:36.963841  4672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48198 (request call id 718) took 1272 ms (client timeout 1602 ms). Trace:
I20260709 12:26:36.963434  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:36.965018  4604 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:36.967833  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:36.968108  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:36.891105  9456 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:26:36.969962  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:36.989948  4598 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:36.983956  4568 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:36.999922  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:36.964432  4672 rpcz_store.cc:269] 0709 12:26:35.691362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.692652 (+  1290us) service_pool.cc:224] Handling call
0709 12:26:36.961952 (+1269300us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b58000d4798. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:26:36.963734 (+  1782us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5463424}
I20260709 12:26:37.025280  4645 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:37.038326  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.028223  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.043372  4755 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:37.044350  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:37.044943  4755 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:37.047921  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:37.050716  4645 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:37.051193  4602 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:26:37.023859  9448 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.090s	user 0.000s	sys 0.004s
I20260709 12:26:37.058598  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.063673  4598 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:37.063978  4754 raft_consensus.cc:2410] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:37.063853  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.060463  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:37.064554  4749 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:37.064036  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.061266  8500 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:37.040828  9458 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.112s	user 0.000s	sys 0.004s
W20260709 12:26:37.041049  9464 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.114s	user 0.004s	sys 0.000s
I20260709 12:26:37.068379  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.026337  8992 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:37.080447  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.081496  4748 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:37.065608  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:37.085031  4749 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:37.050217  9457 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.242s	user 0.004s	sys 0.001s
I20260709 12:26:37.078615  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:37.085867  4754 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.089771  9425 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.090631  9425 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.093787  9425 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:37.000932  9338 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.094758  9338 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:37.112977  9338 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:37.115013  4754 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:37.115588  4747 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:37.117877  8500 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:37.118850  9354 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:37.119326  9454 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:26:37.120142  4754 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:26:37.050308  9022 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:37.121354  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.121490  9365 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:37.122290  4754 raft_consensus.cc:2410] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:26:37.122715  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.123574  4753 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:37.049408  9455 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:37.129670  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.127200  4749 raft_consensus.cc:2410] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:37.021895  9335 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:26:37.131222  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:37.066689  9344 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:37.046257  9456 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 12:26:37.137295  4747 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: [8.5-11.6]
I20260709 12:26:37.157004  4749 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:37.116907  8992 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.168321  4598 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 12:26:37.173141  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.173655  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:37.078507  9355 consensus_queue.cc:1059] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 12:26:37.171764  8992 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:37.187318  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.200299  4749 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:37.177241  9456 consensus_queue.cc:1243] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:26:37.221283  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.247651  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:37.248785  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 196614us
I20260709 12:26:37.252533  5346 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:37.253153  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.251986  4571 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:37.256386  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.258451  4571 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:37.258714  4598 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:37.263185  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:37.259670  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:37.270048  4747 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 12: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:37.273411  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:37.266201  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:26:37.257706  9449 log.cc:927] Time spent T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.253s	user 0.000s	sys 0.005s
I20260709 12:26:37.278079  4571 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:37.251049  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:37.280047  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 450293us
W20260709 12:26:37.268180  9450 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.332s	user 0.002s	sys 0.002s
I20260709 12:26:37.202633  9465 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 12:26:37.269100  9338 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:37.267690  9198 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:37.311395  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 243551us
I20260709 12:26:37.305028  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:37.203126  9468 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:26:37.288174  9381 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:37.310827  9244 raft_consensus.cc:3055] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:37.312419  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.328013  4747 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
W20260709 12:26:37.312024  9472 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:26:37.322047  9409 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:37.318331  9425 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:37.319478  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:37.309402  9169 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:37.290994  9462 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:37.339831  4755 raft_consensus.cc:2410] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 12:26:37.348992  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 63888.7us
I20260709 12:26:37.307415  9338 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:37.294582  9330 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:37.385999  9471 log.cc:927] Time spent T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:26:37.383193  9338 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:37.387465  9236 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.389798  9463 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 12:26:37.391954  9473 log.cc:927] Time spent T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 12:26:37.393361  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 73956.4us
I20260709 12:26:37.342406  9454 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:26:37.394079  4756 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:37.394210  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:37.394531  9454 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:37.394912  9454 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:37.395267  9357 raft_consensus.cc:2764] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:26:37.395779  9454 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:37.364423  9462 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.397025  9462 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:37.359591  9466 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:37.403458  9347 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:37.403613  8500 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:37.404045  9347 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:37.386018  9470 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:26:37.411490  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.409174  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:37.414292  9463 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:37.414746  9463 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.415170  9463 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:37.415721  5346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 628) took 1649 ms. Trace:
I20260709 12:26:37.416133  5346 rpcz_store.cc:276] 0709 12:26:35.766076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.766265 (+   189us) service_pool.cc:224] Handling call
0709 12:26:35.772627 (+  6362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:37.255042 (+1482415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.415468 (+160426us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54007dc154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:37.415656 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3402887936}
I20260709 12:26:37.327435  9244 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:37.418054  4755 raft_consensus.cc:2393] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 12: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:37.417972  9394 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.418241  9244 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.419148  9394 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.419341  9244 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:37.396255  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:37.322949  9474 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:37.430506  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:37.432381  4647 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:37.433919  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:37.341001  9381 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:37.438591  4750 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:37.441293  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:37.344041  9456 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 12:26:37.462422  9022 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:37.466622  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:37.466790  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2461) took 2368 ms. Trace:
I20260709 12:26:37.469125  4571 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:37.471158  9477 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:37.463646  9355 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:37.472108  9030 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.472190  9355 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.472730  9355 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.473371  9169 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:37.473538  9347 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:37.473995  9169 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:37.475028  9169 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.428823  8997 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:37.477877  8997 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:37.467703  4751 rpcz_store.cc:276] 0709 12:26:35.091374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.450541 (+359167us) service_pool.cc:224] Handling call
0709 12:26:35.532271 (+ 81730us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:35.539187 (+  6916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.459308 (+1920121us) spinlock_profiling.cc:243] Waited 60 ms on lock 0x7b5400118350. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:37.459483 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137913728}
I20260709 12:26:37.480654  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:37.481297  4751 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
W20260709 12:26:37.481966  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 85806.3us
I20260709 12:26:37.482280  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:37.435134  9474 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:37.482657  9453 raft_consensus.cc:3055] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:37.482869  4751 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:37.483145  9453 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:37.484714  4571 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:37.482779  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:37.456655  8984 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:37.461812  9371 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:37.482950  9474 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:37.502558  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 182256us
I20260709 12:26:37.490999  9270 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:37.505862  4602 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:37.519434  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:37.521227  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:37.503297  9465 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:37.518532  9224 consensus_meta.cc:287] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.823s	user 0.011s	sys 0.001s
I20260709 12:26:37.537055  4602 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:37.538651  4602 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:37.524786  9270 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.541945  4602 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:37.496384  9338 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.561918  4602 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:37.534098  8984 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:37.538272  9022 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:37.619892  9431 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.021s	sys 0.022s
I20260709 12:26:37.538343  8992 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:37.668529  9431 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.021s	sys 0.022s
I20260709 12:26:37.527280  9453 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.546648  9347 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.127s
I20260709 12:26:37.676419  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:37.544567  9224 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:37.684408  9409 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.701272  9409 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:37.702621  9224 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:37.654915  9371 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:37.703574  9409 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:37.704002  9371 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:37.704612  9482 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
W20260709 12:26:37.706226  9485 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 12:26:37.707226  9371 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:37.546753  9270 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.694428  9431 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:37.711685  9431 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.546197  9425 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:37.711997  4756 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:37.669478  9357 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:37.543612  9381 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.692379  9330 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:37.565685  9466 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:37.782536  9453 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:37.786098  9484 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:26:37.781777  9465 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:37.782788  9371 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:37.790230  9425 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:37.835464  8992 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:37.655807  8984 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:37.768131  9022 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:37.850334  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 193507us
I20260709 12:26:37.843535  4751 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
W20260709 12:26:37.836382  9451 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.081s	user 0.000s	sys 0.004s
I20260709 12:26:37.859257  4748 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: [6.5-9.6]
I20260709 12:26:37.792922  9357 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.849622  9030 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:37.872406  4751 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:26:37.871224  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:37.874823  4751 pending_rounds.cc:85] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 5
I20260709 12:26:37.792384  9462 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:37.876022  4680 raft_consensus.cc:2410] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:26:37.875208  4751 pending_rounds.cc:107] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:26:37.878271  4751 raft_consensus.cc:2889] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:37.886044  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:37.843070  9371 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:37.871922  8992 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.849514  9463 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.915710  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 718) took 2441 ms. Trace:
I20260709 12:26:37.916217  4599 rpcz_store.cc:276] 0709 12:26:35.472739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.472946 (+   207us) service_pool.cc:224] Handling call
0709 12:26:35.481695 (+  8749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:35.674233 (+192538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.913339 (+2239106us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400352fd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:37.913585 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388642048}
W20260709 12:26:37.902805  9236 consensus_meta.cc:287] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.000s
I20260709 12:26:37.895387  9368 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:37.915321  4720 connection.cc:256] RPC call timeout handler was delayed by -2.9669735019997461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
I20260709 12:26:37.899896  9330 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.900223  9338 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:37.921698  4720 connection.cc:256] RPC call timeout handler was delayed by -2.6375735019996682s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1875s for a response
I20260709 12:26:37.904170  9030 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.925868  4747 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:26:37.925807  4720 connection.cc:256] RPC call timeout handler was delayed by -2.3999735019997388s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
I20260709 12:26:37.927788  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:37.923323  9416 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.932160  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:37.939849  4747 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:37.943387  4747 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:37.911424  8992 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.925953  9030 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:37.940992  9465 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:37.968246  9499 raft_consensus.cc:2764] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 12:26:37.968451  9357 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:37.926838  9368 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.929513  9330 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.918635  9425 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:37.934949  9416 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:37.981895  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49818.6us
I20260709 12:26:37.984565  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:37.976718  9244 consensus_meta.cc:287] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.557s	user 0.004s	sys 0.009s
I20260709 12:26:37.987773  4751 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:37.993306  4676 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:26:37.968451  9030 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:38.000819  4601 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:38.014219  8992 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.015130  9010 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.015904  9330 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:38.015658  9010 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:38.004336  9022 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:38.020274  8992 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.020915  8992 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:38.022011  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.022663  9411 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.023113  9411 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:38.023914  8992 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:38.025125  9411 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:38.032778  9454 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.999228  8984 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:38.035300  4749 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:38.005287  9512 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:38.041141  4599 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:26:38.045706  9501 log.cc:927] Time spent T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 12:26:38.049902  9503 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:26:38.050083  4596 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:38.054747  9513 raft_consensus.cc:670] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } [suppressed 2 similar messages]
I20260709 12:26:38.054949  9270 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.056394  9371 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:38.013018  9462 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.060649  9412 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:38.036482  9411 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.069972  9411 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.072360  9411 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:38.073216  9474 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.570s	user 0.003s	sys 0.008s
I20260709 12:26:38.074025  9270 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.073834  9474 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:38.078433  9394 consensus_meta.cc:287] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.659s	user 0.014s	sys 0.000s
I20260709 12:26:38.057781  9463 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.086021  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.093328  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:38.081596  9010 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.049036  9512 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:38.098577  5312 connection.cc:256] RPC call timeout handler was delayed by -1.1988235140001962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
I20260709 12:26:38.119220  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:38.305852  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.306195  4599 raft_consensus.cc:2393] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:38.307261  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:38.158660  9243 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.700s	user 0.000s	sys 0.019s
W20260709 12:26:38.308089  9243 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.849s	user 0.000s	sys 0.019s
I20260709 12:26:38.119004  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.089896  9368 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.308996  4596 raft_consensus.cc:2368] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12.
I20260709 12:26:38.178762  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.310451  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.311525  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:37.298754  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:38.108397  9365 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:38.180074  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:38.157519  9355 consensus_meta.cc:287] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.684s	user 0.015s	sys 0.004s
I20260709 12:26:38.181738  9344 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:38.182906  9186 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:38.157879  9499 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:38.341950  9186 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.344286  9186 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.102470  4755 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:38.117987  4597 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:38.191035  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.346251  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.346715  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:38.347064  9368 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:38.347517  4597 raft_consensus.cc:2393] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:38.347790  8984 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:38.348342  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.119347  9370 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.195814  4600 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:38.349362  9370 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:38.349851  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.350607  4596 raft_consensus.cc:2393] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:38.201293  4571 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:38.351402  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.351603  9344 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.352068  9264 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:38.352676  9264 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:38.202018  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 804) took 1356 ms. Trace:
I20260709 12:26:38.322620  9365 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:38.204109  9452 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:38.353643  9264 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.353858  9452 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.208724  9358 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:38.305230  4497 catalog_manager.cc:5697] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.355604  9358 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.217526  9477 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:38.228888  9345 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.357229  9477 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:38.357613  9345 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:38.358103  9345 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.358134  9477 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.359285  9477 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:38.219769  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2465) took 1252 ms. Trace:
I20260709 12:26:38.351403  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2456) took 1464 ms. Trace:
I20260709 12:26:38.360771  4601 rpcz_store.cc:276] 0709 12:26:36.967713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.967828 (+   115us) service_pool.cc:224] Handling call
0709 12:26:38.219586 (+1251758us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400233cd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:38.219708 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2373432704}
I20260709 12:26:38.362914  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.363574  4601 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:38.364583  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.222915  9416 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.231769  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.237425  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.240871  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.242686  9423 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.242870  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:38.245011  9454 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:38.251194  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:38.252411  9439 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:38.255092  9494 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.410s	user 0.000s	sys 0.003s
I20260709 12:26:38.260051  9354 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:38.262753  9331 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:38.269359  9455 raft_consensus.cc:3055] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:38.271852  9474 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.273929  9415 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:38.275137  9031 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:38.277595  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.278002  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.280639  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.281782  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:38.282711  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.283041  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.404685  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.290228  5343 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 632) took 2250 ms (client timeout 2837 ms). Trace:
I20260709 12:26:38.312507  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:26:38.312978  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 49378.8us
I20260709 12:26:38.314038  4720 leader_election.cc:400] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12.
I20260709 12:26:38.409104  4720 leader_election.cc:403] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:38.327812  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.352217  4596 raft_consensus.cc:2410] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:38.389245  9416 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:38.410701  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.411441  9456 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:38.412222  4596 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:26:38.411935  9456 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.412910  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.375461  9368 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:38.360949  4600 rpcz_store.cc:276] 0709 12:26:36.886731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.886873 (+   142us) service_pool.cc:224] Handling call
0709 12:26:38.351235 (+1464362us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400300a54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:38.351340 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2792050560}
I20260709 12:26:38.409711  4602 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:38.417418  4600 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:38.413465  4596 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:38.239974  9224 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:38.240342  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.240018  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.451671  5342 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:38.370922  4605 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:38.371377  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 170165us
I20260709 12:26:38.365459  4601 raft_consensus.cc:2410] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 13 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:26:38.353124  4750 rpcz_store.cc:276] 0709 12:26:36.845311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.845431 (+   120us) service_pool.cc:224] Handling call
0709 12:26:36.889337 (+ 43906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:37.439242 (+549905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.122133 (+682891us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54000550d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:38.201941 (+ 79808us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1262569344}
I20260709 12:26:38.455300  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.455546  5341 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:38.455931  4605 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:38.387127  4755 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:26:38.390372  4597 raft_consensus.cc:1240] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:38.395275  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:38.397392  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:38.399530  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:38.458400  4680 raft_consensus.cc:2410] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:26:38.400024  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.410444  9416 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:38.413677  4604 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:38.382325  9418 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:38.382515  9519 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.397631  9036 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:38.354151  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.230028  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3034) took 2359 ms. Trace:
I20260709 12:26:38.468336  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.348359  8984 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:38.418480  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 106052us
I20260709 12:26:38.419467  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:38.408624  5343 rpcz_store.cc:269] 0709 12:26:36.039270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.056438 (+ 17168us) service_pool.cc:224] Handling call
0709 12:26:38.290046 (+2233608us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b5400712354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:38.290149 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966073088}
I20260709 12:26:38.390862  9454 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.423620  4602 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:38.424381  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:38.391966  9455 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:38.426540  4600 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:38.474283  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:38.449396  9499 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:38.382870  9355 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:38.462419  9466 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.459883  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:38.462827  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.464205  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:38.416229  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 102368us
W20260709 12:26:38.470144  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 52452.6us
I20260709 12:26:38.467492  4679 rpcz_store.cc:276] 0709 12:26:35.870284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.958927 (+ 88643us) service_pool.cc:224] Handling call
0709 12:26:35.968338 (+  9411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:36.321425 (+353087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.229748 (+1908323us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b540038e5d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:38.229954 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810024832}
I20260709 12:26:38.471556  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:38.471720  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.474973  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:38.429862  4569 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.624s (SENT) [suppressed 28 similar messages]
I20260709 12:26:38.478339  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:38.452706  9337 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:26:38.446652  4498 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52818 (request call id 303) took 32919 ms (client timeout 15000 ms). Trace:
I20260709 12:26:38.451673  9353 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.538069  9345 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.537882  9353 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.540128  9353 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.483872  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.485282  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.541029  9454 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.467314  9462 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:26:38.542963  5342 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:38.475623  9036 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:38.546001  9036 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:38.549369  9368 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:38.486744  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:38.554471  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 66955.5us
I20260709 12:26:38.556181  9416 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:38.556813  9522 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:38.400596  9423 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.554499  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.560698  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.448077  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.464730  9425 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.393317  9439 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.528750  9455 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.529240  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.529966  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.530081  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:38.567417  4756 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:38.530238  9355 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:38.530889  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:38.489293  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.568449  4678 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:38.531937  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:38.527869  4676 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:38.570453  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:38.571069  4676 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:38.571843  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:38.532450  8984 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.533967  9512 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.488678  9488 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:26:38.536145  9337 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:38.525924  9035 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:26:38.530920  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:38.527416  4498 rpcz_store.cc:269] 0709 12:26:05.575359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.575749 (+   390us) service_pool.cc:224] Handling call
0709 12:26:38.446380 (+32870631us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:32.309041 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:32.322271 (+ 13230us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:32.322309 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:26:32.322329 (+    20us) tablet.cc:662] Decoding operations
0709 12:26:32.424951 (+102622us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:32.432649 (+  7698us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:32.434828 (+  2179us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:26:32.496138 (+ 61310us) tablet.cc:701] Row locks acquired
0709 12:26:32.496158 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:26:32.496299 (+   141us) write_op.cc:270] Start()
0709 12:26:32.496427 (+   128us) write_op.cc:276] Timestamp: P: 1783599992496250 usec, L: 0
0709 12:26:32.496450 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:26:32.547631 (+ 51181us) log.cc:844] Serialized 5581 byte log entry
0709 12:26:33.256505 (+708874us) op_driver.cc:464] REPLICATION: finished
0709 12:26:33.938573 (+682068us) write_op.cc:301] APPLY: starting
0709 12:26:33.938736 (+   163us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:33.951400 (+ 12664us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:33.951573 (+   173us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:34.056595 (+105022us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:34.057032 (+   437us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=0,mrs_lookups=0
0709 12:26:34.057080 (+    48us) write_op.cc:312] APPLY: finished
0709 12:26:34.076958 (+ 19878us) log.cc:844] Serialized 129 byte log entry
0709 12:26:34.443553 (+366595us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:34.485986 (+ 42433us) write_op.cc:454] Released schema lock
0709 12:26:34.486059 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:34.549439 (+ 63380us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b1800366a60. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c9522 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:26:34.549589 (+   150us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:34.549628 (+    39us) write_op.cc:454] Released schema lock
0709 12:26:35.080495 (+530867us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
Metrics: {"mutex_wait_us":162311,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":675569,"apply.run_cpu_time_us":54152,"apply.run_wall_time_us":554389,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62563,"num_ops":14,"prepare.queue_time_us":189842,"prepare.run_cpu_time_us":14625,"prepare.run_wall_time_us":406389,"raft.queue_time_us":448361,"raft.run_cpu_time_us":52857,"raft.run_wall_time_us":451012,"replication_time_us":759747,"spinlock_wait_cycles":16603136,"thread_start_us":1181385,"threads_started":5,"wal-append.queue_time_us":543782,"wal-append.run_cpu_time_us":62809,"wal-append.run_wall_time_us":3018822}]]}
I20260709 12:26:38.563803  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:38.564442  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.572079  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.356602  4497 catalog_manager.cc:5697] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.568972  5343 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:38.603017  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:38.474099  9519 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:38.580763  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.606609  4599 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:38.607401  4755 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:38.580931  4679 raft_consensus.cc:3055] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:38.608587  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.590212  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:38.594038  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.568208  9452 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.469305  9418 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.597352  4754 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:38.579223  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:38.604784  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.609005  4679 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:38.616060  4679 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.617079  4679 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:38.618193  9474 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:38.618721  9474 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:38.619534  9474 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:38.620321  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:38.596702  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:38.623055  9423 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.624130  9370 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:38.624233  4497 catalog_manager.cc:5697] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 8 to 13, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 13 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:38.611671  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:38.612176  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.634627  4750 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
W20260709 12:26:38.652822  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 50415.9us
I20260709 12:26:38.584522  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.566165  9244 consensus_queue.cc:1059] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:26:38.612541  4747 raft_consensus.cc:2410] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:26:38.657043  4756 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:38.657763  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:26:38.671628  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 56754us
I20260709 12:26:38.480767  9022 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.130s
I20260709 12:26:38.577966  9331 consensus_queue.cc:1059] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:26:38.664131  4598 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:38.662642  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.661429  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.653855  4748 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:38.594878  9345 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:26:38.611127  9488 consensus_queue.cc:1243] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:38.675902  4750 raft_consensus.cc:3055] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:38.702802  4750 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:38.676684  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:38.690550  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:38.703825  4600 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:38.696965  4678 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:38.586833  9425 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.671499  9423 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.668838  4679 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:26:38.583006  9466 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.658004  9370 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:38.710214  4647 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:38.607118  9355 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:26:38.716457  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:38.669833  9243 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:26:38.720846  4679 pending_rounds.cc:85] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 7
I20260709 12:26:38.728775  5312 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:26:38.724539  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 294691us
W20260709 12:26:38.706183  9453 consensus_meta.cc:287] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.892s	user 0.006s	sys 0.018s
I20260709 12:26:38.728525  4750 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.659138  9452 consensus_queue.cc:1059] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:26:38.707940  9343 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:38.680227  9330 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:38.777989  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 61603.1us
W20260709 12:26:38.778461  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.761664  4750 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:38.764472  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 185352us
I20260709 12:26:38.766832  4749 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:38.780143  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:38.730645  4679 pending_rounds.cc:107] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:26:38.787400  4679 raft_consensus.cc:2889] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:38.788803  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:38.779050  4571 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:38.720441  9488 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:26:38.708514  9383 consensus_queue.cc:1059] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 12:26:38.798892  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:38.813019  9463 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:38.813759  9463 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:38.814849  9463 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.815423  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:38.816483  4675 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:38.722389  9368 consensus_queue.cc:1059] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:26:38.770285  9512 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:38.731370  9344 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.777534  9343 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:38.825613  4568 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:38.797339  9338 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:38.727820  9423 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:38.760215  9244 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 12:26:38.806365  9394 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:38.853734  4568 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:38.626000  4497 catalog_manager.cc:5697] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 7 to 13, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193), now has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 13 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } } pending_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:38.720180  9337 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:38.760600  9370 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:38.789609  9402 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:26:38.877310  4568 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:38.879833  4568 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:38.857079  9338 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.882588  4568 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:38.857424  8992 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:38.892992  9338 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:38.892931  4568 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:38.893615  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.899834  4568 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:38.895588  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.903810  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.908600  4747 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:38.880779  9264 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:26:38.910341  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.913827  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.915843  4719 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:38.880700  9371 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:38.923275  4568 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:38.890117  9394 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.927536  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.840978  9512 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.852829  9343 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:38.879426  9353 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:26:38.942615  4750 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:26:38.891866  9198 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:38.939814  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.950157  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:38.941119  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 161047us
I20260709 12:26:38.954170  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:38.857259  9368 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.916102  9411 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:38.955531  9411 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.956027  9411 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:38.908671  9036 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:38.943122  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.963275  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:38.963856  4755 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:38.898355  9462 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:38.969847  9368 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:38.972662  9368 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.934152  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.945004  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:38.943969  4750 pending_rounds.cc:85] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 7
I20260709 12:26:38.926128  9499 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12.
I20260709 12:26:38.978211  4750 pending_rounds.cc:107] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:26:38.983400  4750 raft_consensus.cc:2889] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:38.979810  4748 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 12:26:38.935196  9526 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 12:26:38.987286  4755 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:38.934561  9394 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:38.928543  9439 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:38.931834  9431 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:39.006803  4756 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:26:38.940522  9512 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:39.007438  4750 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:38.986349  9531 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:26:38.975801  9517 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:38.937423  9418 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.025341  4748 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 12:26:39.030507  9538 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
W20260709 12:26:39.033660  9535 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
W20260709 12:26:39.037774  9031 consensus_meta.cc:287] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.623s	user 0.003s	sys 0.008s
I20260709 12:26:39.038019  9499 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:39.038733  9393 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.038867  9499 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.039260  9393 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.041725  4603 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
W20260709 12:26:39.042176  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.042860  4603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2464) took 2075 ms (client timeout 1942 ms). Trace:
I20260709 12:26:39.043069  9330 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:39.043467  9330 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:39.043277  4603 rpcz_store.cc:269] 0709 12:26:36.967431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.967570 (+   139us) service_pool.cc:224] Handling call
0709 12:26:39.042644 (+2075074us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b5400034654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:39.042752 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4768072448}
I20260709 12:26:39.043920  9330 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:39.046312  9415 consensus_meta.cc:287] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.631s	user 0.010s	sys 0.001s
I20260709 12:26:39.047991  9410 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.048478  9410 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:39.049636  9499 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:39.050452  9499 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:39.051391  9410 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.052074  5344 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:39.052533  9462 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:39.053105  9462 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.055164  9462 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.056028  9330 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:39.058502  9423 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:39.058642  9519 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.061522  9411 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:39.063897  9411 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:39.067272  9534 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
W20260709 12:26:39.083546  9516 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:26:39.082705  9330 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:39.108978  9456 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.089249  9425 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:39.170375  4753 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
W20260709 12:26:39.236996  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.244005  9411 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:39.244686  9411 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.102074  9466 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:39.295051  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.253326  9411 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.314333  9370 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:39.326623  9477 consensus_meta.cc:287] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.967s	user 0.006s	sys 0.006s
I20260709 12:26:39.331797  9393 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.121706  4497 catalog_manager.cc:5697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 5 to 10, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 10 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.344424  9338 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.353334  9512 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:39.372260  9512 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.364284  4497 catalog_manager.cc:5697] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 4 to 7, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 7 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.374437  4497 catalog_manager.cc:5697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 6 to 9, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 9 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.358974  9343 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.381623  9343 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.390965  9338 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.394644  9394 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.420315  9365 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.424052  9365 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:39.437913  4749 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:26:39.438448  4604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.016s	sys 0.044s
I20260709 12:26:39.440150  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:26:39.439289  4604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.016s	sys 0.044s
W20260709 12:26:39.439713  4721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.000s	sys 0.024s
W20260709 12:26:39.437697  9452 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.013s	sys 0.042s
W20260709 12:26:39.446313  4721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.000s	sys 0.024s
W20260709 12:26:39.441308  4680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.009s	sys 0.015s
W20260709 12:26:39.487993  4680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.010s	sys 0.015s
I20260709 12:26:39.437049  9370 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:39.437279  9393 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.439424  9423 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:39.484716  9452 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.013s	sys 0.043s
I20260709 12:26:39.397492  9394 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.486794  9270 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:39.515893  9264 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:26:39.517784  5312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.006s	sys 0.083s
W20260709 12:26:39.524694  5312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.006s	sys 0.083s
I20260709 12:26:39.486003  9466 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:39.491334  9425 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:39.525085  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 639075us
I20260709 12:26:39.490614  9330 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:39.547269  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:39.548169  4598 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:39.528993  9330 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.514005  9370 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.513263  9393 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.559024  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:39.561978  4602 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:39.556533  5312 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:39.517980  9423 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:39.554673  9474 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.049s	sys 0.057s
W20260709 12:26:39.559859  9454 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.021s	sys 0.039s
W20260709 12:26:39.592913  9454 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.022s	sys 0.039s
I20260709 12:26:39.441181  4497 catalog_manager.cc:5697] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 2 to 10. New cstate: current_term: 10 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.529490  9466 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.536693  9425 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:39.585129  9474 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.049s	sys 0.057s
I20260709 12:26:39.593217  9243 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:39.596557  4720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.011s	sys 0.019s
I20260709 12:26:39.597776  5312 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:39.613322  5313 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.031s	sys 0.024s
W20260709 12:26:39.619390  5313 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.032s	sys 0.024s
W20260709 12:26:39.574958  4719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.014s	sys 0.021s
W20260709 12:26:39.622200  4756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.015s	sys 0.011s
W20260709 12:26:39.885782  4756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.015s	sys 0.011s
W20260709 12:26:39.886775  4719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.964s	user 0.020s	sys 0.024s
W20260709 12:26:39.887151  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 971347us
I20260709 12:26:39.887859  4719 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:39.888661  9569 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:26:39.595430  9383 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.006s	sys 0.028s
I20260709 12:26:39.889396  9569 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:39.890413  9569 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:39.623919  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:39.893034  5343 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:39.625041  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:39.634406  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:39.634475  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:39.628955  4754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.821s	user 0.004s	sys 0.022s
W20260709 12:26:39.902808  4754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.095s	user 0.005s	sys 0.022s
I20260709 12:26:39.903627  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3034) took 1375 ms. Trace:
W20260709 12:26:39.635351  5314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.003s	sys 0.024s
W20260709 12:26:39.905767  5314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.999s	user 0.004s	sys 0.024s
I20260709 12:26:39.637108  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:39.639034  4681 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:26:39.909799  4681 pending_rounds.cc:85] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 7
I20260709 12:26:39.909859  5344 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:39.910135  4681 pending_rounds.cc:107] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:26:39.910430  4681 raft_consensus.cc:2889] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:39.911823  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3060) took 1537 ms. Trace:
I20260709 12:26:39.912149  4681 rpcz_store.cc:276] 0709 12:26:38.374146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.374253 (+   107us) service_pool.cc:224] Handling call
0709 12:26:38.392240 (+ 17987us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.910851 (+1518611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.911591 (+   740us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540019f854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:39.911765 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2866316032}
I20260709 12:26:39.643172  9330 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:39.644425  9567 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.629643  9352 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:39.627914  9337 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.011s	sys 0.015s
W20260709 12:26:39.649657  4645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.016s	sys 0.015s
I20260709 12:26:39.644711  9354 raft_consensus.cc:3055] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:39.914186  9354 raft_consensus.cc:740] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:39.913547  9567 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.915232  9354 consensus_queue.cc:260] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.662071  9370 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.662690  9358 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.916492  9567 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:39.643618  9466 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:39.679555  4750 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.008s	sys 0.012s
W20260709 12:26:39.913966  9337 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.012s	sys 0.016s
W20260709 12:26:39.917217  4750 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.008s	sys 0.012s
I20260709 12:26:39.692250  9455 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:39.709707  4755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.009s	sys 0.020s
W20260709 12:26:39.918787  4755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.010s	sys 0.020s
I20260709 12:26:39.920797  9546 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:39.921808  5340 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
W20260709 12:26:39.697299  9533 log.cc:927] Time spent T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 1.016s	user 0.020s	sys 0.001s
I20260709 12:26:39.716331  9393 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:39.719583  4749 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:39.720712  4568 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:39.726716  4602 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
W20260709 12:26:39.914108  4645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.184s	user 0.016s	sys 0.019s
I20260709 12:26:39.734952  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 3031) took 1253 ms. Trace:
I20260709 12:26:39.918887  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2504) took 1013 ms. Trace:
W20260709 12:26:39.739586  9207 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.180s	user 0.004s	sys 0.035s
I20260709 12:26:39.617281  4497 catalog_manager.cc:5697] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 5 to 12, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 12 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.736801  9425 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:39.746294  4678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.005s	sys 0.036s
I20260709 12:26:39.924177  4753 rpcz_store.cc:276] 0709 12:26:38.466195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.589809 (+123614us) service_pool.cc:224] Handling call
0709 12:26:39.719179 (+1129370us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54001b1054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:39.719309 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":856417024}
I20260709 12:26:39.738667  9423 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:39.757660  9568 log.cc:927] Time spent T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 12:26:39.756556  9557 log.cc:927] Time spent T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:26:39.939218  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2493) took 1049 ms. Trace:
W20260709 12:26:39.939461  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 218822us
W20260709 12:26:39.773881  5311 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.012s	sys 0.020s
W20260709 12:26:39.777908  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.799146  9570 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:39.831502  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3080) took 1400 ms. Trace:
I20260709 12:26:39.834117  9344 raft_consensus.cc:3055] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:39.835608  9355 raft_consensus.cc:3055] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:39.837929  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3082) took 1406 ms. Trace:
I20260709 12:26:39.839579  8984 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:39.838335  9186 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:39.841851  9418 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:39.862020  9537 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.952s	user 0.013s	sys 0.004s
I20260709 12:26:39.863351  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3081) took 1431 ms. Trace:
W20260709 12:26:39.863451  9560 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.227s	user 0.002s	sys 0.000s
W20260709 12:26:39.866952  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.955082  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.623505  4720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.011s	sys 0.019s
I20260709 12:26:39.953474  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:39.956063  4753 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:39.955942  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 1.00178e+06us
I20260709 12:26:39.956487  9418 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:39.958634  9418 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:39.959631  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:39.960295  4756 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
W20260709 12:26:39.924103  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 1.19356e+06us
I20260709 12:26:39.954453  4680 rpcz_store.cc:276] 0709 12:26:38.431557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.615680 (+184123us) service_pool.cc:224] Handling call
0709 12:26:38.624475 (+  8795us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.624686 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:38.834590 (+209904us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.862932 (+1028342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.863006 (+    74us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:39.863084 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.863151 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.863289 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":447978,"prepare.queue_time_us":213143,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":375120,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"thread_start_us":765865,"threads_started":2,"wal-append.queue_time_us":804143}
I20260709 12:26:39.962016  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:39.962123  4497 catalog_manager.cc:5697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.963635  4645 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:39.964349  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:39.963835  4497 catalog_manager.cc:5697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.925721  4644 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:39.964941  4603 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:39.947340  8984 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:39.944796  9423 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:39.944963  9344 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:39.963917  4602 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:39.948709  9570 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:39.924680  4750 rpcz_store.cc:276] 0709 12:26:38.905795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.022430 (+116635us) service_pool.cc:224] Handling call
0709 12:26:39.022798 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.023042 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:39.094165 (+ 71123us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.917925 (+823760us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.918209 (+   284us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.918441 (+   232us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.918565 (+   124us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.918605 (+    40us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:39.918653 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.918726 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.918833 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55083,"prepare.queue_time_us":85190,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":104,"raft.queue_time_us":8013,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":209,"thread_start_us":823075,"threads_started":1,"wal-append.queue_time_us":667524}
I20260709 12:26:39.904021  4754 rpcz_store.cc:276] 0709 12:26:38.528446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.639306 (+110860us) service_pool.cc:224] Handling call
0709 12:26:38.654518 (+ 15212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.654719 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:38.806556 (+151837us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.903246 (+1096690us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.903292 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:39.903339 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.903404 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.903564 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1435,"prepare.queue_time_us":182437,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":14317,"raft.queue_time_us":81,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":75,"thread_start_us":1222257,"threads_started":2,"wal-append.queue_time_us":933605}
I20260709 12:26:39.945341  9355 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:39.986588  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:39.966385  9588 consensus_queue.cc:1059] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:26:39.987694  9588 raft_consensus.cc:3055] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:39.988049  9588 raft_consensus.cc:740] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:39.966125  9353 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:39.989974  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:39.990767  5340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 641) took 1705 ms. Trace:
I20260709 12:26:39.966812  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:39.991191  5340 rpcz_store.cc:276] 0709 12:26:38.285545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.535528 (+249983us) service_pool.cc:224] Handling call
0709 12:26:38.535825 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.922249 (+1386424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.990530 (+ 68281us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400280854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:39.990704 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3186930432}
I20260709 12:26:39.935513  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 824) took 1659 ms. Trace:
W20260709 12:26:39.935504  4678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.025s	user 0.005s	sys 0.036s
I20260709 12:26:39.967712  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:39.936662  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.992684  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 750) took 1298 ms. Trace:
I20260709 12:26:39.967444  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:39.994347  4676 raft_consensus.cc:2410] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:26:39.995024  5344 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:39.967912  9555 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:39.967851  8984 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:39.995781  9555 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:39.996670  8984 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:39.997974  9555 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:39.968022  9368 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:39.999003  9368 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:39.934338  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 46546.5us
W20260709 12:26:39.941243  5311 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.033s	user 0.012s	sys 0.021s
I20260709 12:26:40.001127  9368 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:39.937287  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2507) took 1027 ms. Trace:
I20260709 12:26:39.965955  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.016203  4600 raft_consensus.cc:2410] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:26:39.927691  5312 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:40.017097  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 89527.8us
W20260709 12:26:39.935079  9207 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.376s	user 0.004s	sys 0.035s
I20260709 12:26:39.916529  9354 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:39.942414  4674 rpcz_store.cc:276] 0709 12:26:38.431311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.602059 (+170748us) service_pool.cc:224] Handling call
0709 12:26:38.617332 (+ 15273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.617544 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:38.698827 (+ 81283us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.830980 (+1132153us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b18003f7480. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007effeb3cba40 00007effeb3c64e4 00007effeb3c6455 00007effee4ae060 00007effee4c0716 00007effee4d1287 00007effee4d1208
0709 12:26:39.831141 (+   161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.831189 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:39.831236 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.831303 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.831431 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101724,"prepare.queue_time_us":184994,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":4553,"raft.queue_time_us":1010302,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":7795,"spinlock_wait_cycles":5476224,"thread_start_us":124910,"threads_started":1,"wal-append.queue_time_us":233627}
I20260709 12:26:40.022406  4568 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:39.913786  9352 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.945547  4673 rpcz_store.cc:276] 0709 12:26:38.431796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.615725 (+183929us) service_pool.cc:224] Handling call
0709 12:26:38.616117 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.616308 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:38.659299 (+ 42991us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.837502 (+1178203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.837559 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:39.837649 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.837744 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.837866 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":977049,"prepare.queue_time_us":110035,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":962677,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":6569,"thread_start_us":217650,"threads_started":1,"wal-append.queue_time_us":406254}
I20260709 12:26:40.027547  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:39.953474  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:39.889487  9383 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.006s	sys 0.029s
I20260709 12:26:39.967067  9358 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:39.988838  9588 consensus_queue.cc:260] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.971315  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:39.971973  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:40.068428  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:39.980756  9546 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:39.970398  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:39.969767  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:40.008306  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:39.972831  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.073979  4602 raft_consensus.cc:2393] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:40.073894  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:39.976667  9592 consensus_queue.cc:1059] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:26:39.978843  9567 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:40.079245  9567 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.979074  9330 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:39.993321  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:39.967165  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:39.995096  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:39.968173  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:40.085944  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:39.965896  4644 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:39.966264  9500 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:26:39.989970  9353 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:39.966434  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 735) took 1274 ms. Trace:
I20260709 12:26:39.966934  9453 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:39.971963  9570 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:39.971657  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3041) took 1342 ms. Trace:
I20260709 12:26:39.968231  9423 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.992188  4752 rpcz_store.cc:276] 0709 12:26:38.275886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.276033 (+   147us) service_pool.cc:224] Handling call
0709 12:26:38.276359 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:38.276573 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.276603 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:38.328285 (+ 51682us) spinlock_profiling.cc:243] Waited 41.3 ms on lock 0x7b2c000d0108. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:26:38.328668 (+   383us) log.cc:844] Serialized 26 byte log entry
0709 12:26:38.467262 (+138594us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:38.467295 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.467417 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.639810 (+1172393us) spinlock_profiling.cc:243] Waited 4.96 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:39.935269 (+295459us) raft_consensus.cc:1685] finished
0709 12:26:39.935290 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.935440 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106374272,"thread_start_us":112300,"threads_started":1,"wal-append.queue_time_us":222517}
I20260709 12:26:40.014668  9393 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:39.969851  9344 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:39.985635  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:39.942595  5343 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:40.019987  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:39.939674  4755 rpcz_store.cc:276] 0709 12:26:38.889660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.019348 (+129688us) service_pool.cc:224] Handling call
0709 12:26:39.019888 (+   540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.020152 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:39.078553 (+ 58401us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.938631 (+860078us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.938688 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:39.938924 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.939002 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.939108 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":76377,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":1613,"raft.queue_time_us":1223,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":49,"thread_start_us":859499,"threads_started":2,"wal-append.queue_time_us":643764}
I20260709 12:26:40.019517  4719 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:40.021742  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.023151  4750 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:26:40.015343  4749 rpcz_store.cc:276] 0709 12:26:38.909465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.439024 (+529559us) service_pool.cc:224] Handling call
0709 12:26:39.719304 (+280280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.923368 (+204064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.937027 (+ 13659us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54001b1050. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:39.937209 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541589760}
I20260709 12:26:40.129283  4600 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:40.097972  9353 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:39.993054  4678 rpcz_store.cc:276] 0709 12:26:38.693814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.820502 (+126688us) service_pool.cc:224] Handling call
0709 12:26:38.820911 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:38.821224 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.821261 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:38.842129 (+ 20868us) log.cc:844] Serialized 26 byte log entry
0709 12:26:39.992381 (+1150252us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:39.992421 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.992485 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.992525 (+    40us) raft_consensus.cc:1685] finished
0709 12:26:39.992535 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.992636 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1081966,"threads_started":1,"wal-append.queue_time_us":926201}
I20260709 12:26:39.980006  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:40.133379  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 113465us
W20260709 12:26:40.134495  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 154564us
I20260709 12:26:40.045437  4673 raft_consensus.cc:2410] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:26:40.069136  4675 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:39.972281  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.070138  4599 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:26:40.066664  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 44358.1us
I20260709 12:26:40.071234  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.072022  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:39.978542  9425 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:40.074666  4672 raft_consensus.cc:2410] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:26:40.077041  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.140322  4602 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:26:40.077178  9588 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:40.030457  9589 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.068235  9418 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:40.078263  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.023816  9368 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.026120  9175 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:40.023993  9555 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.023862  9455 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:40.153884  9418 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:40.154405  9455 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:40.146844  9466 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:40.155859  9466 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:40.156231  9466 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:40.156582  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:40.156787  9418 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:40.156981  9455 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:40.157202  4675 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:40.143911  9337 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:40.147063  9425 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:40.158326  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:40.159092  4675 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:40.159674  9344 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:40.160060  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:40.160845  4675 raft_consensus.cc:2410] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:26:40.129287  9393 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:40.162235  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:40.162909  4605 raft_consensus.cc:2368] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13.
I20260709 12:26:40.163115  4601 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:40.144353  9431 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:40.163605  9393 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:40.164099  9431 raft_consensus.cc:697] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:40.164461  9368 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:40.087845  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.035717  9352 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:40.166618  9368 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:40.093889  4604 rpcz_store.cc:276] 0709 12:26:38.692110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.700188 (+  8078us) service_pool.cc:224] Handling call
0709 12:26:38.723212 (+ 23024us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:38.723461 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.723486 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:38.724208 (+   722us) log.cc:844] Serialized 26 byte log entry
0709 12:26:39.444711 (+720503us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:39.444772 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.444868 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.966235 (+521367us) raft_consensus.cc:1685] finished
0709 12:26:39.966255 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.966368 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":676913,"threads_started":1,"wal-append.queue_time_us":865758}
I20260709 12:26:40.095723  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:40.097190  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 131367us
I20260709 12:26:40.098361  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:40.081992  9567 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:40.099648  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:40.101687  4751 rpcz_store.cc:276] 0709 12:26:38.628862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.695734 (+ 66872us) service_pool.cc:224] Handling call
0709 12:26:38.696109 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:38.805459 (+109350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.805516 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:38.812713 (+  7197us) log.cc:844] Serialized 26 byte log entry
0709 12:26:39.244295 (+431582us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:39.244335 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.245673 (+  1338us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.862681 (+617008us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:39.971403 (+108722us) raft_consensus.cc:1685] finished
0709 12:26:39.971424 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.971598 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115251,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":15394,"spinlock_wait_cycles":6267136,"thread_start_us":527779,"threads_started":2,"wal-append.queue_time_us":780909}
I20260709 12:26:40.088733  9330 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:40.096390  9569 raft_consensus.cc:3055] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:40.023494  4596 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:40.131541  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:40.128980  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 109552us
I20260709 12:26:40.140853  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.142846  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.181897  4600 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:40.127542  9592 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.981225  9355 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:40.086989  9546 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:40.304735  4644 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:40.110230  9423 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:40.151530  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.149595  9589 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:40.153548  9555 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:40.128893  9513 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:40.165271  9431 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:40.110773  9570 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:40.164505  4602 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:40.106689  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:40.169648  4675 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:40.168224  4676 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:26:40.097774  9453 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:40.260389  4600 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:26:40.306455  4644 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:40.335193  4644 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:40.175438  9569 raft_consensus.cc:740] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:40.396804  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:40.308447  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 157580us
I20260709 12:26:39.965541  4497 catalog_manager.cc:5697] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 7 to 10, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 10 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:40.104110  9358 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:40.426998  4679 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:40.429764  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.175307  9175 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:40.383518  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:40.440958  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:40.441540  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.337836  4676 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:40.325454  9455 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:40.336114  9453 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:40.335415  9536 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:40.426206  9331 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:40.310534  9567 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.139s
I20260709 12:26:40.490669  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:40.510895  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.325047  9383 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.157s
I20260709 12:26:40.485399  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:40.425813  9500 consensus_queue.cc:1175] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "dbe960c35c84451faf4e5b5c9233fa39" responder_term: 14 status { last_received { term: 14 index: 8 } last_committed_idx: 7 last_received_current_leader { term: 14 index: 8 } }
I20260709 12:26:40.450201  9414 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:40.552106  4645 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:40.559651  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.339329  9570 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:40.560135  4600 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:26:40.480792  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 345401us
I20260709 12:26:40.561988  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.574987  4597 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:40.576871  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.576887  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:40.580358  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:40.336732  9513 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:40.593947  9591 log.cc:927] Time spent T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 12:26:40.593765  9567 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:40.594642  9567 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:40.597191  9567 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:40.600006  9567 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:40.600574  9567 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:40.602967  9567 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:40.552035  9264 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:40.566665  9511 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:40.572751  4645 leader_election.cc:400] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13.
I20260709 12:26:40.616722  4645 leader_election.cc:403] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 12:26:40.617381  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49382.1us
I20260709 12:26:40.552276  9331 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:40.620697  4645 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:40.577354  9425 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:40.558120  4673 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:40.578969  9368 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:40.321980  9592 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:40.622701  9358 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:40.623301  9418 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:40.578220  9414 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:40.630255  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 372432us
I20260709 12:26:40.551561  9569 consensus_queue.cc:260] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:40.651541  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:40.655929  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.629004  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:40.658321  9595 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:40.630407  9368 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.642424  9536 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:40.635205  9425 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:40.606506  9175 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:40.621501  9264 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:40.826246  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:40.672713  9344 consensus_meta.cc:287] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.513s	user 0.011s	sys 0.000s
I20260709 12:26:40.673655  9355 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:40.850160  9331 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:40.851271  9592 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:40.851927  9589 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:40.852378  9264 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:40.852792  9570 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:40.854528  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:40.870406  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:40.855132  4677 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:40.858274  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:40.858803  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.809448  9393 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.673436  9368 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:40.674767  9596 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:40.810907  9536 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.876075  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:40.877403  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:40.811658  9330 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:40.886304  4719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.011s	sys 0.033s
W20260709 12:26:40.887626  4719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.011s	sys 0.033s
W20260709 12:26:40.888096  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 710251us
I20260709 12:26:40.846926  9425 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:40.883876  9555 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:40.922922  4749 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:40.886394  9536 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:40.927147  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:40.927244  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:40.991626  4751 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:40.991709  4756 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:40.928226  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:40.992686  4750 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:26:40.993938  4750 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:40.995748  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:40.996296  4750 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:40.929935  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:40.997820  4748 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:40.930791  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:40.998757  4753 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:40.931579  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:40.999493  4754 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:40.932688  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:41.000289  4755 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:40.888370  8984 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.891s	user 0.008s	sys 0.012s
I20260709 12:26:41.006296  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.007167  4751 raft_consensus.cc:2410] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:26:40.942608  9414 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:40.918084  9589 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.008203  9589 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:41.010301  9589 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:40.917765  9330 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.010979  9330 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.013058  9330 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:40.888484  9393 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.014966  9393 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:40.917301  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:40.958972  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:40.878369  9546 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:40.940356  9354 consensus_meta.cc:287] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.914s	user 0.009s	sys 0.000s
I20260709 12:26:41.016453  4752 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:40.880059  9596 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.016939  4674 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:41.016991  9596 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:40.919168  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:41.018137  9593 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 12:26:41.019343  4755 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
W20260709 12:26:41.021667  9352 consensus_meta.cc:287] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.855s	user 0.007s	sys 0.003s
I20260709 12:26:41.022179  9352 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:41.026634  9596 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.034767  9596 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:41.042580  9513 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:41.047346  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:41.048091  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.025214  9352 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:41.057221  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 476898us
I20260709 12:26:40.966030  9344 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.954770  9358 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13.
W20260709 12:26:41.063217  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.974846  9570 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:41.091091  4604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.007s	sys 0.010s
W20260709 12:26:41.091766  4604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.007s	sys 0.010s
I20260709 12:26:41.067232  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:41.092571  4756 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:41.069762  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.093361  4749 raft_consensus.cc:2393] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:26:41.077729  9423 consensus_meta.cc:287] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.746s	user 0.016s	sys 0.001s
I20260709 12:26:41.093513  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:41.094587  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:41.098958  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:41.109516  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:41.110510  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.110934  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:41.057029  9354 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.112170  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:40.672225  4497 catalog_manager.cc:5697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:41.070652  9353 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.893s	user 0.011s	sys 0.001s
I20260709 12:26:41.113240  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:41.060890  9589 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:41.118563  4596 raft_consensus.cc:2393] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:41.090773  9405 raft_consensus.cc:3055] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:41.108831  9596 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:41.140913  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 838) took 1189 ms. Trace:
I20260709 12:26:41.141022  4748 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:41.128043  9405 raft_consensus.cc:740] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:41.142360  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.143049  4748 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:41.143330  9405 consensus_queue.cc:260] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:41.143968  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.144605  4748 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:41.117817  9354 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.146706  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:41.147351  4755 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:26:41.141369  4747 rpcz_store.cc:276] 0709 12:26:39.951841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.993741 (+ 41900us) service_pool.cc:224] Handling call
0709 12:26:39.994027 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:40.332676 (+338649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.332705 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:40.332744 (+    39us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:40.332775 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.140653 (+807878us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b5400072bd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:41.140702 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.140856 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":778354432}
I20260709 12:26:41.145450  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.091701  9344 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.108997  9570 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:41.156404  4753 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:41.158547  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.112603  9423 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:41.161016  4647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.028s	sys 0.058s
W20260709 12:26:41.171705  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 328712us
W20260709 12:26:41.172163  4647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.029s	sys 0.059s
I20260709 12:26:41.173928  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:41.179286  4750 raft_consensus.cc:2368] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13.
W20260709 12:26:41.180402  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:41.181263  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.135457  9596 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:41.119654  9353 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.132350  9589 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.177718  9354 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:41.190083  4755 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:41.191812  4753 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:26:41.173041  4645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.005s	sys 0.011s
W20260709 12:26:41.200620  4645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.005s	sys 0.011s
W20260709 12:26:41.201021  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 580359us
I20260709 12:26:41.184340  9596 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:41.209153  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:41.201398  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:41.210716  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:41.211465  4598 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:41.212576  4600 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:41.210027  4596 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:41.223935  9393 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.224299  9344 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:41.224561  9393 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:41.226737  9393 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:41.226787  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:41.227583  5345 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:41.227967  9353 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:41.228727  9186 raft_consensus.cc:3055] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:41.229101  9186 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:41.229868  9186 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.230775  9186 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:41.234738  9569 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:41.235384  4598 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:41.238782  9570 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:41.239490  4497 catalog_manager.cc:5697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:41.241588  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.242240  4754 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:41.241483  4497 catalog_manager.cc:5697] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 5 to 13. New cstate: current_term: 13 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:41.243070  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.243624  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.244165  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.243566  4497 catalog_manager.cc:5697] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 3 to 11. New cstate: current_term: 11 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:41.244895  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.245186  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.258522  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:41.259491  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:41.246404  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.259567  4645 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:41.260838  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:41.260203  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:41.268337  5343 raft_consensus.cc:2410] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:26:41.255899  4748 raft_consensus.cc:2410] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:26:41.256469  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2501) took 1168 ms. Trace:
I20260709 12:26:41.247692  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1967) took 1273 ms. Trace:
I20260709 12:26:41.269304  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.269510  4679 rpcz_store.cc:276] 0709 12:26:39.973972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.099404 (+125432us) service_pool.cc:224] Handling call
0709 12:26:41.247527 (+1148123us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b540019f854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:41.247637 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":593673088}
I20260709 12:26:41.270006  4748 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:41.247066  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.270780  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:41.271446  4676 raft_consensus.cc:2410] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:26:41.254781  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 53469.9us
I20260709 12:26:41.272838  4647 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:41.269318  4603 rpcz_store.cc:276] 0709 12:26:40.088361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.166821 (+ 78460us) service_pool.cc:224] Handling call
0709 12:26:40.167128 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.167441 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.167467 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.170800 (+  3333us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.874296 (+703496us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:40.874333 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.874398 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.256236 (+381838us) raft_consensus.cc:1685] finished
0709 12:26:41.256257 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.256404 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3440,"raft.queue_time_us":3556,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":119090,"thread_start_us":456237,"threads_started":1,"wal-append.queue_time_us":817123}
I20260709 12:26:41.274784  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.276232  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.276391  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.277375  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.278002  9596 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.278753  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.279136  9423 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:41.279526  4679 raft_consensus.cc:2410] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:41.280360  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.280625  9423 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:41.280988  4679 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 9.
I20260709 12:26:41.281554  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:41.282527  5347 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:41.280416  9596 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:41.249712  9425 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:41.286049  9455 consensus_meta.cc:287] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.739s	user 0.011s	sys 0.000s
I20260709 12:26:41.286645  9455 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:41.287063  9466 consensus_meta.cc:287] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.130s	user 0.013s	sys 0.000s
I20260709 12:26:41.287711  4755 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:41.287501  9466 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.290266  9466 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:41.293784  4675 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:41.294041  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:41.263836  4754 raft_consensus.cc:2410] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:26:41.295783  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.296445  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.297803  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.297864  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:41.298559  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1970) took 1323 ms. Trace:
I20260709 12:26:41.298950  4678 rpcz_store.cc:276] 0709 12:26:39.974728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.150434 (+175706us) service_pool.cc:224] Handling call
0709 12:26:41.297671 (+1147237us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b58000d4798. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:26:41.298508 (+   837us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25607936}
I20260709 12:26:41.245537  4497 catalog_manager.cc:5697] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:41.299952  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.300518  4678 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:41.308441  9405 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:41.271374  4748 raft_consensus.cc:2376] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12. Re-sending same reply.
I20260709 12:26:41.274937  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:41.312386  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 52896.4us
I20260709 12:26:41.313509  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:41.314246  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:41.301431  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:41.319895  4678 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:41.329599  5312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.005s	sys 0.012s
I20260709 12:26:41.330029  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1975) took 1354 ms. Trace:
W20260709 12:26:41.329969  5312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.005s	sys 0.012s
W20260709 12:26:41.332023  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 521391us
I20260709 12:26:41.337047  4645 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:41.342207  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2510) took 1019 ms. Trace:
I20260709 12:26:41.342839  4599 rpcz_store.cc:276] 0709 12:26:40.322832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.323019 (+   187us) service_pool.cc:224] Handling call
0709 12:26:40.811642 (+488623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:40.811820 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.811850 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:40.811909 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:40.811948 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.812016 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.341928 (+529912us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54002e4ad0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:41.342116 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":892978048}
I20260709 12:26:41.325197  4675 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:41.329826  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:41.254956  8984 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:41.247053  9589 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:41.330533  4677 rpcz_store.cc:276] 0709 12:26:39.975971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.093478 (+117507us) service_pool.cc:224] Handling call
0709 12:26:41.329849 (+1236371us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b54001bc454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:41.329972 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1580654976}
I20260709 12:26:41.362257  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:41.363770  4647 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:41.374698  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.375116  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:41.320932  9355 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:41.376618  5311 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.012s	sys 0.020s
I20260709 12:26:41.326874  9500 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:41.375524  4677 raft_consensus.cc:2410] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:26:41.377843  4678 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:41.309974  9423 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:41.380661  4675 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:41.419385  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:41.268401  9368 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.365861  9354 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:41.411116  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.369686  9358 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:41.413560  4647 leader_election.cc:400] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13.
I20260709 12:26:41.379410  9555 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:41.417130  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:41.425697  4678 raft_consensus.cc:2410] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:26:41.425911  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:41.426595  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.427172  4678 raft_consensus.cc:2376] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15. Re-sending same reply.
W20260709 12:26:41.376075  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.419430  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:41.380295  5311 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.012s	sys 0.021s
I20260709 12:26:41.431583  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.443329  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.445881  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.446806  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3086) took 1109 ms. Trace:
I20260709 12:26:41.447203  4678 rpcz_store.cc:276] 0709 12:26:40.337284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.445653 (+1108369us) service_pool.cc:224] Handling call
0709 12:26:41.446730 (+  1077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:41.405664  9500 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.478466  4675 raft_consensus.cc:2393] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:41.417382  9355 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:41.406810  9576 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:26:41.428900  9555 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:41.574821  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.626819  4673 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:41.628113  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.440009  9611 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:41.680919  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:41.587265  4677 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
W20260709 12:26:41.598315  9606 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
W20260709 12:26:41.602384  9588 consensus_meta.cc:287] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.453s	user 0.008s	sys 0.008s
I20260709 12:26:41.577981  9423 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:41.480778  9500 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:41.682729  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3109) took 1017 ms. Trace:
I20260709 12:26:41.560847  9608 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.637704  9555 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:41.697206  4681 raft_consensus.cc:2368] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12.
I20260709 12:26:41.687950  9607 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:41.679116  9466 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.690276  5340 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:41.692960  4677 rpcz_store.cc:276] 0709 12:26:40.665854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.419024 (+753170us) service_pool.cc:224] Handling call
0709 12:26:41.682649 (+263625us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3328}
I20260709 12:26:41.696344  9588 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:41.713928  5314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.032s	sys 0.048s
W20260709 12:26:41.706640  9453 consensus_meta.cc:287] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.211s	user 0.012s	sys 0.008s
I20260709 12:26:41.716029  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2504) took 1609 ms. Trace:
I20260709 12:26:41.705307  9355 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.696305  9589 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260709 12:26:41.712205  9331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.033s	sys 0.046s
W20260709 12:26:41.730264  5314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.032s	sys 0.048s
W20260709 12:26:41.741181  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 609789us
I20260709 12:26:41.732210  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.745209  4677 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
W20260709 12:26:41.727720  4597 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.005s	sys 0.024s
W20260709 12:26:41.749405  4597 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.006s	sys 0.024s
I20260709 12:26:41.685176  9611 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:41.730011  4604 rpcz_store.cc:276] 0709 12:26:40.106367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.167418 (+ 61051us) service_pool.cc:224] Handling call
0709 12:26:40.309708 (+142290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.310139 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.310179 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.312354 (+  2175us) log.cc:844] Serialized 26 byte log entry
0709 12:26:41.137972 (+825618us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:41.138016 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.138090 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.715757 (+577667us) raft_consensus.cc:1685] finished
0709 12:26:41.715791 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.715957 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10738,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":11621,"thread_start_us":825266,"threads_started":1,"wal-append.queue_time_us":1028746}
W20260709 12:26:41.765247  4749 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.044s	sys 0.039s
W20260709 12:26:41.777357  4749 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.046s	sys 0.039s
W20260709 12:26:41.777794  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.776082  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.701289  9608 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.709153  9466 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.780546  4677 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
W20260709 12:26:41.781711  4751 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.019s	sys 0.068s
W20260709 12:26:41.783720  4751 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.019s	sys 0.068s
W20260709 12:26:41.735280  9331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.033s	sys 0.047s
I20260709 12:26:41.785038  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:41.786754  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:41.789681  4680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.008s	sys 0.004s
W20260709 12:26:41.790263  4680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.008s	sys 0.004s
W20260709 12:26:41.796515  4720 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.004s	sys 0.024s
W20260709 12:26:41.796905  4720 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.004s	sys 0.024s
W20260709 12:26:41.797214  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 588543us
I20260709 12:26:41.801234  4677 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
W20260709 12:26:41.801636  4750 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.010s	sys 0.023s
W20260709 12:26:41.801966  4750 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.010s	sys 0.023s
I20260709 12:26:41.800083  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:41.801105  4571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.016s	sys 0.018s
W20260709 12:26:41.803562  4571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.016s	sys 0.018s
W20260709 12:26:41.803972  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 553866us
W20260709 12:26:41.787046  9536 consensus_meta.cc:287] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.729s	user 0.007s	sys 0.005s
I20260709 12:26:41.733415  9453 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:41.735248  9355 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:41.801057  9410 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.003s	sys 0.026s
I20260709 12:26:41.815606  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:26:41.961993  9410 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.003s	sys 0.027s
W20260709 12:26:41.967614  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 114439us
I20260709 12:26:41.800076  9567 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:41.736153  9588 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:41.815865  9368 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:41.989710  4569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.046s	sys 0.069s
I20260709 12:26:41.968577  5314 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:41.994184  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:41.994808  4569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.047s	sys 0.069s
W20260709 12:26:41.995223  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 782396us
I20260709 12:26:41.993561  4672 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:41.999006  9616 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:42.009996  4719 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:42.012998  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:42.016840  4753 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.024s	sys 0.102s
I20260709 12:26:41.852290  9423 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.146s
W20260709 12:26:42.008013  9337 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.034s	sys 0.051s
W20260709 12:26:42.020473  4753 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.024s	sys 0.102s
I20260709 12:26:41.853773  9536 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:42.024585  9337 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.034s	sys 0.051s
I20260709 12:26:42.031736  9453 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:42.033200  9466 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:42.033835  9466 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:42.034606  9466 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:42.034773  9423 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:42.036407  9423 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:42.037384  9588 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:42.038882  9536 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:42.039769  4676 raft_consensus.cc:2393] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:42.040493  9569 consensus_meta.cc:287] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.805s	user 0.005s	sys 0.004s
W20260709 12:26:42.040916  4676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3103) took 1408 ms (client timeout 1595 ms). Trace:
I20260709 12:26:42.041057  4756 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
W20260709 12:26:42.041287  4676 rpcz_store.cc:269] 0709 12:26:40.632590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.277148 (+644558us) service_pool.cc:224] Handling call
0709 12:26:42.040730 (+763582us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b540017f054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:42.040862 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1751549184}
I20260709 12:26:42.042465  9355 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:42.042874  9608 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:42.025270  9186 consensus_meta.cc:287] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.792s	user 0.008s	sys 0.003s
I20260709 12:26:42.044435  9611 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:42.044970  9611 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:26:42.046625  9615 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:42.047760  4679 raft_consensus.cc:2410] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:26:42.048717  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3105) took 1410 ms. Trace:
W20260709 12:26:42.049268  9614 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.264s	user 0.000s	sys 0.000s
I20260709 12:26:42.042430  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:42.049072  4679 rpcz_store.cc:276] 0709 12:26:40.638617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.293759 (+655142us) service_pool.cc:224] Handling call
0709 12:26:42.048553 (+754794us) spinlock_profiling.cc:243] Waited 753 ms on lock 0x7b540017d754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:42.048667 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1731160320}
I20260709 12:26:42.050470  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:42.050812  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:42.051007  4676 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:42.051402  4679 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:42.052217  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:42.052750  4679 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:42.053670  9423 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:42.054160  9423 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:42.054587  9423 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:41.992369  9567 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:42.060359  9567 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:42.061610  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:42.083252  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:42.085700  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:42.085881  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:42.086410  4605 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:42.005004  9500 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:42.088091  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:42.089412  4596 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:42.090677  4601 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:26:42.091835  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:42.086433  4598 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:42.103758  4676 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:42.089195  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:42.092521  4601 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
W20260709 12:26:42.110970  4644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.019s	sys 0.011s
W20260709 12:26:42.124493  4644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.020s	sys 0.011s
I20260709 12:26:42.125509  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 755) took 2131 ms. Trace:
I20260709 12:26:42.126919  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:42.131748  4677 rpcz_store.cc:276] 0709 12:26:39.994051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.124526 (+2130475us) service_pool.cc:224] Handling call
0709 12:26:42.124885 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.125109 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.125147 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.125202 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:42.125247 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.125319 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.125445 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.134716  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 757) took 2140 ms. Trace:
I20260709 12:26:42.135227  4677 rpcz_store.cc:276] 0709 12:26:39.994525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.133792 (+2139267us) service_pool.cc:224] Handling call
0709 12:26:42.134128 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.134334 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.134372 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.134426 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:42.134470 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.134537 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.134653 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.137182  4677 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:26:42.137900  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 758) took 2143 ms. Trace:
I20260709 12:26:42.138356  4677 rpcz_store.cc:276] 0709 12:26:39.994753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.136443 (+2141690us) service_pool.cc:224] Handling call
0709 12:26:42.136792 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.137640 (+   848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.137843 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.140228  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 759) took 2145 ms. Trace:
I20260709 12:26:42.140746  4677 rpcz_store.cc:276] 0709 12:26:39.994972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.139257 (+2144285us) service_pool.cc:224] Handling call
0709 12:26:42.139563 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.139785 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.139825 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.139879 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:42.139932 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.140011 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.140167 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.127853  9622 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:42.161895  9622 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:42.164258  9622 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:42.127239  9608 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:42.164987  9500 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:42.127441  9393 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:42.080147  9610 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.165656  9393 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:42.165920  9610 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:42.167685  9393 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:42.167322  9555 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:42.167903  9610 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:42.143350  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:42.168694  4602 raft_consensus.cc:2393] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:42.168833  4673 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:42.171113  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:42.172468  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:42.165177  9608 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:42.127861  4644 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:42.148514  9405 consensus_meta.cc:287] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.839s	user 0.006s	sys 0.006s
I20260709 12:26:42.153160  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:42.153990  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:42.154659  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:42.185143  4596 raft_consensus.cc:2368] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15.
I20260709 12:26:42.155253  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:42.175486  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3085) took 1838 ms. Trace:
W20260709 12:26:42.176299  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 48551.4us
I20260709 12:26:42.186124  4675 rpcz_store.cc:276] 0709 12:26:40.337008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.443071 (+1106063us) service_pool.cc:224] Handling call
0709 12:26:42.175237 (+732166us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b540017dc54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:42.175406 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78194048}
I20260709 12:26:42.187909  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:42.188489  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 761) took 2161 ms. Trace:
I20260709 12:26:42.188956  4675 rpcz_store.cc:276] 0709 12:26:40.026959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.187501 (+2160542us) service_pool.cc:224] Handling call
0709 12:26:42.187869 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.188069 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.188106 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.188165 (+    59us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:42.188219 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.188287 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.188414 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.191222  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 762) took 2111 ms. Trace:
I20260709 12:26:42.191624  4675 rpcz_store.cc:276] 0709 12:26:40.080204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.190309 (+2110105us) service_pool.cc:224] Handling call
0709 12:26:42.190645 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.190840 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.190879 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.190925 (+    46us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:42.190969 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.191037 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.191163 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.195957  4675 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:42.196810  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 763) took 2116 ms. Trace:
I20260709 12:26:42.197257  4675 rpcz_store.cc:276] 0709 12:26:40.080573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.195377 (+2114804us) service_pool.cc:224] Handling call
0709 12:26:42.195772 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.196507 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.196750 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.182130  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 760) took 2156 ms. Trace:
I20260709 12:26:42.198412  4677 rpcz_store.cc:276] 0709 12:26:40.025278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.177237 (+2151959us) service_pool.cc:224] Handling call
0709 12:26:42.177500 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.177680 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.177705 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.177745 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:42.177773 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.177818 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.182064 (+  4246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.183208  9622 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:42.204751  4675 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:42.205682  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3084) took 1869 ms. Trace:
I20260709 12:26:42.206130  4675 rpcz_store.cc:276] 0709 12:26:40.336702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.204014 (+1867312us) service_pool.cc:224] Handling call
0709 12:26:42.204352 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.205266 (+   914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.205619 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:42.210258  4645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.014s	sys 0.043s
W20260709 12:26:42.211365  4645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.014s	sys 0.043s
I20260709 12:26:42.217744  4677 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:26:42.218358  4677 pending_rounds.cc:85] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 5
I20260709 12:26:42.165095  9423 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:42.218758  4677 pending_rounds.cc:107] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:26:42.183729  9358 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:42.220404  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:42.221800  4605 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:42.226661  4674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1962) took 2254 ms (client timeout 1680 ms). Trace:
W20260709 12:26:42.227229  4674 rpcz_store.cc:269] 0709 12:26:39.972609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.025771 (+ 53162us) service_pool.cc:224] Handling call
0709 12:26:42.226368 (+2200597us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b54001d29d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:42.226522 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2018426240}
I20260709 12:26:42.184312  9393 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:42.231499  4598 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:42.210492  9603 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:42.239580  4677 raft_consensus.cc:2889] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:42.246800  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3088) took 1908 ms. Trace:
I20260709 12:26:42.247290  4677 rpcz_store.cc:276] 0709 12:26:40.338425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.201972 (+1863547us) service_pool.cc:224] Handling call
0709 12:26:42.202327 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.246474 (+ 44147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.246730 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.248667  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:42.242833  4647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.020s	sys 0.038s
W20260709 12:26:42.254942  4647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.020s	sys 0.039s
W20260709 12:26:42.256026  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 800747us
I20260709 12:26:42.256791  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:42.256995  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:42.258800  4672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3094) took 1775 ms (client timeout 1698 ms). Trace:
I20260709 12:26:42.258054  4675 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:26:42.251325  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:42.286412  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:42.291083  4599 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:42.277673  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 462150us
I20260709 12:26:42.267017  4675 pending_rounds.cc:85] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 5
I20260709 12:26:42.292047  4675 pending_rounds.cc:107] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:26:42.292346  4675 raft_consensus.cc:2889] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:42.292956  4602 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
W20260709 12:26:42.258831  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 458802us
W20260709 12:26:42.266716  4672 rpcz_store.cc:269] 0709 12:26:40.483344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.552924 (+ 69580us) service_pool.cc:224] Handling call
0709 12:26:42.258568 (+1705644us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54001bce54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:42.258704 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3243382528}
I20260709 12:26:42.296346  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3100) took 1669 ms. Trace:
I20260709 12:26:42.296876  4672 rpcz_store.cc:276] 0709 12:26:40.626881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.295370 (+1668489us) service_pool.cc:224] Handling call
0709 12:26:42.295748 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.295958 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.295995 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.296050 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:42.296095 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.296163 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.296283 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.303239  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:42.299078  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3112) took 1627 ms. Trace:
I20260709 12:26:42.327983  4672 rpcz_store.cc:276] 0709 12:26:40.671351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.298115 (+1626764us) service_pool.cc:224] Handling call
0709 12:26:42.298466 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.298686 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.298722 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.298777 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:42.298829 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.298898 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.299017 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.330911  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3113) took 1659 ms. Trace:
I20260709 12:26:42.331782  4672 rpcz_store.cc:276] 0709 12:26:40.671666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.329891 (+1658225us) service_pool.cc:224] Handling call
0709 12:26:42.330270 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.330486 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.330524 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.330581 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:42.330640 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.330712 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.330845 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.337615  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3089) took 1998 ms. Trace:
I20260709 12:26:42.217909  9622 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:42.235211  9393 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:42.342319  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 66877us
I20260709 12:26:42.245088  9603 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:26:42.352020  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 48866.3us
I20260709 12:26:42.354001  4596 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:42.387372  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:42.389832  4596 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:42.267742  9355 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:42.373957  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:42.373302  4599 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:42.276181  4605 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:42.251830  9474 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.104s
W20260709 12:26:42.341630  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 66730.9us
I20260709 12:26:42.398186  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:42.399204  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 399502us
I20260709 12:26:42.399820  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:42.358258  9555 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:42.401353  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:42.272851  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3099) took 1641 ms. Trace:
I20260709 12:26:42.402518  4677 rpcz_store.cc:276] 0709 12:26:40.626548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.255300 (+1628752us) service_pool.cc:224] Handling call
0709 12:26:42.255656 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.267159 (+ 11503us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:42.267186 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.267224 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:42.267257 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.267341 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.267488 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11089,"raft.queue_time_us":82439,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":11594}
I20260709 12:26:42.380148  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:42.352803  9631 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:42.404552  9631 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.356525  9511 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:42.363292  4680 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.009s	sys 0.014s
I20260709 12:26:42.378647  4675 rpcz_store.cc:276] 0709 12:26:40.338712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.207160 (+1868448us) service_pool.cc:224] Handling call
0709 12:26:42.207499 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.337295 (+129796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.337523 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.403900  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:42.391430  9603 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:42.411777  4681 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 1956) took 2446 ms (client timeout 1848 ms). Trace:
I20260709 12:26:42.375743  9207 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
W20260709 12:26:42.422185  4681 rpcz_store.cc:269] 0709 12:26:39.965014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.966596 (+  1582us) service_pool.cc:224] Handling call
0709 12:26:42.411524 (+2444928us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b540017f554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:42.411692 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3911035776}
I20260709 12:26:42.419397  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:42.413369  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 854) took 1325 ms. Trace:
I20260709 12:26:42.409873  4747 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:42.426571  4751 rpcz_store.cc:276] 0709 12:26:41.087588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.088310 (+   722us) service_pool.cc:224] Handling call
0709 12:26:41.088547 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.185597 (+ 97050us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:41.784159 (+598562us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.252063 (+467904us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.252142 (+    79us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:42.252184 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.413097 (+160913us) spinlock_profiling.cc:243] Waited 96.8 ms on lock 0x7b5400117954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:42.413162 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.413300 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":841209,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":2751,"raft.queue_time_us":97,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":22056,"spinlock_wait_cycles":222621952,"thread_start_us":1053767,"threads_started":2}
I20260709 12:26:42.406980  9567 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:26:42.431159  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.420701  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:42.432647  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:42.433053  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:42.413102  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2515) took 1240 ms. Trace:
I20260709 12:26:42.434101  4645 leader_election.cc:400] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15.
I20260709 12:26:42.406898  9631 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:42.434481  4645 leader_election.cc:403] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:26:42.437883  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:42.380218  9616 raft_consensus.cc:697] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:42.441717  9627 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:26:42.414311  9355 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:42.446503  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1986) took 1350 ms. Trace:
I20260709 12:26:42.449975  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:42.409086  4672 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:26:42.435803  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 425909us
I20260709 12:26:42.454877  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2517) took 1246 ms. Trace:
I20260709 12:26:42.406623  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3115) took 1596 ms. Trace:
I20260709 12:26:42.457759  4677 rpcz_store.cc:276] 0709 12:26:40.810593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.405789 (+1595196us) service_pool.cc:224] Handling call
0709 12:26:42.406055 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.406247 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.406270 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.406318 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:42.406369 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.406436 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.406549 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:42.446930  4597 rpcz_store.cc:276] 0709 12:26:41.096028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.096422 (+   394us) service_pool.cc:224] Handling call
0709 12:26:41.096831 (+   409us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:41.097316 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:41.774152 (+676836us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.286944 (+512792us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:42.287649 (+   705us) log.cc:844] Serialized 52 byte log entry
0709 12:26:42.287824 (+   175us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:42.287870 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.292034 (+  4164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.446239 (+154205us) raft_consensus.cc:1685] finished
0709 12:26:42.446257 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.446418 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1112616,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":198,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":47776,"thread_start_us":1092238,"threads_started":2,"wal-append.queue_time_us":630487}
I20260709 12:26:42.454083  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 766) took 1024 ms. Trace:
I20260709 12:26:42.473559  4672 rpcz_store.cc:276] 0709 12:26:41.429403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.408375 (+978972us) service_pool.cc:224] Handling call
0709 12:26:42.408697 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.453752 (+ 45055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.454014 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:42.474874  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 478999us
I20260709 12:26:42.477736  4719 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:42.425478  9630 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:42.485936  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 52986.9us
I20260709 12:26:42.429291  9354 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:42.475699  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:42.435673  9608 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:42.487361  9500 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:42.487751  9500 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.488255  9500 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:42.487783  9608 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:42.489135  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 55157.6us
I20260709 12:26:42.490767  9607 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15.
I20260709 12:26:42.491001  4645 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:42.491735  9344 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:42.479712  9466 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:42.492607  9466 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:42.494801  9466 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:42.495311  9466 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:42.498229  4645 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:42.492102  9344 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.502031  9344 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:42.509919  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:42.511900  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 752) took 2518 ms. Trace:
I20260709 12:26:42.512001  4645 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:42.512342  4680 rpcz_store.cc:276] 0709 12:26:39.993273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.338854 (+345581us) service_pool.cc:224] Handling call
0709 12:26:40.442277 (+103423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.487854 (+ 45577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.487905 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.488572 (+   667us) log.cc:844] Serialized 26 byte log entry
0709 12:26:41.254118 (+765546us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b1800230b60. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:26:41.790667 (+536549us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:42.421096 (+630429us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.481636 (+ 60540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.481738 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.481795 (+    57us) raft_consensus.cc:1685] finished
0709 12:26:42.511735 (+ 29940us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.511850 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57728,"raft.queue_time_us":111445,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":60985,"spinlock_wait_cycles":5707648,"thread_start_us":1136114,"threads_started":2,"wal-append.queue_time_us":1660359}
I20260709 12:26:42.434360  4750 rpcz_store.cc:276] 0709 12:26:41.172840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.179831 (+  6991us) service_pool.cc:224] Handling call
0709 12:26:41.186910 (+  7079us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:41.187302 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.187330 (+    28us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:41.188001 (+   671us) log.cc:844] Serialized 78 byte log entry
0709 12:26:41.802450 (+614449us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:42.412547 (+610097us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.412800 (+   253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.412872 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.412909 (+    37us) raft_consensus.cc:1685] finished
0709 12:26:42.412919 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.413052 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":951,"prepare.queue_time_us":347248,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":9262,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":23116,"thread_start_us":1000375,"threads_started":2,"wal-append.queue_time_us":835537}
W20260709 12:26:42.481060  9620 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:42.452653  9634 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:42.525405  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:42.472993  9569 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:42.509985  9628 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:42.521857  9623 log.cc:927] Time spent T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:26:42.554363  4647 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:42.511157  9511 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:42.574507  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 756) took 2580 ms. Trace:
I20260709 12:26:42.568614  4645 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:42.486596  9567 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:42.495980  9466 consensus_queue.cc:1243] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:26:42.579998  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48041.4us
I20260709 12:26:42.588160  4676 rpcz_store.cc:276] 0709 12:26:39.994287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.128826 (+2134539us) service_pool.cc:224] Handling call
0709 12:26:42.129123 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.129376 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:42.527527 (+398151us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.573960 (+ 46433us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.574099 (+   139us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:42.574169 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.574264 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.574419 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7664,"raft.queue_time_us":7833,"thread_start_us":361153,"threads_started":2}
W20260709 12:26:42.592461  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 67117.2us
I20260709 12:26:42.524789  9588 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:42.597550  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:42.599370  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 45072.5us
I20260709 12:26:42.599375  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:42.601886  9418 consensus_queue.cc:1059] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:42.603595  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:42.604131  9418 raft_consensus.cc:3055] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:42.605062  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:42.605870  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:42.606459  4750 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
W20260709 12:26:42.607151  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 192499us
I20260709 12:26:42.607446  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:42.608119  4750 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 12: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:42.608726  4748 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:26:42.608971  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:42.525588  9423 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:42.610251  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:42.456528  4753 rpcz_store.cc:276] 0709 12:26:41.208007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.241100 (+ 33093us) service_pool.cc:224] Handling call
0709 12:26:41.241426 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.241696 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:42.023002 (+781306us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.436060 (+413058us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.436118 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:42.436169 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.436303 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.436437 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":868368,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":11008,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":12935,"thread_start_us":1182872,"threads_started":2}
I20260709 12:26:42.610970  4756 raft_consensus.cc:2368] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15.
I20260709 12:26:42.612030  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:42.612589  4751 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:42.612627  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:42.613839  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:42.614650  4747 raft_consensus.cc:2410] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:26:42.615469  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:42.615984  4751 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:42.617625  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:42.618227  4756 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:42.618988  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:42.619751  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:42.619880  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:42.620348  4756 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:42.613250  4752 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:42.623528  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:42.624101  4752 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:42.620302  5348 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:26:42.628185  4747 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:42.573268  9607 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:42.638096  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:42.647344  4749 raft_consensus.cc:2368] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15.
I20260709 12:26:42.638247  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:42.639362  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:42.524240  9639 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 12:26:42.578740  9630 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:42.604532  9418 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:42.640004  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:42.593990  9567 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:42.609040  9569 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:42.654330  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 281081us
I20260709 12:26:42.671612  4750 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:42.623458  9634 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:42.608412  9555 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:42.608179  9354 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:26:42.609648  9337 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.629987  9466 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:42.608726  9358 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:42.685388  4751 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:42.685663  4756 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:42.682870  4719 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:42.742200  4748 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
W20260709 12:26:42.749922  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 154282us
I20260709 12:26:42.670773  9639 consensus_queue.cc:1243] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:26:42.742846  9606 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:26:42.669754  9431 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:42.763779  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 126995us
I20260709 12:26:42.752323  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:42.756537  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 73775.7us
W20260709 12:26:42.749064  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 273544us
I20260709 12:26:42.728325  9567 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:42.675405  9569 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:42.681001  9634 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:42.766745  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:42.767444  4571 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:42.777876  9632 log.cc:927] Time spent T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 12:26:42.627977  9616 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:42.762053  5340 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:26:42.797006  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 44769.2us
I20260709 12:26:42.806954  4569 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:42.811738  4569 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:42.778489  9569 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:42.769287  9641 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:42.753556  9466 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:42.815035  4569 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:42.780766  9622 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:42.749061  9337 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:42.851866  4569 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:42.830875  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:42.816249  9625 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:42.818537  9643 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:26:42.798278  9431 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:42.878069  9431 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:42.879343  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3098) took 2253 ms. Trace:
I20260709 12:26:42.880276  9431 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:42.819332  9641 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:42.881438  9641 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.882577  9500 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:42.863050  9355 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:42.819058  9661 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:42.867107  9393 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:42.869153  9611 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:42.868892  9663 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:42.869936  9666 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:26:42.870239  9642 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:42.870736  9657 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:42.871388  9653 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:42.873859  9634 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:42.875150  9466 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:42.875766  9423 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:42.875162  4754 raft_consensus.cc:2410] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:26:42.863085  9511 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:26:42.885054  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:42.885376  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:42.879727  4678 rpcz_store.cc:276] 0709 12:26:40.626129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.254443 (+1628314us) service_pool.cc:224] Handling call
0709 12:26:42.254851 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.259444 (+  4593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.259495 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.260121 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.555957 (+295836us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:42.555995 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.556060 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.879125 (+323065us) raft_consensus.cc:1685] finished
0709 12:26:42.879145 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.879285 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24636,"raft.queue_time_us":5455,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":21550,"thread_start_us":295517,"threads_started":1,"wal-append.queue_time_us":617335}
W20260709 12:26:42.897006  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 77536.4us
I20260709 12:26:42.900655  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:42.792771  9418 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:42.888262  9661 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.895533  9622 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:42.917220  9418 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:42.894963  9611 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:42.929781  9611 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:42.930315  9423 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:42.930940  9669 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:42.931299  9423 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:42.937892  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 754) took 2944 ms. Trace:
I20260709 12:26:42.938356  4679 rpcz_store.cc:276] 0709 12:26:39.993816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.062539 (+2068723us) service_pool.cc:224] Handling call
0709 12:26:42.062845 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.067143 (+  4298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.067174 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.067642 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.419385 (+351743us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:42.419426 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.419490 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.937656 (+518166us) raft_consensus.cc:1685] finished
0709 12:26:42.937681 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.937828 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3776,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":350174,"threads_started":1,"wal-append.queue_time_us":566863}
W20260709 12:26:42.940683  9603 consensus_meta.cc:287] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.002s
I20260709 12:26:42.940935  9337 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:42.825423  9344 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:42.915772  4569 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:26:42.944689  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.903299  9657 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:42.912456  9666 raft_consensus.cc:697] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:42.951177  9666 consensus_queue.cc:237] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:42.952121  9588 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:42.907670  9672 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:42.953869  9672 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.954241  9672 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:42.954485  9500 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:42.953184  9588 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:42.957687  9657 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:42.959931  9344 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:42.918956  9661 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:42.961814  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 309920us
I20260709 12:26:42.963392  9630 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:42.963725  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3097) took 2356 ms. Trace:
I20260709 12:26:42.964305  9630 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:42.908524  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:42.971200  4676 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 12. Current term is 13. Ops: [10.10-12.11]
W20260709 12:26:42.971547  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 228628us
I20260709 12:26:42.974733  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:42.975399  4753 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:42.964052  4674 rpcz_store.cc:276] 0709 12:26:40.607858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.228265 (+1620407us) service_pool.cc:224] Handling call
0709 12:26:42.228692 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.228963 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.229001 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.234145 (+  5144us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.555581 (+321436us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:42.670529 (+114948us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.963399 (+292870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.963477 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.963523 (+    46us) raft_consensus.cc:1685] finished
0709 12:26:42.963537 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.963669 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":306147,"prepare.queue_time_us":307452,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":293198,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":75,"thread_start_us":348428,"threads_started":2,"wal-append.queue_time_us":560740}
I20260709 12:26:42.977707  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:42.912909  9653 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:42.981813  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 214456us
I20260709 12:26:42.987583  4571 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:42.989244  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:42.943989  9634 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:42.994282  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 193292us
I20260709 12:26:42.980543  9358 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:42.998237  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:43.016067  4680 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:43.016300  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 764) took 2339 ms. Trace:
W20260709 12:26:43.002364  9646 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.332s	user 0.000s	sys 0.004s
W20260709 12:26:43.006812  9631 consensus_meta.cc:287] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.533s	user 0.007s	sys 0.001s
I20260709 12:26:43.012354  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:43.014638  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 98972.4us
I20260709 12:26:43.016709  4673 rpcz_store.cc:276] 0709 12:26:40.676559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.394325 (+1717766us) service_pool.cc:224] Handling call
0709 12:26:42.394667 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.394900 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:42.591975 (+197075us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.016020 (+424045us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:43.016074 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:43.016124 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.016183 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.016259 (+    76us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115082,"prepare.queue_time_us":363730,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"thread_start_us":490665,"threads_started":2,"wal-append.queue_time_us":349558}
I20260709 12:26:43.023360  5314 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:42.941179  9603 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:43.028448  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.029552  4569 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:26:43.037680  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 60170.8us
W20260709 12:26:43.037912  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 50464.4us
W20260709 12:26:43.038574  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.038853  9603 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:43.039414  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:43.039937  9679 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:43.040393  9679 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.040233  9589 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:43.040782  9679 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:43.040822  4571 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:43.035544  9500 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.041730  9393 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:43.042260  9625 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:43.042058  9500 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.042703  9625 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:43.043162  9625 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:43.044358  9500 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.035532  9631 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.050832  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:43.048987  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:43.054554  4597 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:43.048014  4569 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:43.056519  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 44281.5us
I20260709 12:26:43.059352  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:43.068020  4571 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:43.068195  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:43.073091  4748 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:26:43.073881  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:43.076246  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 45242.8us
I20260709 12:26:43.067788  4755 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:43.080964  4748 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:43.090951  4597 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:43.035538  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:43.052788  9431 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:43.072592  9264 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:26:43.047611  9414 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:26:43.095227  9676 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
W20260709 12:26:43.108073  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 55613.4us
I20260709 12:26:43.109720  9358 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.096122  4571 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:43.111961  9344 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.112479  9344 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.113502  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:43.097728  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:43.134620  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 111371us
I20260709 12:26:43.097059  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:43.028647  9611 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.139712  4749 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:43.072737  9466 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.149178  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:43.143385  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 45790.7us
I20260709 12:26:43.145702  4599 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:43.056929  9674 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:43.162886  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 49490.2us
I20260709 12:26:43.163816  4571 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:43.165103  4647 leader_election.cc:400] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15.
I20260709 12:26:43.116302  9628 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.166523  9628 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.169176  9628 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:43.145951  9618 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:26:43.160972  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2517) took 1073 ms. Trace:
I20260709 12:26:43.161168  9588 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:43.104809  9414 consensus_queue.cc:1243] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:26:43.095196  9683 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:43.175014  4600 rpcz_store.cc:276] 0709 12:26:42.087357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.088654 (+  1297us) service_pool.cc:224] Handling call
0709 12:26:42.088905 (+   251us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.094889 (+  5984us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:42.094932 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.095493 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.407247 (+311754us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:42.407285 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.407354 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.001739 (+594385us) spinlock_profiling.cc:243] Waited 5.07 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:43.146163 (+144424us) raft_consensus.cc:1685] finished
0709 12:26:43.146184 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.160876 (+ 14692us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5600,"raft.queue_time_us":75551,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"spinlock_wait_cycles":11666304,"thread_start_us":264382,"threads_started":1,"wal-append.queue_time_us":436379}
I20260709 12:26:43.178237  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:43.217744  9683 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:43.218184  9683 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:43.218304  5344 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:43.218607  9683 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:43.222193  9693 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.222748  9693 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.224888  9693 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:43.224433  9679 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.165946  9657 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:43.230019  4645 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:43.232142  9607 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:43.232822  9607 raft_consensus.cc:697] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:43.172509  9666 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:43.232162  9354 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.002s
W20260709 12:26:43.175344  9652 log.cc:927] Time spent T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:26:43.233783  9607 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:43.240154  9344 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.241776  9631 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.174566  4571 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:43.243373  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 68917us
I20260709 12:26:43.244822  4644 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:43.246264  4571 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:43.178879  4596 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:43.099145  9664 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:43.115489  9685 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:43.145738  9645 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:26:43.171842  9675 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:43.202378  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:43.277997  4599 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:43.178261  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:43.141422  9264 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:43.202838  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:43.206312  8984 raft_consensus.cc:3055] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:43.197852  9588 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.099320  9569 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:43.175072  9616 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:43.214334  9674 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:43.145687  9653 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:43.255085  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 195853us
I20260709 12:26:43.174108  9663 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:43.279330  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 101191us
I20260709 12:26:43.279913  9344 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:43.279244  9645 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:43.254642  9423 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:43.301848  4749 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:43.176819  9567 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.251230  9664 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:43.176571  9466 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:43.293220  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:43.283329  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 80613.8us
I20260709 12:26:43.307940  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:43.283325  9431 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:43.283712  9410 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:43.254910  9685 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.313210  9685 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:43.309724  9664 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:43.287497  9631 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:43.277451  9589 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:26:43.307024  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:43.331027  4720 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:43.289296  9616 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:26:43.288362  9588 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:43.288663  9628 consensus_queue.cc:1059] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:26:43.315244  9567 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:43.283469  8984 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:43.286367  9641 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:43.319415  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:43.301208  9645 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:43.357327  9692 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:43.358480  9692 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:43.358963  9692 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:43.297973  9663 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:43.327701  9431 raft_consensus.cc:697] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:43.361814  9663 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:43.362601  9431 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.363690  9672 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:43.365028  5345 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:26:43.332767  9616 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:43.336285  9667 raft_consensus.cc:3055] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:43.309942  9640 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:43.317191  9693 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:43.330485  9631 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:43.338824  9611 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.338910  9511 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:43.333710  9466 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:43.318598  9410 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.357684  9474 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:43.356936  8984 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.337945  9674 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.367659  9667 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:43.351428  9639 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:26:43.370991  9640 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:43.391376  9640 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:43.392490  9641 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:43.372603  9693 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:43.393312  9693 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:43.395736  9667 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:43.396509  9588 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.379083  9410 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:43.373992  9511 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:43.398176  9511 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:43.398804  9588 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.400431  9625 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.403462  9625 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.387380  9589 raft_consensus.cc:3055] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:43.390508  9639 raft_consensus.cc:3055] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:43.411855  9639 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:43.413197  9682 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.170s	user 0.000s	sys 0.003s
W20260709 12:26:43.405220  9686 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.163s	user 0.003s	sys 0.000s
I20260709 12:26:43.384444  9569 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:43.386229  9657 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:43.373656  9631 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:43.410801  9589 raft_consensus.cc:740] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:43.380266  9679 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.392649  9640 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:43.421209  9589 consensus_queue.cc:260] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.422449  9589 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:43.410955  9667 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:43.412768  9639 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:43.426584  9639 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:43.428026  9663 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.432673  9683 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.424927  9661 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.434360  9672 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.444882  9663 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.447999  9685 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.449807  9685 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.451929  9683 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:43.436002  9664 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.433008  9693 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.454757  9661 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:43.455448  9693 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:43.457005  9664 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.440397  9645 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.445852  9692 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:43.461241  9692 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:43.282192  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:43.490346  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:43.491196  4599 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:43.493958  4603 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:43.200304  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 123718us
I20260709 12:26:43.258190  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:43.292156  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:43.499784  8984 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:43.507797  9645 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:43.511761  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:43.512336  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.516916  4752 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:26:43.533185  4752 pending_rounds.cc:85] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 9
I20260709 12:26:43.533720  4752 pending_rounds.cc:107] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:26:43.534004  4597 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:43.534191  4752 raft_consensus.cc:2889] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:43.534888  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 226913us
I20260709 12:26:43.537643  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:43.538892  4750 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:26:43.539813  9667 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:43.540447  9667 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.540932  9667 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:43.518878  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:43.523284  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.335175  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:43.545408  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.536540  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:43.559115  9569 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:43.559821  9569 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:43.560326  9569 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:43.561044  9664 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:26:43.564193  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 305996us
I20260709 12:26:43.573678  9645 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:43.574542  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 217449us
I20260709 12:26:43.575651  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:43.577178  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:43.577407  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:43.549324  4645 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:43.578058  9692 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:43.586099  4750 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:43.586892  9555 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:43.583488  9692 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:43.594192  9639 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:26:43.590456  9692 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:43.618161  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:43.622188  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:43.623860  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:43.607169  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:43.615846  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:43.617036  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:43.633092  5348 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:43.640005  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:43.640499  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:43.642155  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:43.642652  4752 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:26:43.642864  4755 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:43.643190  4754 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:43.643886  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:43.644506  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:43.646574  4604 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:26:43.651656  9350 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:26:43.655393  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 126221us
I20260709 12:26:43.657001  9630 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:43.657351  9588 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:43.657598  9630 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.658140  9630 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:43.658185  9588 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.658977  9555 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:43.659751  9555 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:43.660964  9588 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:43.669811  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 51854.1us
W20260709 12:26:43.670125  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 133879us
I20260709 12:26:43.671473  9630 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.671727  9358 consensus_queue.cc:1059] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:26:43.677790  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 100792us
I20260709 12:26:43.678355  9358 raft_consensus.cc:3055] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:43.678687  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:43.678864  9358 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:43.679868  9634 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:43.680658  9634 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:43.682495  9683 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:43.683115  9683 raft_consensus.cc:697] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:43.683354  9634 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:43.684440  9683 consensus_queue.cc:237] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.680536  9358 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:43.687314  9358 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:43.670994  4647 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:43.699836  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 46626.3us
I20260709 12:26:43.688609  9555 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:43.710445  9555 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:43.710901  9555 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:43.711263  4647 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:43.712381  9588 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:43.712873  9588 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:43.713372  9588 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:43.714006  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:43.715802  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:43.713125  4647 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:43.711344  9555 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:43.770520  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 57588.7us
I20260709 12:26:43.772935  4647 leader_election.cc:400] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15.
I20260709 12:26:43.773902  9629 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:43.774677  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:43.777735  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:43.778611  5348 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:43.781798  9607 consensus_queue.cc:1059] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:43.782876  9607 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:43.784554  9628 raft_consensus.cc:3055] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:43.785033  9628 raft_consensus.cc:740] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:43.786230  9628 consensus_queue.cc:260] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.787947  9628 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:43.789745  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:43.790340  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:43.775019  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.791211  4599 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:43.791978  9630 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.798807  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:43.802057  4750 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:43.816952  4604 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:43.817991  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:43.823406  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:43.769292  9588 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.837136  9344 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.837322  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:43.837805  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:43.838721  4599 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:43.839136  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:43.846736  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:43.783490  9607 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.847340  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:43.849812  9607 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.850134  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:43.851686  9629 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:43.852578  9629 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.856329  9629 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.859186  9410 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:43.861320  4602 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:43.844141  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:43.865689  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:43.867595  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 94908us
I20260709 12:26:43.825484  9634 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.870255  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:43.846616  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:43.876183  5348 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:43.883236  4603 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:43.868641  9410 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:43.895462  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:43.896003  4756 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:43.902740  9641 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:43.903735  9641 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.904165  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:43.905750  9634 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.908087  9634 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:43.888950  9344 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.910985  9344 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:43.911684  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:43.912273  4751 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:43.918422  4597 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:43.918943  4600 raft_consensus.cc:2368] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
I20260709 12:26:43.919588  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:43.921272  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:43.922304  4681 raft_consensus.cc:2410] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:26:43.923131  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:43.923955  4681 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 15: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:43.897552  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:43.892243  4571 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:43.866219  9667 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:43.934871  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:43.950513  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:43.950830  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:43.951924  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:43.952310  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:43.952620  4748 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:26:43.955781  9474 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:43.956871  9474 raft_consensus.cc:3055] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:43.957216  9474 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:43.959151  4750 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:43.959599  4749 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 11: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:43.957365  9641 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:43.967828  9631 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:43.972594  9616 consensus_meta.cc:287] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.600s	user 0.014s	sys 0.000s
I20260709 12:26:43.975332  9692 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:44.032330  4747 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:44.033488  9692 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:44.033746  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 157342us
I20260709 12:26:44.034525  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:44.035925  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:44.037326  4680 raft_consensus.cc:2410] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:26:44.038738  9674 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:44.039959  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:43.973621  4748 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:44.042610  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:44.043432  4603 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:44.045233  4680 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:44.049768  9616 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:44.052045  9616 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:44.053931  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:44.053848  9474 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:44.055306  9474 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:43.994144  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:43.999818  4751 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:44.062618  4679 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:44.063735  4748 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 7.
I20260709 12:26:44.015158  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:44.017311  9667 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:44.018146  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 130205us
I20260709 12:26:44.069001  4647 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:44.019824  9589 consensus_meta.cc:287] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.597s	user 0.010s	sys 0.007s
I20260709 12:26:44.069314  5314 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:44.021018  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:44.070986  9511 consensus_meta.cc:287] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.672s	user 0.002s	sys 0.014s
I20260709 12:26:44.071087  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:44.071591  9511 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:44.072476  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:44.073141  4675 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:44.023861  4749 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:44.024742  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:44.026721  9631 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:44.027400  4719 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:43.976948  9569 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:44.075776  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:44.077150  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:44.034525  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:44.019857  9344 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:43.951001  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:44.075742  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 48541.8us
I20260709 12:26:44.079540  4754 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:44.086382  4672 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:44.091470  4678 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:44.092609  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:43.889699  9555 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:44.087944  9692 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:44.079850  9569 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:44.132282  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:44.138304  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.144930  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:44.147863  4675 raft_consensus.cc:2368] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12.
I20260709 12:26:44.149070  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:44.149735  5348 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:26:44.150995  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:44.157480  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:44.088244  9616 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:26:44.178593  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:44.175426  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:44.176483  5312 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:44.181512  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:44.184257  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 115392us
I20260709 12:26:44.142485  9692 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:44.174270  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:44.140125  9511 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:44.204881  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:44.211805  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:44.196173  9692 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:44.189742  4681 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:44.193606  9344 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:44.219156  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 184741us
I20260709 12:26:44.199826  9576 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:44.162487  9414 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.289s
I20260709 12:26:44.228178  4678 raft_consensus.cc:2393] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:26:44.204054  4720 connection.cc:256] RPC call timeout handler was delayed by -1.2148154090000389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 12:26:44.169852  4680 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:26:44.224575  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:44.206861  9588 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:44.231318  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:44.231405  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:44.231925  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:44.232556  5348 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:44.240391  4604 raft_consensus.cc:2393] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 7: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:44.235168  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:44.213624  9423 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:44.243721  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:44.218801  9569 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:44.253520  4681 raft_consensus.cc:2368] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 13.
W20260709 12:26:44.258050  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 81716.9us
I20260709 12:26:44.221992  9641 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:44.234081  9344 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:44.270512  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:44.275138  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 51314.6us
I20260709 12:26:44.275592  9431 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:26:44.276885  9663 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:44.277233  9207 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:44.277756  9663 consensus_queue.cc:1059] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:44.277864  9207 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:44.248039  9661 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:44.286762  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:44.290222  4674 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:44.264711  9576 raft_consensus.cc:697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:44.298225  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:44.321669  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 98279.3us
I20260709 12:26:44.311012  9569 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:44.326097  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 73065.4us
I20260709 12:26:44.310105  9555 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:44.300920  9641 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:44.340198  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:44.341071  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:44.337497  9513 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:44.347581  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:44.351059  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:44.354130  4679 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:44.355460  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:44.369820  9576 consensus_queue.cc:237] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:44.379103  9641 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:44.381235  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:44.383965  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:44.388752  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:44.390786  4676 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:44.392230  4677 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:44.394724  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:44.396621  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:44.397169  4676 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:44.398227  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 122285us
I20260709 12:26:44.403678  4681 raft_consensus.cc:2368] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 12. Current term is 13.
I20260709 12:26:44.404497  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:44.405112  4679 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:44.405385  4678 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:26:44.407687  4675 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:44.396495  9511 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:44.402947  9641 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:44.421725  4571 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:44.423825  9715 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:26:44.425002  4672 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:26:44.426579  4672 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:26:44.426862  9207 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:44.427816  4672 pending_rounds.cc:85] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 6
W20260709 12:26:44.428263  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 93927.6us
I20260709 12:26:44.428105  4672 pending_rounds.cc:107] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:26:44.430326  4672 raft_consensus.cc:2889] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:44.435510  5348 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEARNER]: Advancing to term 16
I20260709 12:26:44.436467  4647 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:44.444410  4680 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 12:26:44.446049  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 62222us
I20260709 12:26:44.450361  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:44.454133  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.437875  9589 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:44.400864  9344 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:44.457029  4680 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:44.459837  4755 raft_consensus.cc:2393] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 19: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:44.460310  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:44.472864  4677 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:44.475642  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:44.479854  9725 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:26:44.482736  4673 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:44.466174  9628 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.498281  4677 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:44.501796  4680 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:44.564165  9672 raft_consensus.cc:3055] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:44.564625  9672 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:44.568361  5348 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:44.571008  4679 raft_consensus.cc:2410] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:26:44.572638  4678 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:44.505475  9569 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:44.576670  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 116625us
I20260709 12:26:44.568718  9674 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:44.573670  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:44.584830  9653 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:44.574213  9555 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:26:44.582934  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:44.565424  9672 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:44.700469  4676 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:44.581943  9692 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:44.493510  9628 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:44.732327  4750 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 12:26:44.733521  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 56515.1us
I20260709 12:26:44.735124  4681 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:44.603156  9724 consensus_queue.cc:1059] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.304s
W20260709 12:26:44.739382  9716 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
W20260709 12:26:44.729700  9723 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:26:44.729400  9641 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:44.746730  5340 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
W20260709 12:26:44.753274  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 302651us
I20260709 12:26:44.732929  9569 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:44.761842  9724 consensus_queue.cc:1059] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:44.763725  9685 raft_consensus.cc:3055] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:44.765039  9685 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:44.766196  9685 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:44.768385  4571 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:44.769541  9674 consensus_queue.cc:1059] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:44.771539  9724 consensus_queue.cc:1059] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:26:44.781128  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 219336us
I20260709 12:26:44.754443  9653 raft_consensus.cc:3055] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:44.767904  9685 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:44.789073  9663 raft_consensus.cc:3055] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:44.789743  9663 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:44.791180  9663 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:44.792780  9663 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:44.831506  9661 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:44.832345  9661 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:44.851768  9721 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.442s	user 0.014s	sys 0.012s
I20260709 12:26:44.857445  9344 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:44.867578  9344 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:44.871491  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:44.872644  9631 consensus_queue.cc:1059] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:44.875044  9631 raft_consensus.cc:3055] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:44.875543  9631 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:44.878288  9667 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:44.882972  9589 consensus_queue.cc:1059] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:44.873656  9588 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:44.891389  9344 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:44.876799  9631 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:44.893385  9631 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:44.904918  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3118) took 1178 ms. Trace:
I20260709 12:26:44.905916  4674 rpcz_store.cc:276] 0709 12:26:43.737802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.758829 (+ 21027us) service_pool.cc:224] Handling call
0709 12:26:43.801064 (+ 42235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:44.320840 (+519776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.904505 (+583665us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b5400200b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:44.904788 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":806408960}
I20260709 12:26:44.756340  9569 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:44.914017  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 128072us
W20260709 12:26:44.915033  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.783448  9664 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:44.922426  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:44.787366  9653 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:44.925199  4603 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:26:44.917259  9576 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.061s	sys 0.066s
I20260709 12:26:44.916018  9569 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:44.779951  9641 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:44.931254  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:44.933972  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 179827us
W20260709 12:26:44.934438  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 252609us
I20260709 12:26:44.934966  9344 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.936399  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:44.973064  4674 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:44.936512  9555 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:44.939790  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:44.934859  9423 consensus_queue.cc:1059] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:26:44.963255  4599 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:44.966462  4602 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:44.970029  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 54017.1us
W20260709 12:26:44.969235  9576 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.061s	sys 0.067s
I20260709 12:26:44.974905  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:44.927886  9664 raft_consensus.cc:697] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:44.984012  9692 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:45.002926  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.005324  4647 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:44.994025  9423 raft_consensus.cc:3055] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:44.991413  9628 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:44.979539  9344 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:45.022393  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49621.6us
I20260709 12:26:45.023661  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:44.996261  9667 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:44.994971  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:44.983744  9555 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.017405  9423 raft_consensus.cc:740] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:45.069013  9419 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.069720  9419 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.070755  9672 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:45.072065  9419 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:45.072737  9733 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:45.025527  9628 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.034565  9724 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:45.084520  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 61007.4us
I20260709 12:26:45.085388  4569 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:45.085922  9735 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:45.086385  9735 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:45.086614  4569 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:45.087399  9735 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.087953  4569 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:45.089031  9735 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:45.075678  9667 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:26:45.091869  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 108539us
W20260709 12:26:45.092777  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.095242  4569 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:45.097785  9641 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:45.099432  9423 consensus_queue.cc:260] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.100823  9423 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:45.101804  9733 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:45.101683  9628 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.104238  9628 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:45.106382  9344 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:45.109112  9555 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:45.093734  9667 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.109714  9736 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:45.110985  9667 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:45.014153  9653 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.117453  4674 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
W20260709 12:26:45.120070  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 138581us
I20260709 12:26:45.115767  9431 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:45.130926  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:45.127077  9653 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:45.134105  9674 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:45.134519  9674 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:45.135330  9674 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.136543  9674 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:45.094115  9664 consensus_queue.cc:237] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:45.129280  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.120669  9736 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:45.147027  9724 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:45.143043  9739 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:45.149703  9207 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.150316  9207 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.158869  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:45.160725  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:45.161485  4750 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:45.162038  4569 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:45.163414  9735 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:45.163578  4569 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:45.164304  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:45.164659  9724 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:45.165246  9724 raft_consensus.cc:697] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:45.165843  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:45.166270  9724 consensus_queue.cc:237] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.168588  4750 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:45.138392  9661 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.171504  9661 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:45.177372  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3129) took 1255 ms. Trace:
I20260709 12:26:45.189190  4756 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:45.177166  9735 consensus_queue.cc:1059] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:45.198850  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:45.177932  9631 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:45.178591  4569 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:45.190887  4681 rpcz_store.cc:276] 0709 12:26:43.921440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.406418 (+484978us) service_pool.cc:224] Handling call
0709 12:26:44.421260 (+ 14842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:44.735587 (+314327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.177118 (+441531us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b5400085a54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:45.177306 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":721264000}
I20260709 12:26:45.216374  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:45.218116  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:45.219345  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:45.220724  4748 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:45.221524  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:45.222348  4749 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:45.223027  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:45.223734  4747 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 15: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:45.228843  4754 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:45.233773  4751 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:45.233875  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:45.227448  9667 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.236692  9569 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.237829  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:45.238580  4755 raft_consensus.cc:2393] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 16: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:45.239377  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:45.240110  4753 raft_consensus.cc:2393] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 13: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:45.217355  9423 consensus_queue.cc:1059] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:26:45.241979  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:45.242715  4756 raft_consensus.cc:2393] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:45.244079  9667 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:45.244719  9569 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:45.244782  9667 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:45.245744  9589 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:45.247905  9589 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:26:45.250118  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 119259us
I20260709 12:26:45.252004  4720 leader_election.cc:400] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
I20260709 12:26:45.252314  4720 leader_election.cc:403] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:45.253772  9661 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
W20260709 12:26:45.254894  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 76404us
I20260709 12:26:45.255110  9628 raft_consensus.cc:3055] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:45.255612  9628 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:45.256651  4569 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:45.258221  4752 raft_consensus.cc:2393] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 13: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:45.258518  9733 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:45.259415  9733 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:45.259456  9264 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:45.259925  9733 raft_consensus.cc:697] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:45.260597  9264 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:45.260999  9264 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:45.261834  9431 raft_consensus.cc:3055] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:45.262226  9431 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:45.263060  9431 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.264348  9431 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:45.265072  9264 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.267690  9588 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:45.268121  9588 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:45.269148  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:45.266769  9264 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:45.269940  4599 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:45.270211  9423 raft_consensus.cc:3055] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:45.270627  9423 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:45.271462  9423 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.268885  9588 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.273520  9588 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:45.274315  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:45.274578  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:45.264053  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:45.275254  4605 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:45.275710  4602 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:45.276432  4600 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:45.277029  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:45.277697  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:45.278021  4599 raft_consensus.cc:2410] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:45.258111  9736 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:45.281383  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:45.291268  4645 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:45.293937  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:45.294836  4676 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:26:45.256610  9628 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.153378  9207 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:45.302701  4645 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:45.306643  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:45.306846  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:45.318367  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:45.326164  5345 raft_consensus.cc:2410] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:45.260921  9733 consensus_queue.cc:237] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.329736  4600 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:26:45.330156  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48871.2us
I20260709 12:26:45.333467  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.335852  4644 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:45.337105  4678 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:45.302318  9667 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:45.341773  4598 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:45.311832  9423 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:45.343199  4753 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:45.313294  9344 raft_consensus.cc:2764] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:26:45.345101  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:45.349815  4755 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:45.297642  9653 consensus_queue.cc:1059] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:26:45.292384  9631 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.326635  9739 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:45.356391  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.357546  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:45.360294  4571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.029s	sys 0.026s
W20260709 12:26:45.360610  4571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.029s	sys 0.026s
W20260709 12:26:45.360908  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 574461us
I20260709 12:26:45.358799  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:45.342660  9667 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.345777  9555 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:45.378149  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:45.361477  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:45.380059  4600 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:45.363906  9576 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.399637  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:45.400054  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:45.360767  9740 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:45.406562  4598 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
W20260709 12:26:45.436673  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.385134  9667 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:45.403321  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:45.467051  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:45.468366  4754 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:45.469447  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:45.472455  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 111024us
I20260709 12:26:45.465173  4755 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
W20260709 12:26:45.486163  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.470256  4753 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:45.487325  4752 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:45.507507  9740 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.508359  9740 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:45.518810  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 238623us
W20260709 12:26:45.522192  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.528066  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 131808us
I20260709 12:26:45.530615  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:45.384615  9739 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.489848  9735 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:45.535422  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:45.536320  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:45.491309  9589 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:45.545410  9589 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.538699  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:45.549296  9589 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:45.551971  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:45.556362  4605 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:45.538698  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:45.571624  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:45.573001  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:45.574038  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:45.575064  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:45.543790  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:45.557315  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:45.576411  4672 raft_consensus.cc:2410] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:26:45.557732  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:45.457628  9576 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.576941  4676 raft_consensus.cc:2393] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:45.564699  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:45.577924  4674 raft_consensus.cc:2410] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:26:45.530026  9555 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:45.566510  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:45.580031  4673 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 15: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:45.579092  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:45.582386  4602 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:45.569658  9641 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:45.534133  5312 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:45.585829  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:45.591404  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 233993us
I20260709 12:26:45.596156  4605 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:45.596578  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.597184  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:45.591976  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 57961.1us
I20260709 12:26:45.599491  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:45.589692  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 67735us
I20260709 12:26:45.536923  9674 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
W20260709 12:26:45.595570  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 137808us
I20260709 12:26:45.583523  9661 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:45.609596  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:45.600687  9672 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:45.627159  4602 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
W20260709 12:26:45.636023  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.614149  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:45.642130  4605 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
W20260709 12:26:45.617609  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.599041  9603 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:45.632688  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:45.584702  9653 consensus_queue.cc:1243] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:45.667264  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:45.673686  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 83011.9us
I20260709 12:26:45.666869  4675 raft_consensus.cc:2393] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:45.589411  9757 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.669754  4647 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.677788  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.678898  4647 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.680538  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.676379  4601 raft_consensus.cc:2393] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:26:45.676316  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 68699.8us
I20260709 12:26:45.613336  9751 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.625996  9701 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:45.623093  9344 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:45.686959  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:45.695071  5345 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:45.695173  9344 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:45.696687  4644 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:45.628100  9760 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:45.596518  9641 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.693043  9753 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:26:45.694053  9745 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:26:45.676939  9569 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:45.720312  4754 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
W20260709 12:26:45.694770  9741 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:26:45.634109  9423 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
W20260709 12:26:45.695174  9746 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:26:45.730257  4756 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:45.630182  9754 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:26:45.636431  9661 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.685985  9631 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:45.747438  4753 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:45.748250  4750 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:45.683007  9589 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:45.675765  9555 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:26:45.730732  9731 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.688875  9701 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.720207  9735 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:45.760599  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.755151  4751 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:26:45.762842  9555 raft_consensus.cc:3055] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:45.701766  9737 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:26:45.751998  9576 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:45.766178  9752 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:45.722623  9760 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.775288  9641 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:45.775763  9749 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:45.806375  9767 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:26:45.782744  9731 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.811861  9731 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:45.797610  9663 raft_consensus.cc:3055] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:45.817446  9735 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.819227  9760 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:45.698016  9751 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:45.788342  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 107870us
I20260709 12:26:45.824569  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.783367  9701 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:45.779580  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.684653  9757 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.826059  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.784274  9555 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:45.829694  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 228321us
W20260709 12:26:45.830142  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 156553us
I20260709 12:26:45.838696  5312 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:45.839524  9207 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:45.839347  9419 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:45.840054  9207 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.840515  9207 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:45.821043  9431 consensus_meta.cc:287] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.555s	user 0.005s	sys 0.006s
I20260709 12:26:45.793798  9752 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:45.813555  9663 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:45.835492  9749 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:45.845296  9767 consensus_queue.cc:1243] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:45.848374  4751 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:26:45.851747  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:45.857257  4678 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:45.862051  4681 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 12:26:45.862073  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 82613.3us
I20260709 12:26:45.847007  9774 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:45.852206  4748 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:45.864104  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:45.845947  9603 raft_consensus.cc:697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:45.849136  9663 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.863016  9667 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.858532  9661 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:45.887598  9603 consensus_queue.cc:237] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.864773  9207 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.858533  9754 consensus_queue.cc:1059] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:26:45.857893  4673 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:45.860735  4679 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:45.877971  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.867138  4647 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.892921  4647 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:45.859602  9588 consensus_meta.cc:287] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.000s
I20260709 12:26:45.894515  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.845044  9589 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:45.899376  9589 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:45.860248  9752 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:45.847975  9664 consensus_queue.cc:1059] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:26:45.878608  9737 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:26:45.902350  9207 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:45.847795  9751 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:45.851795  9423 consensus_queue.cc:1059] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:26:45.848891  9769 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:26:45.907871  9757 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:45.859053  9749 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.858736  9739 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:45.865108  9672 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:45.928498  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:45.862350  9770 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:45.917089  9589 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:26:45.889349  9667 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:45.941968  9769 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:45.942678  9769 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.916103  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:45.898258  9631 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:45.907059  9742 raft_consensus.cc:3055] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:45.918972  4647 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:45.897991  9661 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:45.962986  9742 raft_consensus.cc:740] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:45.964390  9770 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:45.964252  9742 consensus_queue.cc:260] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:45.965802  9664 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:45.919587  9776 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:45.972424  9701 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.900853  9774 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:45.922053  9749 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:45.914089  4680 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:45.929703  9685 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:26:45.973951  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.922693  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:45.974822  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 55924.7us
I20260709 12:26:45.984601  9631 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:45.986696  9742 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:45.986804  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 70783.3us
I20260709 12:26:46.002442  9667 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:45.993197  9781 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:46.014447  4647 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:46.023379  9781 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:46.024262  4647 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:46.025389  9753 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:46.025890  9753 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.026064  9344 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:46.003435  9740 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:46.026257  9753 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:46.008455  9737 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.987737  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:45.992357  9769 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:46.008935  9588 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:46.015792  9631 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:46.028329  9628 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:46.029109  9569 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:46.034384  9631 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:46.045656  9780 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:26:46.050936  9774 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:46.055213  9701 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:46.056389  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:46.057696  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:46.058890  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:46.060499  9751 raft_consensus.cc:2764] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:46.061195  9751 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:46.061666  9751 raft_consensus.cc:697] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:46.028949  9667 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:46.062539  9751 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.063395  9739 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:46.064016  9739 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:46.064395  9667 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:46.066423  9765 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:46.066774  4720 leader_election.cc:400] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12.
I20260709 12:26:46.069811  4720 leader_election.cc:403] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:46.070461  9765 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:46.071173  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:46.072631  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:46.075187  9569 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:46.071237  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:46.024688  9781 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.035658  9588 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.081547  4751 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:26:46.079857  9692 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:46.079869  9641 raft_consensus.cc:2764] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:26:46.045189  9752 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:46.090773  9749 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.080782  9776 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:46.033437  9737 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.114291  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:46.174700  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:46.259536  4596 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:26:46.250200  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 226013us
I20260709 12:26:46.116824  4749 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:46.127683  4752 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:46.240280  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 54372.6us
I20260709 12:26:46.174036  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:46.267902  4599 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
W20260709 12:26:46.199669  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.284013  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:46.341318  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:46.279520  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:46.158020  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:46.359885  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 494370us
I20260709 12:26:46.601189  4601 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:26:46.102293  9739 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:46.158502  4496 catalog_manager.cc:5697] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:46.165839  9589 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.122071  9631 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:46.155128  9663 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:46.376438  9736 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.028s	user 0.014s	sys 0.034s
I20260709 12:26:46.128026  9667 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:46.156132  9740 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:46.093453  9753 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.605397  4753 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:26:46.104583  9264 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:46.173552  9431 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.607678  4750 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:46.105894  9781 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:46.622112  9736 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.277s	user 0.019s	sys 0.040s
I20260709 12:26:46.610333  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:46.611529  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:46.611791  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:46.612357  9588 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:46.599856  4749 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:46.603753  4756 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
W20260709 12:26:46.556731  9344 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.024s	sys 0.040s
I20260709 12:26:46.616628  4751 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:46.622471  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:46.673561  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:46.107841  9555 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:46.636691  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:46.639472  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:46.602708  9776 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:46.640547  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.610265  4675 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:46.641360  4672 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:46.623211  9661 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:46.648241  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:46.640709  9765 consensus_meta.cc:287] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.000s
I20260709 12:26:46.675724  5345 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
W20260709 12:26:46.624305  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 510028us
I20260709 12:26:46.686165  4602 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:46.631274  9742 consensus_meta.cc:287] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.610s	user 0.004s	sys 0.005s
I20260709 12:26:46.701329  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:46.648192  9628 consensus_meta.cc:287] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.604s	user 0.009s	sys 0.000s
I20260709 12:26:46.694525  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:46.645375  5347 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:46.698732  4679 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:46.635286  9264 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:46.675590  9344 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.024s	sys 0.040s
I20260709 12:26:46.621740  9589 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.648942  9736 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:46.717149  4568 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:46.721194  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:46.726511  4756 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:46.727052  4752 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:46.746899  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:46.750101  9752 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:46.647487  9740 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:46.752677  9740 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:46.764334  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 47270.8us
W20260709 12:26:46.765036  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 63794us
I20260709 12:26:46.643774  9431 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.713644  9724 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.712029  9765 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12.
I20260709 12:26:46.721755  9742 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.778630  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:46.717248  9344 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:46.716270  9739 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:46.770725  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:46.673844  9667 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:46.785916  4675 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:46.722733  9350 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:46.790992  9736 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.791644  4720 leader_election.cc:400] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 13.
I20260709 12:26:46.792776  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:46.764312  9264 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.793699  9350 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:46.794587  9264 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:46.796196  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.704222  9776 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:46.810009  9350 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:46.822875  9589 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:46.826661  5344 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:46.827687  5344 raft_consensus.cc:606] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:46.779342  9555 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:46.783407  9767 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:46.832424  4681 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:46.834411  9767 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:46.835857  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 2009) took 1311 ms. Trace:
I20260709 12:26:46.836164  9767 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:46.836433  4600 rpcz_store.cc:276] 0709 12:26:45.524096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.524249 (+   153us) service_pool.cc:224] Handling call
0709 12:26:46.833954 (+1309705us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b5400034654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:46.835740 (+  1786us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1074696064}
I20260709 12:26:46.841305  9664 raft_consensus.cc:3055] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:46.785907  9724 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.788693  9742 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:46.865360  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 72672.8us
I20260709 12:26:46.875334  4602 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:46.878389  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:46.879676  4754 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 16: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:46.859603  9664 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:46.881126  9569 consensus_meta.cc:287] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.805s	user 0.008s	sys 0.004s
I20260709 12:26:46.878441  9762 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.881651  9569 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:46.881876  9762 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.883847  9792 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.884333  9783 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.884521  9653 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:46.884912  9783 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:46.885082  9653 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:46.886473  9431 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:46.887029  9431 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:46.887452  9783 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:46.888636  9749 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:46.888960  9783 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:46.889652  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:46.858089  9740 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.889966  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:46.890403  9740 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.892799  9740 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:46.895398  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:46.896329  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 257896us
I20260709 12:26:46.898211  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:46.899214  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:46.900867  4756 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:46.885383  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:46.857893  9628 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.902877  4601 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:46.905696  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:46.887334  9653 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:46.897094  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:46.918375  4675 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:46.919451  5344 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:46.925061  4754 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:46.926369  4679 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:46.927004  4753 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:46.923704  5344 raft_consensus.cc:606] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:26:46.947829  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 50809.2us
I20260709 12:26:46.879050  9739 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:46.893409  9762 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:46.903606  9765 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:46.905066  9733 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:46.962740  4675 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:46.906716  9737 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:46.919554  9751 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:46.907447  9569 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:46.968533  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:46.887540  9431 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:46.957827  9576 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:46.965166  9794 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:46.964679  9749 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:46.962430  9589 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.975684  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:46.972195  9740 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:46.994634  9740 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:46.995323  9740 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:46.904729  9664 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.978955  9768 raft_consensus.cc:3055] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:46.997980  9768 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:46.979290  4680 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:26:47.001832  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 129783us
I20260709 12:26:46.884382  9792 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.919660  9628 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.906584  9344 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.987339  9589 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:46.964818  9762 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:46.988137  9776 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:46.989724  9742 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:46.964782  9787 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.010918  4679 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:46.971182  9753 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:46.980051  9576 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:47.028298  4645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.001s	sys 0.016s
I20260709 12:26:46.982456  9733 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:46.964622  9739 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:47.041446  4645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.001s	sys 0.017s
I20260709 12:26:47.042689  9742 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:26:47.037535  4596 consensus_meta.cc:287] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.777s	user 0.012s	sys 0.002s
I20260709 12:26:47.043785  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:47.045719  4596 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
W20260709 12:26:47.050297  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 870691us
I20260709 12:26:47.098276  9767 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.083822  9739 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:47.011045  9794 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:47.132321  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 172648us
I20260709 12:26:47.055920  9667 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:47.094883  9792 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.043454  9733 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.040725  9576 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:47.038749  9787 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.072381  9742 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:47.042220  9789 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.152s	user 0.005s	sys 0.000s
I20260709 12:26:47.034880  9762 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.077353  9431 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:47.071017  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:46.999329  9768 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.206384  9628 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.002153  9749 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.085089  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:47.079851  4496 catalog_manager.cc:5697] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 10 to 12, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 12 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:47.168115  4645 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:47.037173  9631 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:47.086913  9589 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.126636  9724 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.193931  9776 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.168496  9661 raft_consensus.cc:697] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:26:47.199770  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 301644us
I20260709 12:26:47.218180  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 810) took 1045 ms. Trace:
I20260709 12:26:47.202620  9757 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:47.199764  4644 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:47.200793  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 231758us
W20260709 12:26:47.206884  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 284403us
I20260709 12:26:47.064143  9751 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:47.216521  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48495.4us
W20260709 12:26:47.237962  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:47.237918  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 175527us
W20260709 12:26:47.244236  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 275656us
I20260709 12:26:47.245121  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:47.238485  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.251204  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:47.219372  9739 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:47.235673  9768 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:47.212381  9784 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:26:47.235301  9628 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:47.250067  5312 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:47.259366  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:47.259976  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:47.260345  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:47.266999  4752 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:26:47.267172  4747 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:47.262287  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:47.269407  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.242774  9724 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.275148  4672 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:26:47.276188  4680 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:47.277271  4679 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:26:47.282416  4679 pending_rounds.cc:85] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 7
I20260709 12:26:47.282685  4679 pending_rounds.cc:107] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:26:47.282915  4679 raft_consensus.cc:2889] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:47.240298  9757 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:47.280121  4747 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:47.280910  5312 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:47.280977  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:47.282439  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:47.283869  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.284574  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.249872  9631 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:47.235889  4598 rpcz_store.cc:276] 0709 12:26:46.172488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.172666 (+   178us) service_pool.cc:224] Handling call
0709 12:26:46.173206 (+   540us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:46.173802 (+   596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:46.173849 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:46.339510 (+165661us) log.cc:844] Serialized 52 byte log entry
0709 12:26:47.001493 (+661983us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:47.209852 (+208359us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.217765 (+  7913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.217861 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:47.217916 (+    55us) raft_consensus.cc:1685] finished
0709 12:26:47.217934 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.218111 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":179396,"prepare.queue_time_us":83274,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":6828,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":28024,"spinlock_wait_cycles":16018688,"thread_start_us":546530,"threads_started":2,"wal-append.queue_time_us":501946}
I20260709 12:26:47.266664  4756 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:47.287372  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:47.261211  9815 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:47.235328  9431 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:26:47.288925  9799 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:26:47.261445  9264 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:26:47.290531  9762 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:47.291170  9793 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.291754  9793 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.295608  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:47.296856  9603 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:47.297314  9603 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.297681  9603 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:47.298368  9733 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:47.298076  9661 consensus_queue.cc:237] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.260861  9569 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:47.250164  9589 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:26:47.300632  9749 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:47.300938  9810 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:47.302107  9787 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.278740  9792 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:26:47.255000  9344 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.306283  9794 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.239408  9776 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.307838  9753 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.308009  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:47.308388  9753 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.308729  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:47.307291  9603 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.309921  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:47.310509  9753 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.310778  9603 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.311270  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:47.261657  9805 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:26:47.312024  5343 raft_consensus.cc:2368] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16.
I20260709 12:26:47.312860  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:47.313251  4676 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:47.307293  9474 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:26:47.314121  9628 raft_consensus.cc:697] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:47.315102  9628 consensus_queue.cc:237] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.318521  9264 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:47.318984  9264 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.319363  9264 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:47.321898  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 77332.2us
I20260709 12:26:47.305976  9344 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.285053  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:47.324671  4673 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:47.262451  9736 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.334755  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:47.286509  4750 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:47.335355  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:47.331453  9793 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.331684  9776 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:47.323274  9740 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.327162  9724 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:47.335646  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 54812.3us
I20260709 12:26:47.330837  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.322750  4720 leader_election.cc:400] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 12. Current term is 13.
I20260709 12:26:47.350066  4720 leader_election.cc:403] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:47.335378  4673 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:26:47.339570  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.339828  9799 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.342399  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:47.343978  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:47.345463  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:47.345986  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:47.352888  4679 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:47.352926  4676 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:47.346100  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:47.350304  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:47.355237  4672 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:26:47.323616  9631 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.357498  9796 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:47.357604  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:47.359447  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:47.355927  9751 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:47.360450  9804 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:47.361442  9804 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:47.362085  9804 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:47.362401  9804 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:47.355726  9414 raft_consensus.cc:993] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:26:47.336930  9815 raft_consensus.cc:697] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:47.364132  9796 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:47.364825  9815 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.365218  9796 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.365347  9431 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.366436  9631 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:47.366576  9431 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.367908  9414 raft_consensus.cc:1081] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:26:47.368896  9431 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.347146  9775 consensus_queue.cc:1059] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:26:47.364184  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:47.370949  4752 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:47.371794  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:47.363464  9804 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.372190  4720 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:47.373062  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:47.375811  4672 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:26:47.385165  4749 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:47.370549  9775 raft_consensus.cc:3055] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:47.401541  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:47.403743  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.404418  4600 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:47.338505  9736 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.408634  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:47.385103  9733 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.411593  4675 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:47.411748  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:47.419275  4752 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:26:47.414750  5312 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:47.400542  9794 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.422247  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:47.422655  5345 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:47.422835  4600 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:47.399076  4750 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:42632
I20260709 12:26:47.423763  4750 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:47.424129  4750 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:47.401027  9775 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:47.466881  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:47.407217  9804 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:47.371140  9799 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.552305  4597 raft_consensus.cc:2393] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:47.558133  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:47.422071  9739 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:47.554862  9788 raft_consensus.cc:1064] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:26:47.572500  4599 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:47.466575  9474 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:47.468554  9794 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.552102  9628 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.581161  4600 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:47.581760  4750 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.555473  9414 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:26:47.591629  9794 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:47.420408  9733 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.592417  9628 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.601917  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.597810  9474 raft_consensus.cc:697] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:47.604225  9474 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.604733  9775 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.587809  9740 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:47.613531  4750 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:47.617547  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 380707us
I20260709 12:26:47.618832  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:47.619354  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 207682us
I20260709 12:26:47.621002  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:47.624096  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:47.632144  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.638137  9776 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.638374  9555 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:47.638814  9776 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.639149  9555 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.640324  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:47.641000  9776 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:47.642191  9825 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:26:47.643338  9801 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.643911  9801 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.654352  4753 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:47.641283  9555 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.657970  9835 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:47.659454  9264 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.629107  9735 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.629240  9793 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:47.668053  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:47.646138  9787 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:47.669209  5348 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:26:47.632803  9834 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:47.671609  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.674023  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.675509  9837 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:47.676148  9837 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.678560  9837 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:47.662887  9804 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.710393  9804 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.681766  9264 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.712816  9804 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:47.698603  9576 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.699900  9736 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:47.700440  9834 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:47.700908  9344 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.701133  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:47.702054  9799 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.703984  9733 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:47.705440  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.770586  4568 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:47.670291  9735 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.777329  9781 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:47.778113  9735 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.724552  4496 catalog_manager.cc:5697] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:47.778661  4600 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:47.670727  9787 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.779454  9787 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:47.779654  9757 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.780793  9757 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:47.784967  9796 raft_consensus.cc:993] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:26:47.785595  9796 raft_consensus.cc:1081] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:26:47.735364  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.735836  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.787073  4597 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:47.787717  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.787896  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.736287  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.788429  4597 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:47.788779  4604 raft_consensus.cc:2410] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:26:47.789604  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.790401  4604 raft_consensus.cc:2410] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:47.789254  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.791214  4597 raft_consensus.cc:2410] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:47.791082  4604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:26:47.790624  9788 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.791651  4604 raft_consensus.cc:606] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:47.737344  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.792317  4604 raft_consensus.cc:614] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:47.694581  9828 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:26:47.759941  5348 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:47.762780  4644 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:47.764542  9834 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.775240  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:47.736821  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.792387  4597 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:26:47.738200  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.747174  9740 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.797950  4644 leader_election.cc:400] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16.
I20260709 12:26:47.802460  4644 leader_election.cc:403] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:47.803544  4644 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:47.768796  9801 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.798367  4674 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 19.
I20260709 12:26:47.798605  4597 pending_rounds.cc:85] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 9
I20260709 12:26:47.806037  4597 pending_rounds.cc:107] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:26:47.806408  4597 raft_consensus.cc:2889] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:47.805274  5312 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:47.774936  9768 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.004s
I20260709 12:26:47.794477  4568 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:47.815897  4601 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:47.810108  9752 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16.
I20260709 12:26:47.809294  4644 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:47.822391  9674 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:47.822880  9674 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.823288  9674 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:47.824990  5348 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:47.825693  5348 raft_consensus.cc:606] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:47.801967  9344 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:47.829226  4645 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:47.832549  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:47.835075  4596 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:26:47.835661  4596 pending_rounds.cc:85] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 9
I20260709 12:26:47.835925  4596 pending_rounds.cc:107] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:26:47.836261  4596 raft_consensus.cc:2889] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:47.812588  9793 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:47.865641  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.834542  9801 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:47.810714  9740 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:47.905143  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 284187us
I20260709 12:26:47.874325  9589 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:47.849764  9733 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.907253  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:47.830891  9752 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:47.947603  9739 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:47.944999  9589 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:47.949026  9631 consensus_meta.cc:287] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.582s	user 0.012s	sys 0.000s
I20260709 12:26:47.874691  9555 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
W20260709 12:26:47.949326  9751 consensus_meta.cc:287] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.588s	user 0.012s	sys 0.000s
I20260709 12:26:47.958546  4604 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:47.908452  9801 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.951524  9740 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.962689  4596 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:26:47.965508  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:47.969964  4596 raft_consensus.cc:606] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:47.970221  4597 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:47.970320  4596 raft_consensus.cc:614] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:47.908483  9807 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:47.977921  9768 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.978554  9768 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:47.980727  9768 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:47.946305  9344 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.964347  9752 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:47.994282  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 347495us
I20260709 12:26:47.965396  9589 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:47.964612  9739 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:47.967137  9628 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:48.001201  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:47.999341  9414 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:47.971359  9801 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:48.004245  4720 connection.cc:256] RPC call timeout handler was delayed by -1.2303944639998008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
I20260709 12:26:48.014678  4747 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 19.
W20260709 12:26:48.015511  4720 connection.cc:256] RPC call timeout handler was delayed by -1.1619944639996902s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
W20260709 12:26:48.016562  4720 connection.cc:256] RPC call timeout handler was delayed by -1.1169944639996174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
W20260709 12:26:48.017642  4720 connection.cc:256] RPC call timeout handler was delayed by -1.0269944639999267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1726s for a response
I20260709 12:26:47.982182  9751 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 12. Current term is 13.
I20260709 12:26:47.977986  9740 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:47.996222  9752 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:47.972700  9631 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:47.949497  9733 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.003472  9739 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.016757  9765 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:48.018034  4720 connection.cc:256] RPC call timeout handler was delayed by -1.0260944639999252s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
I20260709 12:26:48.036656  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:48.034296  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 201843us
W20260709 12:26:48.033267  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 142995us
I20260709 12:26:47.996018  9344 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.002866  9674 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:48.047783  4645 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:48.051445  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:48.057433  4645 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:48.059309  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:48.061355  4645 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:48.032482  9740 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:48.067734  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:48.058563  9840 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:48.040791  9765 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.060441  9555 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:48.060194  9800 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:48.061477  9667 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:48.053249  9793 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:48.114276  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 46636.6us
I20260709 12:26:48.069145  9751 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:48.115808  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:48.070170  9628 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:48.118235  5348 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:48.121670  9840 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:48.122313  9840 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:48.122627  9631 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:48.123117  9733 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:48.123335  9840 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.125815  9839 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:26:48.112978  9555 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:48.129954  9555 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:48.130776  9787 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.133064  9787 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:48.135196  9344 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:48.136087  9751 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:48.116526  9667 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.126881  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:48.117679  9804 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:48.140982  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:26:48.149348  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.155256  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:48.160595  5340 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:48.125252  9768 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:48.141502  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 82553.9us
I20260709 12:26:48.161010  5340 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:48.159868  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:48.158011  5345 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.170058  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.174556  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:48.144153  9667 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:48.182500  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:48.184439  9765 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:48.184547  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.185117  9751 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:48.186182  9835 consensus_meta.cc:287] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.001s
I20260709 12:26:48.186223  4603 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:26:48.190188  4571 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:48.192936  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 65755.7us
I20260709 12:26:48.117177  9849 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:26:48.142253  9787 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:48.203747  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.201713  4571 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:48.202456  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:48.206935  4747 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:48.176846  9733 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:48.208747  4748 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:48.167424  9674 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:48.213915  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:48.223189  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:48.225060  4747 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:48.231163  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 55463.6us
I20260709 12:26:48.235299  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:48.199690  9751 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:48.203328  9781 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:48.215086  9799 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:48.240159  5345 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:48.241274  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.249545  4748 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:48.250049  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.242071  4675 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:26:48.174363  9768 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.206499  9835 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:48.246991  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.239672  4674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:48198
I20260709 12:26:48.209936  9757 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:26:48.246347  9751 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:48.254006  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.216607  9733 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.235343  9800 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:48.262602  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.243033  9834 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:48.240168  9765 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:48.250964  4644 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:48.266187  9264 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:48.245399  9781 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.215984  9787 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:48.248741  9799 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.277309  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.240669  9739 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.280979  4755 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:48.300406  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:48.267041  9674 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.215298  9793 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:48.295571  9781 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.311817  4749 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:48.279042  9733 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:48.297504  9799 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:48.298532  9264 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:48.315651  9752 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:48.317757  9810 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.320606  9739 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:48.297744  9849 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:48.321756  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.297222  9800 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:48.326133  4681 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:26:48.327793  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:48.334221  4676 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:48.295230  9801 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:48.268088  9740 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:48.298974  9765 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.342634  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:48.348289  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:48.350417  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:48.351336  4676 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:48.353338  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:48.358174  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:48.358884  4681 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.362885  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:48.363233  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:48.363865  4673 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
W20260709 12:26:48.365270  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.365549  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:48.366442  4673 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:48.377669  5344 raft_consensus.cc:2393] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 20: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:48.386212  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:48.329015  9674 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.386965  4677 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:48.379652  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:48.387393  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:48.393188  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 119435us
I20260709 12:26:48.393368  4678 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:48.338595  9849 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:48.333703  9810 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.418742  4677 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:48.400718  9804 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:48.419999  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:48.422053  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:48.421538  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 53864.8us
I20260709 12:26:48.420682  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.432406  9871 consensus_queue.cc:1059] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:48.433738  4747 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:48.434221  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.434808  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.435511  4747 raft_consensus.cc:2368] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16.
I20260709 12:26:48.435382  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.435644  4677 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:48.437033  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:48.437824  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:48.438383  4751 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:48.439780  4681 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:48.442018  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.442646  4754 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:48.446511  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.448191  9876 raft_consensus.cc:3055] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:48.448773  9876 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:48.450078  9876 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.451779  9876 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.440326  9667 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:48.453238  9858 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:26:48.453500  9264 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:48.458469  9867 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:48.493129  9867 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:48.494072  9861 log.cc:927] Time spent T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:26:48.493976  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.495434  9867 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:48.467327  9801 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:48.495743  9752 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:48.497998  9787 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:48.441349  9800 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.502458  9810 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:48.505307  9849 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:48.472488  9555 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:48.473491  5374 heartbeater.cc:646] Failed to heartbeat to 127.3.89.254:36317 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.3.89.254:36317 timed out after 15.000s (SENT)
I20260709 12:26:48.473677  9768 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:48.476307  4751 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:48.487298  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.418087  9661 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:48.465075  9589 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:48.515448  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 119495us
W20260709 12:26:48.492405  9860 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 12:26:48.519681  4676 raft_consensus.cc:2393] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:48.519860  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:48.521368  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:48.521996  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 115743us
I20260709 12:26:48.522246  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.523121  4754 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:26:48.525014  4681 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:48.526548  4674 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:48.528997  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:48.529037  4674 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:48.529878  4681 raft_consensus.cc:2393] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 20: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:48.500679  9799 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.526058  9768 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:48.521307  9661 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:48.532944  4756 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:48.534168  9661 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:48.578190  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 129833us
I20260709 12:26:48.580745  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.581766  4754 raft_consensus.cc:2410] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:48.476864  9474 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:26:48.522738  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:48.582777  4569 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:48.584131  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 61460.3us
W20260709 12:26:48.579890  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49062us
I20260709 12:26:48.585366  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.588271  4756 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:48.593717  9864 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:26:48.594955  9589 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:48.597281  9751 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.600023  9751 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:48.603556  9768 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:48.608860  9768 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:48.609828  4674 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:48.604240  9674 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:48.620016  4748 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:48.620381  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.621486  4672 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 13. Ops: []
I20260709 12:26:48.622300  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:48.622238  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:48.611548  9799 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:48.630468  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 47768.1us
I20260709 12:26:48.635413  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:48.635728  9768 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:48.636444  4749 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:48.637779  4569 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:48.638520  9851 raft_consensus.cc:3055] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:48.639024  9851 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:48.640110  9799 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:48.640014  9851 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:48.641525  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:48.614964  9667 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:48.616070  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.603303  9733 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.641348  9851 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.618160  9874 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:48.619066  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:48.619074  4755 raft_consensus.cc:3055] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:48.623172  4752 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.612463  9555 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:48.623383  4748 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:48.625535  9797 raft_consensus.cc:3055] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:48.620280  9765 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:48.584374  9849 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.635514  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:48.646248  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:48.647802  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:48.617991  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.652406  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:48.662741  4753 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:48.657253  4571 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:48.643776  9768 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:48.656270  4755 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:48.663008  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:48.662565  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.651453  9751 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:48.637461  9842 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 12:26:48.630336  9781 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.663826  9667 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:48.665529  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:48.658913  9799 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:48.667790  4647 leader_election.cc:400] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16.
I20260709 12:26:48.668632  4571 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:48.671371  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.687463  4751 raft_consensus.cc:2410] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:48.684926  4569 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:48.654139  9603 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:26:48.664440  9768 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.672976  4755 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.705273  4755 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.663332  9767 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:48.665870  9674 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:48.658308  9797 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:48.712821  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 44267.4us
I20260709 12:26:48.669425  9555 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:48.713413  4569 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:48.724730  4569 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:26:48.713133  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 45428.3us
I20260709 12:26:48.671015  9264 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:48.722846  4752 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:48.725823  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:48.727803  4753 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 20.
I20260709 12:26:48.728682  4647 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:48.732074  4753 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:26:48.675031  9799 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.724452  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:48.738523  4647 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:48.712489  5312 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.193:45077 timed out after 1.677s (SENT) [suppressed 25 similar messages]
I20260709 12:26:48.747828  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:48.670233  9810 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:26:48.753207  4647 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:48.712103  9768 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:48.727494  9264 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.665851  9849 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:48.754745  4748 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:48.722384  9757 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:48.762362  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.733045  9874 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:48.749662  9799 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:48.673485  9751 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:48.733232  9793 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:48.844775  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 132311us
I20260709 12:26:48.725965  9877 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:48.752034  9667 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:48.807173  9264 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.779773  9752 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:48.838049  9757 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.754930  9787 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:48.752907  9767 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:48.848990  9555 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:48.855731  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 131078us
I20260709 12:26:48.857860  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:48.790004  9661 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:48.858590  4680 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:48.859817  4680 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:48.836838  9874 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.858315  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:48.848984  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 95860.9us
W20260709 12:26:48.862452  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 123537us
W20260709 12:26:48.890005  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.861377  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:48.861446  4750 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:48.924088  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:48.932566  4677 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:48.931814  4680 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:48.932704  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:48.848157  9810 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:48.942375  9879 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 12:26:48.944684  9880 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:26:48.856688  9667 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:48.856091  9757 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:48.908336  9765 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.948846  4677 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:48.949299  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:48.923511  9874 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:48.910280  9555 raft_consensus.cc:697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:48.841522  9739 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:48.955979  4680 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:48.958814  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:48.860328  9767 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.847838  9877 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.911584  9800 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.962232  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:48.887166  9661 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.974928  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:48.986302  4749 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:48.986361  4677 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:48.955339  9667 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:48.988232  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.990273  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.991070  4749 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:26:48.957706  9733 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:48.958658  9740 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:48.999428  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:48.975574  9767 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:49.003973  4753 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:49.013239  4680 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
W20260709 12:26:49.013902  4601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 818) took 1738 ms (client timeout 1677 ms). Trace:
I20260709 12:26:48.908481  9797 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.008953  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:48.992679  9752 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:49.015458  4601 rpcz_store.cc:269] 0709 12:26:47.263668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.265124 (+  1456us) service_pool.cc:224] Handling call
0709 12:26:49.002226 (+1737102us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b540004f454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:49.002352 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1215711616}
I20260709 12:26:48.958894  9765 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:48.985702  9877 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:48.997607  9781 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.029727  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:48.996166  9883 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:48.976223  9739 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.041478  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:49.057202  4749 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
W20260709 12:26:49.038368  9881 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 12:26:49.043339  9787 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:48.995867  9800 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.016420  9793 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 12:26:49.088290  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 79432.5us
I20260709 12:26:49.040796  9810 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:49.089871  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:49.094043  4647 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:49.066968  9867 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:49.095996  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:49.062284  9264 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.087754  9674 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.129922  4753 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:49.064213  9555 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:49.131090  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.131878  4755 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:26:49.134310  4755 pending_rounds.cc:85] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 7
I20260709 12:26:49.134668  4755 pending_rounds.cc:107] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:26:49.132043  4647 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:49.135021  4755 raft_consensus.cc:2889] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:49.140812  9890 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:26:49.147425  4497 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 165) took 15676 ms (client timeout 15000 ms). Trace:
I20260709 12:26:49.148303  4496 catalog_manager.cc:5697] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 5 to 8, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 8 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:49.097379  9751 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:49.151912  9878 log.cc:927] Time spent T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.331s	user 0.002s	sys 0.000s
I20260709 12:26:49.169956  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:26:49.158612  9873 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.457s	user 0.002s	sys 0.000s
I20260709 12:26:49.159102  9739 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:49.126408  9867 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.162977  9800 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:49.096681  9787 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:49.163898  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.165022  9264 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.165516  9752 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:49.065059  9883 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.166908  9849 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:49.169282  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.101934  9740 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:49.153499  4674 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:49.182560  4496 catalog_manager.cc:5697] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 10 to 13, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 13 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
W20260709 12:26:49.208648  9888 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:26:49.209388  4496 catalog_manager.cc:5697] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 8 to 10, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 10 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:49.211129  4496 catalog_manager.cc:5697] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:49.184341  4672 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:26:49.212589  4496 catalog_manager.cc:5697] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 11 to 12, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 12 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:49.215250  4496 catalog_manager.cc:5697] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 5 to 8, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 8 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:49.179203  9742 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.218057  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.185153  4753 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:26:49.219103  9883 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:49.179065  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:49.219681  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.220489  4672 raft_consensus.cc:2393] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:49.220640  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:49.187251  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:49.170107  9674 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.221935  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.222903  4672 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:49.223995  9674 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:49.172042  4497 rpcz_store.cc:269] 0709 12:26:33.471251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.471584 (+   333us) service_pool.cc:224] Handling call
0709 12:26:49.146877 (+15675293us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:42.501771 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:42.502447 (+   676us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:42.502497 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:26:42.502515 (+    18us) tablet.cc:662] Decoding operations
0709 12:26:42.503947 (+  1432us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:42.504046 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:42.504071 (+    25us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:26:42.505302 (+  1231us) tablet.cc:701] Row locks acquired
0709 12:26:42.505364 (+    62us) write_op.cc:260] PREPARE: finished
0709 12:26:42.505518 (+   154us) write_op.cc:270] Start()
0709 12:26:42.572066 (+ 66548us) write_op.cc:276] Timestamp: P: 1783600002505454 usec, L: 0
0709 12:26:42.575278 (+  3212us) op_driver.cc:348] REPLICATION: starting
0709 12:26:42.816433 (+241155us) log.cc:844] Serialized 6981 byte log entry
0709 12:26:43.342408 (+525975us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:43.462362 (+119954us) op_driver.cc:464] REPLICATION: finished
0709 12:26:43.463733 (+  1371us) write_op.cc:301] APPLY: starting
0709 12:26:43.463797 (+    64us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:43.774918 (+311121us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:43.775015 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:44.478188 (+703173us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:44.478584 (+   396us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 12:26:44.478645 (+    61us) write_op.cc:312] APPLY: finished
0709 12:26:44.479251 (+   606us) log.cc:844] Serialized 156 byte log entry
0709 12:26:44.739048 (+259797us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:44.739442 (+   394us) write_op.cc:454] Released schema lock
0709 12:26:44.739551 (+   109us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:44.740135 (+   584us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:44.740166 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13350,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":1135,"apply.run_cpu_time_us":49463,"apply.run_wall_time_us":1276329,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75085,"mutex_wait_us":16602,"num_ops":17,"prepare.queue_time_us":596190,"prepare.run_cpu_time_us":26677,"prepare.run_wall_time_us":576521,"raft.queue_time_us":26699,"raft.run_cpu_time_us":1302,"raft.run_wall_time_us":1322,"replication_time_us":886919,"spinlock_wait_cycles":4268928,"thread_start_us":802736,"threads_started":5,"wal-append.queue_time_us":517204,"wal-append.run_cpu_time_us":12307,"wal-append.run_wall_time_us":2551801}]]}
I20260709 12:26:49.224306  9874 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.224473  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.221320  4496 catalog_manager.cc:5697] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 6 to 11, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 11 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:49.199867  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.183020  9588 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:49.226814  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.232908  4674 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:49.221858  4681 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:49.233718  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.235783  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.237299  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.238375  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.239125  4679 raft_consensus.cc:2393] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 11: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:49.239171  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.239436  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2603) took 1503 ms. Trace:
I20260709 12:26:49.240026  4601 raft_consensus.cc:2368] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14.
I20260709 12:26:49.239969  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.240904  9576 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:49.241173  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:49.242368  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.242982  4599 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 7.
I20260709 12:26:49.243062  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.243697  9264 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.244210  9264 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.179425  9890 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.233775  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.247624  9890 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.247694  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.227403  9793 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:26:49.248519  4674 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 15: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:49.249285  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:49.249485  9752 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:49.250435  9752 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.251861  9867 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:49.228152  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:49.202910  9810 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.188992  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:49.229604  9664 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:49.231163  4672 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:26:49.231666  9851 consensus_meta.cc:287] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.575s	user 0.005s	sys 0.006s
I20260709 12:26:49.227869  9757 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.254247  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.254760  4679 raft_consensus.cc:2368] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:26:49.267614  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.268195  4599 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:49.268548  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.239809  4603 rpcz_store.cc:276] 0709 12:26:47.736511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.737969 (+  1458us) service_pool.cc:224] Handling call
0709 12:26:49.239285 (+1501316us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54002e59d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:49.239383 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":891435264}
I20260709 12:26:49.257018  4680 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:49.260460  4597 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 12:26:49.226754  9765 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:49.270277  9661 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:49.262051  4681 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:49.255878  9740 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 20 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:49.264727  9588 raft_consensus.cc:697] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:49.266211  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:49.270083  9768 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.261327  4672 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:49.266819  4496 catalog_manager.cc:5697] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 6 to 12, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 12 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:49.259574  4755 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:49.273473  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.282493  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:49.287398  4603 raft_consensus.cc:2368] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:26:49.304395  4680 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:26:49.307425  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.308198  4603 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:49.309461  4597 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:49.311970  9588 consensus_queue.cc:237] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.316416  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.288112  9805 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:49.271155  9877 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.320906  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.328027  9787 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.309279  9264 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.337548  4571 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:49.352350  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:49.364701  4603 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:49.373240  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:49.325711  4496 catalog_manager.cc:5697] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 7 to 12, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 12 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:49.335000  9776 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.373936  4601 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:49.366858  9787 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.336835  9742 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.368956  9877 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.351837  9805 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:49.369773  4602 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:49.379122  4496 catalog_manager.cc:5697] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 10 to 11, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 11 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:49.380501  9805 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.381690  9805 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:49.381870  4496 catalog_manager.cc:5697] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 9 to 12, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 12 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:49.372025  4596 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:49.372143  9874 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.316844  9767 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.375293  4681 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:49.379472  9899 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:49.335695  9799 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.386157  4571 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:49.384461  4496 catalog_manager.cc:5697] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 3 to 8, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 8 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:49.379227  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:49.396004  4680 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:49.397814  9736 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:49.397120  4496 catalog_manager.cc:5697] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 8 to 10, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 10 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:49.398212  9736 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.370726  9902 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.403370  9902 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.398583  4571 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:49.400687  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:49.387394  9742 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.398572  9736 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:49.402766  5348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 682) took 1408 ms. Trace:
I20260709 12:26:49.379099  9757 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.407217  5348 rpcz_store.cc:276] 0709 12:26:47.994244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.994408 (+   164us) service_pool.cc:224] Handling call
0709 12:26:48.117902 (+123494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.120864 (+  2962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.402479 (+1281615us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540084ce50. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:49.402664 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283160960}
I20260709 12:26:49.411759  4680 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:49.413115  4599 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:49.394534  9899 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.443392  4681 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:49.442322  4496 catalog_manager.cc:5697] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 7 to 9, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 9 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:49.337656  9740 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.412848  4571 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:49.448915  9757 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:49.449333  9757 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.449846  9757 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:49.413028  9874 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:49.454037  9874 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:49.455152  9799 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.395205  9877 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:49.455055  9874 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.378252  9776 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.406994  9765 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.457914  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:49.468454  4748 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:49.447597  4496 catalog_manager.cc:5697] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:49.476471  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:49.477587  4681 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 16: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:49.480279  4601 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
W20260709 12:26:49.484143  4600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2596) took 1810 ms (client timeout 1544 ms). Trace:
I20260709 12:26:49.485249  4598 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12. Ops: []
W20260709 12:26:49.484594  4600 rpcz_store.cc:269] 0709 12:26:47.673625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.673810 (+   185us) service_pool.cc:224] Handling call
0709 12:26:49.483947 (+1810137us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54002e4ad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:49.484075 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239156480}
I20260709 12:26:49.487552  4748 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:26:49.488551  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:49.489558  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:49.488831  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:49.440162  9876 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:26:49.445334  9899 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:49.503151  4496 catalog_manager.cc:5697] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 9 to 12, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 12 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:26:49.505648  4748 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:26:49.498288  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:49.510869  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:49.511798  4749 raft_consensus.cc:2393] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:49.518440  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:49.519335  4755 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:49.463035  9883 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:49.490753  9667 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.526290  9835 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:49.526881  9835 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:49.527431  9901 log.cc:927] Time spent T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:26:49.529373  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:49.531908  4748 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:49.532857  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:49.533655  4748 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:49.490360  9799 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:49.537778  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:49.542580  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:49.543687  9724 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:49.544248  9904 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:26:49.544524  9787 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:49.507346  9768 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.468250  9877 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.546445  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:49.502184  9735 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:49.547161  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:49.548336  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 103236us
I20260709 12:26:49.491139  9765 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.550850  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:49.553663  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:49.525184  9810 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:49.542307  9661 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:49.536023  9883 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.562822  4748 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
W20260709 12:26:49.565852  9896 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:26:49.569808  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:49.561388  9810 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:49.570464  4681 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:49.545948  9799 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.555583  9902 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:49.573139  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:49.575670  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2606) took 1064 ms. Trace:
I20260709 12:26:49.580847  4756 rpcz_store.cc:276] 0709 12:26:48.511104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.511281 (+   177us) service_pool.cc:224] Handling call
0709 12:26:49.575442 (+1064161us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b5400183154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:49.575587 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26492160}
I20260709 12:26:49.589722  9899 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.576968  4751 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:49.594303  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.597468  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.602424  9667 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.603259  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:49.603643  9767 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.604118  4750 raft_consensus.cc:2393] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 20: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:49.604545  9907 log.cc:927] Time spent T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:26:49.578562  9835 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:49.557137  9735 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.576318  9765 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:49.574607  9810 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.620242  9730 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:49.592628  4681 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:49.579937  9776 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.597723  9742 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:49.590436  9736 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.615912  4756 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:49.614310  4569 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:49.626513  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:49.621896  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:49.562633  9661 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.631426  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:49.632373  9742 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:49.633854  4756 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:49.634747  9890 raft_consensus.cc:3055] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:49.634871  9876 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:49.635422  9890 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:49.636534  9757 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.635701  4756 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:49.636535  9890 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.639871  9661 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:49.640069  9890 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:49.640944  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.641822  9883 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:49.642393  9883 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.642831  9883 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.644905  9883 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.647035  4645 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:49.648069  9736 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.632407  9664 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:49.651456  9899 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.651832  4755 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:49.653501  4569 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:49.654392  9264 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:49.655169  9264 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:49.656026  9264 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.656889  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3182) took 1295 ms. Trace:
I20260709 12:26:49.630169  9865 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:49.657269  4676 rpcz_store.cc:276] 0709 12:26:48.361013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.362535 (+  1522us) service_pool.cc:224] Handling call
0709 12:26:49.656683 (+1294148us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400086954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:49.656810 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346642816}
I20260709 12:26:49.657872  9865 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.658284  9865 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:49.659154  4602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2584) took 2375 ms (client timeout 2172 ms). Trace:
W20260709 12:26:49.659708  4602 rpcz_store.cc:269] 0709 12:26:47.283339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.283594 (+   255us) service_pool.cc:224] Handling call
0709 12:26:49.658951 (+2375357us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b5400353ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:49.659063 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4789583232}
I20260709 12:26:49.648161  9776 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:49.661468  9776 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.661906  9776 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:49.669552  9735 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:49.671957  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.673527  4754 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:49.674921  9757 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.681188  4569 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:49.682495  9868 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:49.683259  9868 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:49.687908  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.689208  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.690268  4603 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:49.698500  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.699721  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.700770  4676 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:49.701295  4602 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:49.701647  4673 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:49.703559  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.705744  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.706492  4680 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:26:49.689044  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:49.711619  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 89775.7us
I20260709 12:26:49.712212  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.711735  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.711897  4596 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:26:49.714401  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.715776  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.716657  4596 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 12: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:49.620276  9674 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.721710  4676 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:26:49.694245  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.722524  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.728454  4680 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:26:49.723753  4601 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:49.723532  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.724306  4603 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:49.726213  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.729082  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.730304  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.731527  4601 raft_consensus.cc:2410] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:49.732298  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.732926  4601 raft_consensus.cc:2393] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:49.732568  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:49.734362  4678 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:49.734251  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:49.735353  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.735586  4596 raft_consensus.cc:2393] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 12: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:49.735926  4605 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:49.730054  4604 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:49.738041  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.738744  4601 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:49.739766  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:49.740392  4601 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:26:49.743868  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.744613  4672 raft_consensus.cc:2393] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:49.748086  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.748908  4596 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:49.749953  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:49.684628  9868 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.701398  9776 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.730028  9850 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:49.712608  9661 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:49.754861  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.757983  4680 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:26:49.761786  4673 raft_consensus.cc:2368] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
W20260709 12:26:49.728370  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.763228  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:49.763283  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:49.764923  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.765694  4596 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:49.768326  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:49.769250  4605 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:49.769387  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:49.770097  4596 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:49.774507  9661 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.776984  9661 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:49.777908  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:49.778661  4605 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:49.778702  9776 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:49.779995  4605 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:26:49.782363  4605 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:49.783694  4605 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:49.760710  9835 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.787081  9835 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.787755  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 946) took 1155 ms. Trace:
I20260709 12:26:49.788126  4751 rpcz_store.cc:276] 0709 12:26:48.632763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.704816 (+ 72053us) service_pool.cc:224] Handling call
0709 12:26:49.576748 (+871932us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.593310 (+ 16562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.787550 (+194240us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b5400183150. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:49.787674 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1948833664}
I20260709 12:26:49.789168  9835 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:49.790705  9912 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
I20260709 12:26:49.794020  9800 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.794584  9800 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:49.794615  5344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53808 (request call id 618) took 1628 ms. Trace:
I20260709 12:26:49.796758  5344 rpcz_store.cc:276] 0709 12:26:48.166091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.166297 (+   206us) service_pool.cc:224] Handling call
0709 12:26:49.794409 (+1628112us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54005749d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:49.794535 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432072064}
I20260709 12:26:49.798576  9775 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:49.798295  9672 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:26:49.804280  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.836783  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 73652.7us
I20260709 12:26:49.838367  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:49.839999  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 82330.6us
W20260709 12:26:49.841001  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.735003  9674 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:49.851591  4599 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:49.682502  4569 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:49.843616  9835 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:49.867511  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 185070us
I20260709 12:26:49.870530  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:49.880046  4569 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:49.883275  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:49.884452  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:49.885068  4672 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:49.892060  4601 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:49.893180  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.893903  4677 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:49.894779  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:49.895448  4677 raft_consensus.cc:2368] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13.
I20260709 12:26:49.897836  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:49.898391  4676 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:49.917639  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:49.918788  4676 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:49.919325  4672 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:49.922751  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:49.923550  4678 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:49.929607  4569 leader_election.cc:400] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13.
I20260709 12:26:49.930140  4569 leader_election.cc:403] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:49.930819  9757 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:49.930967  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:49.931233  9757 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.931629  9757 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:49.931913  4569 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:49.933311  9853 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.177s
I20260709 12:26:49.880573  9871 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:49.879561  9661 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:49.917191  9765 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:49.915858  9867 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:49.938369  4596 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:49.916875  9877 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:49.939534  9653 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:49.942492  9899 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13.
I20260709 12:26:49.942340  9735 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:49.938352  9740 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:49.947729  9874 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:49.976073  4603 raft_consensus.cc:2410] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:26:49.980144  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:49.981230  4571 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:49.981055  9892 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:49.940753  9835 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:26:49.983088  9653 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:49.987931  9869 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.554s	user 0.004s	sys 0.004s
I20260709 12:26:49.951788  9765 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:49.949206  9871 raft_consensus.cc:697] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:49.995790  4571 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:49.980191  9797 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:50.017453  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:49.960947  9799 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:50.009037  9207 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:50.019315  4571 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:26:50.043716  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.000059  9867 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.020193  5340 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:50.156716  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 137508us
I20260709 12:26:50.002231  9664 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:50.158529  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:50.002707  9661 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:50.167022  5340 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:50.010327  9765 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:50.005148  9874 raft_consensus.cc:697] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:50.162712  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:50.010353  9735 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:50.146953  9899 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:50.146092  9776 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:50.147727  9877 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:50.003557  9865 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.198571  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:50.006089  9740 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.202826  4596 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:26:50.224844  5312 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.010s	sys 0.014s
I20260709 12:26:50.228837  4678 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:26:50.166139  9890 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:50.230441  4602 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:50.166036  9736 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:50.231602  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.166036  9588 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:50.236092  9431 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:50.246306  9588 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.246995  9431 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:50.236084  9603 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.247531  9431 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.247738  9603 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.248858  9588 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:50.251287  4600 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:50.176208  9810 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:50.255692  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3173) took 1932 ms. Trace:
I20260709 12:26:50.257825  4675 rpcz_store.cc:276] 0709 12:26:48.323501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.323683 (+   182us) service_pool.cc:224] Handling call
0709 12:26:48.324110 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.324551 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.324579 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.325048 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 12:26:48.422081 (+ 97033us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:48.422119 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.422180 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.987281 (+1565101us) spinlock_profiling.cc:243] Waited 7.63 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:50.252098 (+264817us) raft_consensus.cc:1685] finished
0709 12:26:50.252119 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.255612 (+  3493us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14301,"raft.queue_time_us":4942,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":9710,"spinlock_wait_cycles":17557376,"thread_start_us":86266,"threads_started":1,"wal-append.queue_time_us":102843}
I20260709 12:26:50.166563  9867 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.264560  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.264829  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:50.242313  9589 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:50.172379  9661 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.274035  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:50.274856  5344 raft_consensus.cc:2393] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 20: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:50.276224  9757 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.216737  9776 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.277909  9796 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.278617  9796 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.279356  9776 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:50.281473  9796 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:50.282171  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:50.282883  4751 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
W20260709 12:26:50.283143  9918 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:26:50.268747  9871 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.229168  9653 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.283732  9874 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.304404  9674 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:50.302647  9735 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.305471  9757 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:50.309183  9865 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:50.315196  9765 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.324715  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2601) took 1899 ms. Trace:
I20260709 12:26:50.350099  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:50.349231  4747 rpcz_store.cc:276] 0709 12:26:48.425057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.436072 (+ 11015us) service_pool.cc:224] Handling call
0709 12:26:48.436352 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.484768 (+ 48416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.484818 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.501869 (+ 17051us) log.cc:844] Serialized 26 byte log entry
0709 12:26:49.545193 (+1043324us) spinlock_profiling.cc:243] Waited 7.84 ms on lock 0x7b1800454d20. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:26:49.628519 (+ 83326us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:49.628548 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.628594 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.324372 (+695778us) raft_consensus.cc:1685] finished
0709 12:26:50.324397 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.324651 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40465,"raft.queue_time_us":40591,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":9801,"spinlock_wait_cycles":18028800,"thread_start_us":83139,"threads_started":1,"wal-append.queue_time_us":1820302}
I20260709 12:26:50.351009  4754 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:50.325879  9661 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:50.355996  4604 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:50.330298  4596 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:50.301358  9892 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:26:50.310882  9740 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:50.332284  9603 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:50.339469  9867 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:50.341562  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 436643us
I20260709 12:26:50.375186  4720 leader_election.cc:400] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:26:50.375499  4720 leader_election.cc:403] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:26:50.346800  5340 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:26:50.347069  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.347591  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:50.354269  4603 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 13. Ops: []
I20260709 12:26:50.377624  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:50.377843  4601 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:26:50.378198  4601 pending_rounds.cc:85] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 6
I20260709 12:26:50.378228  4603 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:50.378548  4601 pending_rounds.cc:107] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:26:50.354121  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:50.379081  4601 raft_consensus.cc:2889] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:50.379449  4605 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:50.364274  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:50.365557  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:50.380968  4596 raft_consensus.cc:2393] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 12: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:50.382011  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:50.383885  4596 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:26:50.371645  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.385489  4597 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:50.387089  4598 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:26:50.387626  5340 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53808 (request call id 616) took 2222 ms (client timeout 2505 ms). Trace:
I20260709 12:26:50.387790  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:50.372014  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.372874  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:50.390017  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:50.390486  5344 raft_consensus.cc:3055] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:50.392795  5345 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:50.395349  5343 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:50.390895  5344 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:50.398276  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:50.388007  5340 rpcz_store.cc:269] 0709 12:26:48.154740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.154940 (+   200us) service_pool.cc:224] Handling call
0709 12:26:49.847921 (+1692981us) spinlock_profiling.cc:243] Waited 4.66 ms on lock 0x7b4c000d36c0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effec782a20 00007effec793a29 00007effec759644 00007effec780f2d 00007effec780b93 00007effee4b118b 00007effee4b1cb5 00007effee4acf30 00007effee4ae7de 00007effee4ba27c 00007efff347bb17
0709 12:26:50.019723 (+171802us) spinlock_profiling.cc:243] Waited 86.3 ms on lock 0x7b54006002a0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee43406f 00007effee4acf60 00007effee4ae7de 00007effee4ba27c 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2
0709 12:26:50.376622 (+356899us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209301632}
I20260709 12:26:50.393306  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:50.396386  5344 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.403065  5344 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:26:50.402511  4645 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:50.407442  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.370036  9661 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.414769  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:50.428680  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:50.433161  5343 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.408825  9740 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:50.439810  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.444214  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:50.453555  4601 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:50.454488  5347 raft_consensus.cc:2368] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14.
I20260709 12:26:50.404315  9853 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:26:50.418453  9867 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:26:50.463622  9867 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:50.464795  9867 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:50.466486  9867 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:50.474304  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 71873.8us
I20260709 12:26:50.475000  9667 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:50.476161  9667 consensus_queue.cc:1059] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:50.477619  9776 raft_consensus.cc:3055] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:50.478098  9776 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:50.479135  9776 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.480448  9776 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.476184  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:50.483522  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:50.486296  9667 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:50.487049  9667 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:50.485463  4645 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:50.496647  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:50.498910  4645 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:50.543430  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 44732.8us
W20260709 12:26:50.518139  4719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.000s	sys 0.057s
W20260709 12:26:50.544241  4719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.000s	sys 0.057s
W20260709 12:26:50.435199  9796 raft_consensus.cc:670] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } [suppressed 5 similar messages]
I20260709 12:26:50.521001  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.487792  9667 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.515640  4605 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:26:50.561450  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.568403  5347 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:50.570080  5344 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:50.424801  9661 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:50.591647  9922 log.cc:927] Time spent T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:26:50.519966  9853 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.561592  9740 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:50.598287  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.602216  5343 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
W20260709 12:26:50.598145  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 409426us
I20260709 12:26:50.613009  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:50.616966  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.616061  9588 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:50.607823  9740 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:50.625862  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.626679  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.626729  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.627767  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.593406  9928 consensus_queue.cc:1059] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:26:50.628743  4644 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:50.630482  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.634613  5347 raft_consensus.cc:2368] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13.
I20260709 12:26:50.621366  9588 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.635455  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.608011  9853 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.645792  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.641135  5344 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:50.633750  9740 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:50.630060  9800 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:50.633442  9935 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:50.641105  9588 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.637950  9765 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:50.624030  9431 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:50.671352  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 270012us
I20260709 12:26:50.662791  9935 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:50.692345  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:50.695474  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 297641us
I20260709 12:26:50.697631  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:50.704118  9930 log.cc:927] Time spent T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:26:50.708217  9775 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:50.710666  9936 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:50.711287  9936 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.714730  4568 leader_election.cc:400] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14.
I20260709 12:26:50.714538  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:50.715047  4568 leader_election.cc:403] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:26:50.715256  5345 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:50.717846  9892 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:50.718454  9931 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:50.720801  9931 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:50.722405  9934 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:26:50.696583  9935 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:50.732394  4568 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:50.695995  9765 consensus_queue.cc:1059] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:26:50.704874  9850 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:50.740617  4568 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:50.735611  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:50.739904  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 194987us
I20260709 12:26:50.741472  4676 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.721415  9931 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.742815  4568 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:50.730361  9871 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14.
I20260709 12:26:50.746429  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:50.747961  4747 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.748721  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:50.749394  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:50.752162  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:50.752068  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 206354us
I20260709 12:26:50.752817  4748 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:50.754129  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:50.754706  4754 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:50.754814  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:50.729852  9794 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:50.752061  9937 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:50.760006  4676 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:50.744314  9735 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:26:50.767127  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 138488us
I20260709 12:26:50.746975  9931 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:50.749948  9653 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:50.747164  9757 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:50.753298  9765 raft_consensus.cc:3055] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:50.777117  4644 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:50.792760  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:50.775517  9735 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.803478  9735 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.770982  9661 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:50.807551  9940 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:50.800817  9794 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.812999  9794 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.740506  9942 consensus_queue.cc:1059] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:26:50.813896  9661 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:50.814436  9661 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.816648  9661 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:50.817144  9661 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:50.817456  9661 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:50.817999  9794 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:50.818454  9661 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.821892  9853 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:50.801018  9939 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:26:50.825299  9920 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.572s	user 0.001s	sys 0.000s
I20260709 12:26:50.827528  9892 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:50.828437  9892 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:50.828761  9892 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:50.832767  4568 leader_election.cc:400] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13.
I20260709 12:26:50.833695  4568 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:50.833989  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2612) took 1128 ms. Trace:
I20260709 12:26:50.834582  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:50.835235  4755 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:50.834437  4602 rpcz_store.cc:276] 0709 12:26:49.705966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.707317 (+  1351us) service_pool.cc:224] Handling call
0709 12:26:50.224014 (+516697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.237282 (+ 13268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.833743 (+596461us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b540032c3d0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:50.833892 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1187429632}
I20260709 12:26:50.837705  4756 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:50.774993  9937 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.838413  9892 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.839733  9892 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:50.840018  9937 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:50.764869  9850 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:50.841423  9850 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.841876  9931 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.802405  9589 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:50.843278  9589 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:50.843973  9931 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:50.798797  9936 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:50.782007  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:50.799721  9765 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:50.845160  4749 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 16: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:50.772812  9776 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:50.846887  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:50.850147  5314 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:50.868916  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:50.873754  4755 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.870085  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:50.871246  9431 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:50.818812  9794 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:50.872279  4753 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:50.873466  4749 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
W20260709 12:26:50.879269  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 45679us
W20260709 12:26:50.852234  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.849679  4756 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:50.881086  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:50.881472  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 128403us
I20260709 12:26:50.880940  4568 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:50.847611  4748 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:50.780968  4747 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:50.868444  9757 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:50.872259  9735 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.847316  9765 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.890589  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:50.889741  4645 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:50.888610  9765 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:50.898083  9937 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:50.898810  9937 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:50.900470  4749 raft_consensus.cc:3055] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:50.882328  9772 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:50.908072  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.935639  4749 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:50.907132  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:50.940441  4755 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:50.858434  9867 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:50.943395  4755 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:50.947818  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:50.890244  9776 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.948482  4747 raft_consensus.cc:2376] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15. Re-sending same reply.
I20260709 12:26:50.938661  4749 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.958455  9776 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.959139  4749 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:50.960860  9776 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:50.951114  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:50.962136  4756 raft_consensus.cc:2393] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:26:50.965706  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.968323  4749 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:26:50.968899  4749 pending_rounds.cc:85] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 6
I20260709 12:26:50.969118  4749 pending_rounds.cc:107] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:26:50.969336  4749 raft_consensus.cc:2889] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:50.888696  9805 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.949023  9867 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.954139  9807 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:50.970032  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 274233us
I20260709 12:26:50.907714  9767 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:50.935181  9653 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:50.956945  9588 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:50.937944  9772 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.906811  9936 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:26:50.961422  9776 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:50.983261  9772 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:50.983170  9757 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.983271  9807 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:50.980216  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 273723us
I20260709 12:26:50.977175  4720 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:51.002547  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.888957  9667 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.981513  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:51.046162  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 69077.7us
I20260709 12:26:51.047359  4753 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:51.047201  4720 leader_election.cc:400] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
I20260709 12:26:51.048287  4720 leader_election.cc:403] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:50.981178  9775 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:50.906275  9951 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 12:26:50.988471  9935 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:50.982053  9653 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.995316  9588 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:51.030014  9931 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.095510  4679 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:51.044023  9924 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.433s	user 0.000s	sys 0.005s
I20260709 12:26:51.057104  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:51.059089  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:50.979161  9805 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:51.052932  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:51.105481  4751 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:26:50.980456  9767 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:51.053972  9775 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.099148  4753 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:51.029631  9947 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:26:51.103271  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:51.107484  9955 raft_consensus.cc:3055] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:51.109978  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 961) took 1396 ms. Trace:
I20260709 12:26:51.110374  9835 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:51.111948  9589 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:51.113170  9796 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:51.130692  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:51.132411  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:51.098170  9653 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:51.029794  9776 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:51.150558  4753 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:51.125183  4752 rpcz_store.cc:276] 0709 12:26:49.713128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.713284 (+   156us) service_pool.cc:224] Handling call
0709 12:26:49.744266 (+ 30982us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.750965 (+  6699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:49.751017 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:49.756472 (+  5455us) log.cc:844] Serialized 26 byte log entry
0709 12:26:50.317914 (+561442us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:50.317954 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.318030 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.989048 (+671018us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:51.109661 (+120613us) raft_consensus.cc:1685] finished
0709 12:26:51.109684 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.109903 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9341,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":3029,"spinlock_wait_cycles":2526080,"thread_start_us":471812,"threads_started":1,"wal-append.queue_time_us":706511}
I20260709 12:26:51.098594  9931 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:51.142876  4599 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 14: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:51.152293  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:51.154980  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 377946us
I20260709 12:26:51.124682  9767 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:51.160130  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:51.128983  9775 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:51.136991  9923 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.006s	sys 0.040s
I20260709 12:26:51.127593  9955 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:51.192219  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:51.193070  4674 raft_consensus.cc:2393] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:51.129395  9835 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:51.194615  4672 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:26:51.146951  9928 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.010s	sys 0.032s
I20260709 12:26:51.195909  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:51.196581  4672 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:51.136561  9796 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:51.200697  4750 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:51.203547  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:51.207294  4672 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:26:51.181447  4751 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:26:51.212735  4672 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:26:51.237308  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 53114.9us
I20260709 12:26:51.240805  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:51.190955  9923 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.007s	sys 0.040s
I20260709 12:26:51.241155  4674 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:51.241636  5344 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:51.235931  9796 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:51.245936  5347 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:51.093173  9937 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:51.200914  9835 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:51.259192  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.261773  4681 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:51.262372  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:51.262810  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:51.264307  4753 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 7.
I20260709 12:26:51.199352  9752 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:51.237293  9952 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
I20260709 12:26:51.263212  4754 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:51.246907  9928 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:51.290153  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:51.298110  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 391109us
W20260709 12:26:51.270680  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:51.267148  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 210112us
I20260709 12:26:51.256283  9765 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:51.276182  4571 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:51.310916  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:51.319140  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:51.322744  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.330490  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:51.332023  4647 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:51.333187  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 57117.5us
I20260709 12:26:51.333942  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:51.334474  4754 raft_consensus.cc:2393] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:26:51.335671  4600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2610) took 1695 ms (client timeout 1545 ms). Trace:
I20260709 12:26:51.335994  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:51.336130  4600 rpcz_store.cc:269] 0709 12:26:49.640570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.640714 (+   144us) service_pool.cc:224] Handling call
0709 12:26:51.335482 (+1694768us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b540013e7d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:51.335610 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1401965952}
I20260709 12:26:51.336704  4753 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:26:51.318645  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:51.338148  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 390017us
I20260709 12:26:51.340929  4754 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:51.342664  4720 leader_election.cc:400] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14.
I20260709 12:26:51.347867  4720 leader_election.cc:403] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:51.349005  4720 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:51.331933  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.354043  4645 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:51.332774  9899 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.335515  9892 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:51.360159  4753 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:51.363169  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:51.338851  9664 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:51.338675  9947 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:51.339618  9961 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:51.277156  9752 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:51.389250  9953 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.389708  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:51.390377  9935 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:51.390475  9588 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:26:51.390805  9962 log.cc:927] Time spent T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:51.391847  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:51.392365  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:51.363981  9867 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:51.392869  9661 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:51.393610  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:51.431555  4753 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:51.393951  9794 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:51.334342  9928 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:51.357969  9959 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:51.276790  9955 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:51.404422  9942 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:51.380789  9969 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14.
I20260709 12:26:51.363121  9667 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:51.359055  9735 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:51.368005  9807 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:51.364440  9431 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:26:51.420979  5349 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:26:51.430243  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:51.416131  9740 consensus_meta.cc:287] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.756s	user 0.006s	sys 0.006s
I20260709 12:26:51.438199  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:51.457600  5345 raft_consensus.cc:2376] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18. Re-sending same reply.
I20260709 12:26:51.415133  9805 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:51.440809  9966 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:51.376163  9899 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:51.419034  9871 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:26:51.455928  5343 raft_consensus.cc:2393] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 20: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:51.428347  9953 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:51.492899  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.449432  9955 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:51.417847  9588 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:51.442250  9589 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:51.510813  9588 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:51.512459  9588 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:51.516373  9928 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:51.443157  9931 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:51.468938  4485 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e8a74137ba104fe0b465cab6399c4940 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:26:51.469967  9958 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:26:51.475096  9899 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:51.442919  9942 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:51.456305  4753 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:51.448628  9776 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:51.440910  9935 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:51.500765  4719 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:51.464887  9264 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:51.535629  4719 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:51.538839  4753 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:26:51.541019  4753 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:51.464255  9740 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:51.547117  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 219803us
W20260709 12:26:51.549176  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 161067us
I20260709 12:26:51.535058  9942 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:51.554250  4753 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:51.557361  4753 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:51.531363  9892 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:51.537829  9966 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:51.538091  9935 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:51.558467  4753 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:51.601306  4753 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:51.542963  4485 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 181547us
W20260709 12:26:51.597957  9958 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:51.541203  9264 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:51.603502  9935 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:51.639106  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 280211us
I20260709 12:26:51.643283  4753 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 12:26:51.643735  4753 pending_rounds.cc:85] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 6
I20260709 12:26:51.643973  4753 pending_rounds.cc:107] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 12:26:51.644213  4753 raft_consensus.cc:2889] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:51.664810  9752 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:51.669296  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:51.670758  9874 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:51.671320  9874 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:26:51.646384  9966 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.673552  9874 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:51.673749  9865 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:51.676227  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:51.676522  9966 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:51.678494  9983 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.680594  9983 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:51.681527  9966 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:51.677093  4673 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:51.682051  9951 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:51.682916  4680 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:51.683403  9951 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:51.684738  4681 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:26:51.686806  9865 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:51.687335  9983 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:51.687754  9865 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:51.689708  9805 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.689654  9892 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:51.690338  9805 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:51.692404  9805 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:51.685497  9951 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:51.693910  9740 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:51.652024  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:51.697495  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:51.701184  4680 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
W20260709 12:26:51.704228  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 52290us
I20260709 12:26:51.705766  4571 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:51.698596  4569 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:26:51.718539  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 375460us
W20260709 12:26:51.719098  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 377552us
I20260709 12:26:51.720939  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:51.721432  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:51.721874  4681 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:26:51.724694  4647 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:51.710855  9958 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:26:51.694869  9752 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:51.790606  9835 consensus_meta.cc:287] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.000s
W20260709 12:26:51.772848  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47871.6us
I20260709 12:26:51.765231  9667 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:51.870236  9752 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:51.870647  9899 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:26:51.805876  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 205847us
I20260709 12:26:51.699870  9874 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.170s
I20260709 12:26:51.869665  9835 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:51.874024  9874 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:26:51.736526  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.874956  4755 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:51.862514  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 499392us
I20260709 12:26:51.873971  9899 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:51.878914  9835 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:51.765007  9965 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:51.876449  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:51.772620  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 54228.6us
I20260709 12:26:51.879184  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:26:51.888481  9775 consensus_meta.cc:287] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.684s	user 0.010s	sys 0.001s
I20260709 12:26:51.728853  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:51.879707  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:51.880049  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.715333  9937 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:51.799099  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.724000  9951 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.152s
W20260709 12:26:51.895216  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.771330  9742 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:51.816511  9772 consensus_meta.cc:287] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.768s	user 0.015s	sys 0.000s
I20260709 12:26:51.715945  9733 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:51.887665  5343 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 12:26:51.918586  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.986646  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:51.987299  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.920830  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:51.991338  4747 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:51.992648  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:51.727826  9954 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.194s
I20260709 12:26:51.736716  9805 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:51.922027  4681 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:51.922863  4605 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:26:51.885763  9796 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.617s	user 0.008s	sys 0.003s
I20260709 12:26:51.724849  9740 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.233s
I20260709 12:26:51.930753  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:51.814661  9264 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:51.936211  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57193.6us
I20260709 12:26:52.002873  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:51.862347  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 140985us
I20260709 12:26:52.004638  4645 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:51.820786  9431 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.842669  9882 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:51.862308  9953 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:51.941223  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 212436us
I20260709 12:26:51.882444  9757 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:51.946056  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:51.927069  9653 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.770s	user 0.010s	sys 0.001s
I20260709 12:26:51.957858  4677 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:51.958820  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:51.728010  9946 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.212s
I20260709 12:26:51.999136  9835 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.046723  9835 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.048866  9835 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:52.049891  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.801559  9868 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.156s
I20260709 12:26:51.993458  4747 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:26:51.931421  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:52.062127  5349 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:51.943434  9867 raft_consensus.cc:2764] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:26:52.012578  4720 leader_election.cc:400] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:26:52.070695  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:52.070993  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 58589.5us
I20260709 12:26:52.072062  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:51.949559  9735 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:26:51.952584  9892 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:51.953927  9775 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.041378  9589 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:52.082944  4571 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:51.959589  9965 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:52.051205  4645 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:52.052330  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:52.051798  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:52.058120  4749 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:26:52.087414  5343 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:26:52.063473  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.018702  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:52.074752  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.078907  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:52.016453  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:52.085228  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 68186.9us
I20260709 12:26:51.969815  9937 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.036496  5346 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:26:52.089411  4719 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:52.095186  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 44051.5us
W20260709 12:26:52.094833  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 78503.7us
I20260709 12:26:52.091701  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:52.090566  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:52.096038  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:52.060672  9805 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:52.077589  9984 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:52.017131  9667 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
W20260709 12:26:52.083279  9765 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.748s	user 0.006s	sys 0.006s
I20260709 12:26:52.099355  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:52.015045  9796 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.081712  9733 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.085944  9892 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:52.105738  5341 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:52.063549  9653 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:51.998490  9772 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.020910  9776 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 12:26:52.018875  9431 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.110474  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:52.111045  4568 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:52.097481  9867 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:52.088526  9835 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:52.077747 10005 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:26:52.033834  9882 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.122615  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:52.051671  9757 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.125972  5342 raft_consensus.cc:2410] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:26:52.126708  9796 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.128898  9772 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.109702  9966 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:52.113588  9952 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:52.105124  9589 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:52.138533  9965 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.142985  9990 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:52.100826  9931 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:52.089135  9899 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:26:52.102464  9988 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:52.098040  9871 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:26:52.148013  9835 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:52.145527  9983 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:26:52.110586 10001 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:52.097803  9865 consensus_queue.cc:1059] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:26:52.121017  9892 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:52.147009  9992 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 12:26:52.146270  9867 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:52.149739  9664 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:26:52.180009  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 84091us
I20260709 12:26:52.237195  4647 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:52.153051  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:52.169817  9805 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.114092  9765 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:52.152729  9984 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:52.110399  9850 consensus_queue.cc:1059] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:26:52.187584  9653 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:52.148213  9954 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:26:52.188701  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:52.248332  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 95385.6us
I20260709 12:26:52.177675  9931 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:52.193046  9661 consensus_meta.cc:287] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.755s	user 0.009s	sys 0.000s
I20260709 12:26:52.180770  9966 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:52.273290  9946 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:52.274438  9966 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.275029  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:52.275539  9946 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:52.277091  9946 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.184070  9835 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:52.277704  9801 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.278221  9801 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.280256  9801 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:52.281821  4748 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:26:52.282184  4748 pending_rounds.cc:85] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 9
I20260709 12:26:52.282411  4748 pending_rounds.cc:107] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:26:52.282629  4748 raft_consensus.cc:2889] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:52.283931  4756 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
W20260709 12:26:52.198153  9800 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 12:26:52.199290  9767 consensus_meta.cc:287] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.998s	user 0.011s	sys 0.000s
I20260709 12:26:52.179092  9952 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:52.202615  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:52.121978  9733 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.222460  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.223253  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:52.225237  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.234452  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:52.234860  4596 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:52.238025  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 295264us
I20260709 12:26:52.243080  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.270344  9951 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.011s
W20260709 12:26:52.343863  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.242353  9984 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:52.268579  9850 raft_consensus.cc:3055] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:52.204221 10001 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.353793  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.354701  4596 raft_consensus.cc:2410] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:26:52.354579  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:52.250556  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:52.243896  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.355592  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:52.333698  4597 raft_consensus.cc:2410] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:26:52.330929  4753 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:52.178385  9988 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.239673  9867 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:52.347661  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.273202  9931 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:52.351866  4569 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:52.239547  9664 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.238495  9989 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:52.353448  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.360471  4596 raft_consensus.cc:2368] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15.
W20260709 12:26:52.363536  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 113101us
I20260709 12:26:52.392349  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:52.267736 10008 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:52.377170  4753 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:26:52.333649  9765 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:52.394743  5314 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:52.382143  9742 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:52.368252  9653 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
W20260709 12:26:52.362843  9588 consensus_meta.cc:287] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.850s	user 0.012s	sys 0.001s
I20260709 12:26:52.396494  4603 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:52.404399  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.405093  4677 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:26:52.407262  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.408078  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.412266  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.413069  4680 raft_consensus.cc:2393] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:52.413683  4679 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16. Ops: [14.8-15.9]
I20260709 12:26:52.413689  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.415227 10013 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.415817 10013 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:52.417537 10003 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
W20260709 12:26:52.417907 10002 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:26:52.418556  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:52.419891  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.286182  9994 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.141s
I20260709 12:26:52.347434  9850 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:52.415261  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:52.414539  4674 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:26:52.433346  4672 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:52.433876  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.395872  9796 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:52.396873  9876 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:52.360186  9667 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.428259  5343 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:52.443214  4645 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:52.444382  5312 leader_election.cc:400] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15.
I20260709 12:26:52.444778  5312 leader_election.cc:403] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:26:52.423859  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.281730  9946 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.398769  9765 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:52.382561 10012 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:52.361184 10001 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:52.364938  9988 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:52.426509  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.459342  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.459420  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:52.435063  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:52.461225  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:52.435846  9989 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:52.385072  9966 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:52.436053  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:52.468173  4675 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:26:52.441334  9876 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.345765  9767 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.397977  9664 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:52.457590  9667 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.427202  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:52.497767  4753 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:52.462133  9943 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:52.397732 10008 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.445899  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.347204  9984 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:52.425493  9661 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.448382  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.503572  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.504984  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.451215  5349 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:52.403343  9775 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.366634  9264 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:26:52.400688  9800 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:52.454272  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.330566  9952 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.455931  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:52.434487  9992 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:26:52.437208  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.530536  4681 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:26:52.470150  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.476006 10014 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.532877 10014 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.533408  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:52.396363  9742 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.473788 10021 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:52.395097 10009 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 12:26:52.476296  5343 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:52.546696  4753 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:52.547246  5343 raft_consensus.cc:606] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:52.547833  5343 raft_consensus.cc:614] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:52.477296  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.439026  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.494282  5347 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:52.496136  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.456792  9796 raft_consensus.cc:697] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:52.453322  9765 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:52.458997  9990 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:52.500133  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.397526  9757 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.455648 10012 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:52.507556  4672 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:52.514042  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:52.502130 10008 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:52.557056  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:52.481753  9966 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.516965  9853 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:52.467720  9801 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:26:52.467300  9954 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.522176  4678 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:52.523245  9667 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.478857  9876 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:52.524533  9935 consensus_meta.cc:287] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.873s	user 0.010s	sys 0.000s
I20260709 12:26:52.524906  9882 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:52.465879  9989 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:52.525676 10017 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:26:52.476438  9951 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:52.493852  9431 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.526032  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:52.526954  9946 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:52.527472  9955 consensus_meta.cc:287] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.001s	user 0.011s	sys 0.000s
I20260709 12:26:52.527892  9835 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.528255  9767 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:52.531888  9800 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:52.534379  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 78506.9us
I20260709 12:26:52.531544  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.618860  4681 raft_consensus.cc:2410] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:26:52.535487  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:52.619416  4603 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:52.610678  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:52.535913  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:52.538331  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.534709  9661 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:52.580950  9998 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.582s	user 0.000s	sys 0.004s
I20260709 12:26:52.551290  4680 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:52.628074  5341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 693) took 1470 ms. Trace:
I20260709 12:26:52.629031  4680 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:52.553179  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.545075  9867 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.579900  5343 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:52.558868  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:52.633128  4753 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 16: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:52.579602  5342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53808 (request call id 625) took 1266 ms. Trace:
I20260709 12:26:52.578186  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:52.499260  9850 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:52.559530  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 45622.7us
I20260709 12:26:52.573441  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:52.633905  5348 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:26:52.608826  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2641) took 1443 ms. Trace:
I20260709 12:26:52.634501  4752 raft_consensus.cc:2368] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15.
I20260709 12:26:52.617230  5349 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 12:26:52.576718 10004 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.533s	user 0.009s	sys 0.000s
I20260709 12:26:52.644286  9850 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:52.624372  4677 raft_consensus.cc:2393] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 17: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:52.633596  5342 rpcz_store.cc:276] 0709 12:26:51.313353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.372995 (+ 59642us) service_pool.cc:224] Handling call
0709 12:26:52.579418 (+1206423us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b54004c5ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:52.579546 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1621219200}
I20260709 12:26:52.540413 10013 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.588943  9899 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.631599  4568 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:52.635344  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.628420  5341 rpcz_store.cc:276] 0709 12:26:51.157620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.157786 (+   166us) service_pool.cc:224] Handling call
0709 12:26:51.201852 (+ 44066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.113475 (+911623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.627865 (+514390us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b5400712354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:52.628022 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2078213888}
I20260709 12:26:52.656677  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.634696  4751 rpcz_store.cc:276] 0709 12:26:51.165417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.213136 (+ 47719us) service_pool.cc:224] Handling call
0709 12:26:51.216750 (+  3614us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:51.217258 (+   508us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:51.532302 (+315044us) log.cc:844] Serialized 14 byte log entry
0709 12:26:51.717412 (+185110us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:51.893112 (+175700us) log.cc:844] Serialized 556 byte log entry
0709 12:26:51.916504 (+ 23392us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:51.916551 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.916635 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:52.141045 (+224410us) spinlock_profiling.cc:243] Waited 933 us on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:52.589439 (+448394us) raft_consensus.cc:1685] finished
0709 12:26:52.608616 (+ 19177us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:52.608765 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41306,"prepare.queue_time_us":171381,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":55,"raft.queue_time_us":84,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":24801,"spinlock_wait_cycles":2145536,"thread_start_us":443093,"threads_started":2,"wal-append.queue_time_us":264135}
W20260709 12:26:52.594537  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.555805 10016 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:52.642436 10006 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:52.642815 10025 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.647876  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:52.648494  4569 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:52.559125 10012 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:52.626065  4645 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:52.656520  4750 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:52.609436  9935 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.602665  9989 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:52.655701  9664 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.610083  9954 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.671416  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.695717  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.696071  4752 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:26:52.665876  9955 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:26:52.672199  9835 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:52.676374  9752 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.712149  9882 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.712787  9882 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.572885  9796 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.715138  9882 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:52.696033 10021 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:52.682964  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 51472.2us
I20260709 12:26:52.717669  4568 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:52.718219  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.718813  4753 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:52.719717  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.720474  4753 raft_consensus.cc:2410] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 15 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:26:52.721123  9978 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:52.685117  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.722697  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.723537  4601 raft_consensus.cc:2393] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 15: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:52.724306  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.631770  9767 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:52.725081  9767 raft_consensus.cc:697] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:52.695838  9942 consensus_meta.cc:287] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.092s	user 0.006s	sys 0.007s
I20260709 12:26:52.726163  9942 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:52.691833  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 65850.5us
I20260709 12:26:52.561456  9990 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.727486  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:52.728223  4754 raft_consensus.cc:2368] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15.
I20260709 12:26:52.681697 10022 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:52.728962  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.729095  9990 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.556389  9984 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.729413  9942 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:52.697597  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.730798  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.731189  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.700987 10014 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:52.731853  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.732064  4644 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:52.702121  9954 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.732923  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.703336  9867 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:52.733495  9983 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:52.703480  4674 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:52.733932  9983 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.734323  9983 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:52.735167  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:52.701406 10016 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:52.736531  4672 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:26:52.736342 10016 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.694314 10006 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.738302  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:52.738245  9752 consensus_queue.cc:1059] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:52.739190 10016 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.739482 10021 raft_consensus.cc:3055] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:52.739864 10021 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:52.740064 10006 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:52.709975  4605 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:52.710070  9835 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:52.740703 10021 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.710716  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.742013  4752 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:26:52.742266  9954 consensus_queue.cc:1059] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:52.742929  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.743471  9988 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.743848  9776 raft_consensus.cc:3055] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:52.744294  9776 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:52.746515  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3197) took 1848 ms. Trace:
I20260709 12:26:52.745256  9776 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.746898  4748 rpcz_store.cc:276] 0709 12:26:50.902287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.902514 (+   227us) service_pool.cc:224] Handling call
0709 12:26:50.902815 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.283017 (+1380202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.746312 (+463295us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54001b0b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:52.746461 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3170564480}
I20260709 12:26:52.747843  9776 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:52.709476  9937 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.725997  9767 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.727921  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:52.732260  4747 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 12: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:52.672478  9977 raft_consensus.cc:3055] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:52.720522  9874 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:26:52.735966  4674 raft_consensus.cc:2410] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:26:52.702039  9664 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:52.741465  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 977) took 1190 ms. Trace:
I20260709 12:26:52.748756  4754 raft_consensus.cc:2410] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:52.666368  9899 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.704929  9800 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:52.752213  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:52.752794  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:52.754453  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:52.755051  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:52.744045  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.721280  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.671478  9966 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:52.749817  4602 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:52.749531  9961 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:52.808348  9972 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:26:52.760582  9954 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:52.809341  9954 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.811933  9954 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.819029  9765 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.820238  9664 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:52.803483  9740 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:52.820012  9954 raft_consensus.cc:2955] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index 3 to 10, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:52.821238  9740 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:52.821658  9740 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:52.822407  9986 log.cc:927] Time spent T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.899s	user 0.003s	sys 0.000s
I20260709 12:26:52.824944  9765 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:52.826187  9961 raft_consensus.cc:697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:52.827234  9961 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.827370  9931 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.831507  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.833477  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.837814  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:52.773676  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 125240us
I20260709 12:26:52.690220  9946 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.731681  9984 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:52.789932  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:52.790046  4672 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:26:52.846763  4672 pending_rounds.cc:85] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 9
I20260709 12:26:52.847095  4672 pending_rounds.cc:107] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 13.10
W20260709 12:26:52.847259  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.847393  4672 raft_consensus.cc:2889] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:52.741271  9835 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.712044  9952 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:52.797308  4752 raft_consensus.cc:2410] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 12:26:52.797444  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 44740us
I20260709 12:26:52.694772 10025 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.918537  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:52.920037  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:52.798211  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:52.798369  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.922274  4753 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:52.773167  4676 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:52.801039  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 48953.3us
I20260709 12:26:52.923561  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:52.764127  9775 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.106s
W20260709 12:26:52.802587 10028 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:26:52.804288  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:52.804730  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.763475  9977 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:52.737023  9892 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:52.789331 10006 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:52.776609  9937 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:52.846648  4569 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:52.914865  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:52.790565  4755 rpcz_store.cc:276] 0709 12:26:51.550957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.616564 (+ 65607us) service_pool.cc:224] Handling call
0709 12:26:51.642081 (+ 25517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:51.878524 (+236443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.741260 (+862736us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400072bd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:52.741408 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535117568}
I20260709 12:26:52.924515  4571 leader_election.cc:400] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15.
I20260709 12:26:52.801072  5346 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:26:52.869068  9990 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.942382  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.938484  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:52.765899 10001 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:52.945945  4571 leader_election.cc:403] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:26:52.947458  4647 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:52.937341  5343 raft_consensus.cc:606] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
W20260709 12:26:52.963518  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 116940us
I20260709 12:26:52.924942  9965 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:52.968194  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:52.973428  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:52.974190  4752 raft_consensus.cc:2393] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 11: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:52.974699  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:52.925879  9988 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:52.928594  9664 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:26:53.045723  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 98350.4us
I20260709 12:26:53.054221  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:53.047923  4755 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:53.056383  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3215) took 1048 ms. Trace:
I20260709 12:26:53.056703  4755 rpcz_store.cc:276] 0709 12:26:52.007714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.976008 (+968294us) service_pool.cc:224] Handling call
0709 12:26:53.047475 (+ 71467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.056150 (+  8675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.056331 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:52.915853  9946 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.050928  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:53.053090  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 128700us
I20260709 12:26:52.947345  9951 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:53.068192  4604 raft_consensus.cc:2368] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17.
I20260709 12:26:53.070039  4603 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:26:53.071733  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:53.075814  4645 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:53.057804  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 278577us
I20260709 12:26:53.076216  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:52.973198  9936 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:53.079850  9937 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:53.081951  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:53.046276  9978 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:53.081305  4645 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:53.059073  9664 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.051949  9966 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:53.055380  9955 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:52.934401  9977 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.089954  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:53.069483  9951 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:53.073429  9431 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15.
I20260709 12:26:53.077066  9965 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:53.088065 10016 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:52.976962  9990 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.086826  9978 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.105099  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:53.122457  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:53.089395  9937 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:53.139912  4603 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:53.089150  9664 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:53.082647  9264 consensus_queue.cc:1059] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:26:53.127240  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:53.113150  9951 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:53.137090  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.193084  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 120172us
I20260709 12:26:53.089133  9874 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:26:53.198642  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3217) took 1118 ms. Trace:
W20260709 12:26:53.204432  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.204363  4752 rpcz_store.cc:276] 0709 12:26:52.080013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.127596 (+1047583us) service_pool.cc:224] Handling call
0709 12:26:53.197960 (+ 70364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.198203 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.198240 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:53.198297 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:53.198343 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.198422 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.198564 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:53.113662  9765 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.084564 10022 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:26:53.115692  9965 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:53.127775  9935 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:53.128858  9978 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:53.096300  9966 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.140475  9955 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:53.215884 10012 consensus_meta.cc:287] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
I20260709 12:26:53.201246  9588 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:53.281031  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 984) took 1193 ms. Trace:
I20260709 12:26:53.281534  4751 rpcz_store.cc:276] 0709 12:26:52.087887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.239417 (+1151530us) service_pool.cc:224] Handling call
0709 12:26:53.280395 (+ 40978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.280616 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.280652 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:53.280708 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:53.280753 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.280821 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.280963 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:53.272359  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:53.293977  9947 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.294811  9947 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:53.277251  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 194943us
I20260709 12:26:53.305338  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:53.306583  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.307304 10021 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:53.310660  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2658) took 1074 ms. Trace:
I20260709 12:26:53.126235  9742 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.120s
I20260709 12:26:53.238641  9965 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:53.237892 10022 consensus_queue.cc:1243] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:53.312922  4677 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:53.315171  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3218) took 1151 ms. Trace:
I20260709 12:26:53.311091  4605 rpcz_store.cc:276] 0709 12:26:52.235877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.391189 (+155312us) service_pool.cc:224] Handling call
0709 12:26:52.391522 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.740803 (+349281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.310393 (+569590us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b5400218c54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:53.310590 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731972352}
I20260709 12:26:53.317821  4752 rpcz_store.cc:276] 0709 12:26:52.163914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.245928 (+1082014us) service_pool.cc:224] Handling call
0709 12:26:53.314485 (+ 68557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.314689 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.314729 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:53.314779 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:53.314843 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.314930 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.315090 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:53.334713  4677 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:53.276201  9899 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:53.280289  9954 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.238511  9765 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.364329  4751 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:53.368134  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2658) took 1087 ms. Trace:
I20260709 12:26:53.368563  4751 rpcz_store.cc:276] 0709 12:26:52.280260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.330149 (+1049889us) service_pool.cc:224] Handling call
0709 12:26:53.364059 (+ 33910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.367792 (+  3733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.368065 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:53.370102  4752 raft_consensus.cc:1240] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:53.370944  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 987) took 1085 ms. Trace:
I20260709 12:26:53.371351  4752 rpcz_store.cc:276] 0709 12:26:52.285406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.365319 (+1079913us) service_pool.cc:224] Handling call
0709 12:26:53.365675 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.370696 (+  5021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.370886 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:53.366911  5344 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.007s	sys 0.015s
W20260709 12:26:53.374352 10038 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:26:53.375864 10032 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
W20260709 12:26:53.376734 10041 log.cc:927] Time spent T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 12:26:53.377367 10034 log.cc:927] Time spent T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
W20260709 12:26:53.386796 10029 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 12:26:53.387429 10001 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:53.387899  9947 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:53.388433  9892 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:53.389833 10031 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 12:26:53.453421  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3250) took 1023 ms. Trace:
I20260709 12:26:53.453869  4680 rpcz_store.cc:276] 0709 12:26:52.429658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.630428 (+200770us) service_pool.cc:224] Handling call
0709 12:26:52.630746 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.646812 (+ 16066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.646842 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.664946 (+ 18104us) log.cc:844] Serialized 26 byte log entry
0709 12:26:52.938433 (+273487us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:52.938488 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.938581 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.453155 (+514574us) raft_consensus.cc:1685] finished
0709 12:26:53.453179 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.453329 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20285,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":10036,"thread_start_us":268087,"threads_started":1,"wal-append.queue_time_us":403846}
I20260709 12:26:53.456264  4680 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:53.390596 10037 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 12:26:53.399053 10008 consensus_meta.cc:287] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.776s	user 0.009s	sys 0.000s
I20260709 12:26:53.337126  9835 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:53.363842 10048 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.399781 10025 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:53.366622 10013 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
W20260709 12:26:53.420436  5344 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.007s	sys 0.015s
W20260709 12:26:53.422152  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 107694us
I20260709 12:26:53.332671  9603 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:53.370786  9954 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.311785 10012 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:53.480866  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 358486us
W20260709 12:26:53.482389  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.365797  9899 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.501286  9899 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:53.502041 10033 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.412s	user 0.004s	sys 0.000s
I20260709 12:26:53.504194  9899 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:53.500555  9947 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.491716  9946 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:53.512076  9776 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.764s	user 0.016s	sys 0.000s
I20260709 12:26:53.515556  9951 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.531621  9947 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.532473  4677 raft_consensus.cc:1240] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:53.493583  9966 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:53.498417  9775 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:26:53.534729  9775 consensus_queue.cc:1243] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:26:53.518980 10012 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:53.520385  9876 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.896s	user 0.012s	sys 0.008s
W20260709 12:26:53.522971  9983 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.788s	user 0.015s	sys 0.003s
I20260709 12:26:53.536315  9951 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:53.536686  9983 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.525818  9765 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:53.526085  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3214) took 1518 ms. Trace:
W20260709 12:26:53.527516  9740 consensus_meta.cc:287] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.705s	user 0.013s	sys 0.006s
I20260709 12:26:53.528614  5349 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:53.529634  9954 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:53.490216 10048 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.486785 10008 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.555759  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:53.579120  5349 raft_consensus.cc:606] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:53.579702  5349 raft_consensus.cc:614] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:53.542564  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3248) took 1114 ms. Trace:
I20260709 12:26:53.581676  4681 rpcz_store.cc:276] 0709 12:26:52.427892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.630493 (+202601us) service_pool.cc:224] Handling call
0709 12:26:53.541633 (+911140us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b58000d4798. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff348ffc0 00007efff347a929 00007efff347a3d2 00007effec9f1bd4
0709 12:26:53.541890 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.542130 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.542170 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:53.542226 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:53.542278 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.542346 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.542492 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4348672}
I20260709 12:26:53.566598  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:53.557991  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:53.585127  4597 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:53.585201  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:53.577242  4748 rpcz_store.cc:276] 0709 12:26:52.007434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.747826 (+740392us) service_pool.cc:224] Handling call
0709 12:26:52.748131 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.754051 (+  5920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.754102 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.754771 (+   669us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.051667 (+296896us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:53.051709 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.051775 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.525847 (+474072us) raft_consensus.cc:1685] finished
0709 12:26:53.525872 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.526026 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14689,"raft.queue_time_us":8081,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":14541,"thread_start_us":296539,"threads_started":1,"wal-append.queue_time_us":360400}
I20260709 12:26:53.593194  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:53.594099  4680 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:26:53.556125  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:53.595217  4600 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:53.596469  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:53.598137  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:53.599591  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:53.600589  4681 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:53.601401  4748 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:53.602402  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2661) took 1141 ms. Trace:
I20260709 12:26:53.602859  4748 rpcz_store.cc:276] 0709 12:26:52.460749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.592508 (+1131759us) service_pool.cc:224] Handling call
0709 12:26:53.601171 (+  8663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.602106 (+   935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.602337 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:53.603931  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:53.587356  4597 raft_consensus.cc:2955] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index 3 to 10, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:53.604871  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:53.614493  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:53.615831  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 67860.9us
I20260709 12:26:53.616408  4603 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:26:53.493059  9733 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:53.552774  9801 raft_consensus.cc:3055] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:53.623450  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:53.551605  9776 consensus_queue.cc:1059] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 12:26:53.652982  4680 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:26:53.626256  4748 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:53.540239  9661 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:53.645948  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:26:53.648623  4599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2646) took 1425 ms (client timeout 1851 ms). Trace:
I20260709 12:26:53.649921  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:53.650288  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:53.591843  9951 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.597550 10022 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:53.591504 10012 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:53.605340  9899 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:53.659957  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2662) took 1199 ms. Trace:
I20260709 12:26:53.671247  4748 rpcz_store.cc:276] 0709 12:26:52.461011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.604675 (+1143664us) service_pool.cc:224] Handling call
0709 12:26:53.626006 (+ 21331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.658679 (+ 32673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.659878 (+  1199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:53.661755  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3247) took 1234 ms. Trace:
I20260709 12:26:53.651823  9801 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:53.663262  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:53.658118  4599 rpcz_store.cc:269] 0709 12:26:52.222874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.223025 (+   151us) service_pool.cc:224] Handling call
0709 12:26:53.648343 (+1425318us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b540013e2d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:53.648548 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1437795200}
I20260709 12:26:53.655663  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:53.544771 10054 consensus_queue.cc:1059] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
I20260709 12:26:53.676595  9899 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.667536  9951 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.684149  9951 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:53.684785 10048 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:53.687371  9899 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:53.695405 10008 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:53.697685  9989 consensus_meta.cc:287] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.940s	user 0.009s	sys 0.008s
I20260709 12:26:53.698163  9989 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.700430  9989 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:53.701076  4756 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:53.702136  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 3222) took 1116 ms. Trace:
I20260709 12:26:53.702447  4756 rpcz_store.cc:276] 0709 12:26:52.585426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.667014 (+ 81588us) service_pool.cc:224] Handling call
0709 12:26:53.701970 (+1034956us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54000bf9d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:53.702072 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2310460288}
I20260709 12:26:53.708190  5342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:53.708858  5342 raft_consensus.cc:606] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:53.591579  9876 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.711359  9876 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:53.662595  9661 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:53.714151  9661 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.714366  5340 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.716248  9661 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:53.718458  9801 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.719600  9801 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:53.720757  9990 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:53.721370  9952 consensus_queue.cc:1059] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:53.722187  9765 raft_consensus.cc:3055] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:53.660445  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:53.662573  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 359468us
I20260709 12:26:53.671315  9835 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:53.722988  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 62633.5us
I20260709 12:26:53.723270  9835 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.724182  4645 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:53.660228  9946 raft_consensus.cc:3055] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:53.725800  9946 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:53.672418  4673 rpcz_store.cc:276] 0709 12:26:52.427644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.562498 (+134854us) service_pool.cc:224] Handling call
0709 12:26:52.562768 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.728907 (+166139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.728936 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:52.728978 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:52.729026 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.661512 (+932486us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54001bce54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:53.661558 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.661703 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381630720}
I20260709 12:26:53.729866  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:53.597615  9740 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.673266 10012 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.722538  9765 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:53.668089 10025 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:53.671030  9984 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.668305  9966 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:53.666546  9776 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:53.733598  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2657) took 1453 ms. Trace:
I20260709 12:26:53.726706  9946 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.764886  4749 rpcz_store.cc:276] 0709 12:26:52.280027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.306432 (+1026405us) service_pool.cc:224] Handling call
0709 12:26:53.306827 (+   395us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:53.314443 (+  7616us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.314474 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:53.315151 (+   677us) log.cc:844] Serialized 52 byte log entry
0709 12:26:53.661321 (+346170us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:53.661361 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.661438 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.733336 (+ 71898us) raft_consensus.cc:1685] finished
0709 12:26:53.733355 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.733522 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7138,"raft.run_cpu_time_us":2208,"raft.run_wall_time_us":27985,"thread_start_us":328382,"threads_started":1,"wal-append.queue_time_us":379743}
W20260709 12:26:53.743315  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.743083  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:53.744652  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:53.668783  9742 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.664384  9983 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:53.677917  9947 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:53.716995  9661 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:53.716995  9735 raft_consensus.cc:3055] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:53.735934  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:53.711359 10042 log.cc:927] Time spent T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.463s	user 0.002s	sys 0.000s
I20260709 12:26:53.746383 10008 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:53.789881 10008 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.790474 10008 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:53.790901  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3254) took 1360 ms. Trace:
I20260709 12:26:53.791309  4674 rpcz_store.cc:276] 0709 12:26:52.430763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.788886 (+358123us) service_pool.cc:224] Handling call
0709 12:26:52.789173 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.789497 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.789526 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.790018 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.085456 (+295438us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:53.085510 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.085610 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.710299 (+624689us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:53.790578 (+ 80279us) raft_consensus.cc:1685] finished
0709 12:26:53.790598 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.790831 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5981,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":3845,"spinlock_wait_cycles":65300864,"thread_start_us":289557,"threads_started":1,"wal-append.queue_time_us":318374}
I20260709 12:26:53.781301  9661 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:53.794147  9661 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:53.708214  9978 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.781702  9735 raft_consensus.cc:740] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:53.798837  9735 consensus_queue.cc:260] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.799973  9978 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:53.800544  9735 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:53.802297  9740 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:53.804250  9765 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.805429  9765 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:53.806456 10035 log.cc:927] Time spent T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.723s	user 0.000s	sys 0.001s
I20260709 12:26:53.810169  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2660) took 1365 ms. Trace:
W20260709 12:26:53.811021 10021 consensus_meta.cc:287] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.503s	user 0.006s	sys 0.014s
I20260709 12:26:53.811918  4681 raft_consensus.cc:2410] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:26:53.813586 10012 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:53.781702  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2660) took 1321 ms. Trace:
I20260709 12:26:53.814620  4752 rpcz_store.cc:276] 0709 12:26:52.460534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.372650 (+912116us) service_pool.cc:224] Handling call
0709 12:26:53.373008 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.447254 (+ 74246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.447300 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:53.484154 (+ 36854us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.760300 (+276146us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:53.760357 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.760455 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.781423 (+ 20968us) raft_consensus.cc:1685] finished
0709 12:26:53.781450 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.781641 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20282,"raft.queue_time_us":73758,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":104,"thread_start_us":235261,"threads_started":1,"wal-append.queue_time_us":265566}
I20260709 12:26:53.816052  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2659) took 1356 ms. Trace:
I20260709 12:26:53.816406  4751 rpcz_store.cc:276] 0709 12:26:52.459911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.371877 (+911966us) service_pool.cc:224] Handling call
0709 12:26:53.372277 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.372711 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.372751 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:53.457937 (+ 85186us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.747153 (+289216us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:53.747195 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.747267 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.815845 (+ 68578us) raft_consensus.cc:1685] finished
0709 12:26:53.815870 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.815998 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10328,"raft.queue_time_us":96,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":50863,"thread_start_us":247787,"threads_started":1,"wal-append.queue_time_us":325382}
I20260709 12:26:53.756633 10025 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.819396  9588 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:26:53.819739  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.820297  9588 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:53.821036 10025 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:53.821349  9588 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:53.822192  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:53.822749  9951 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:53.822985  5344 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 15: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:53.823165  9951 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:53.824153  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:53.824100  9951 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.750324  9990 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:53.672174 10022 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.825522  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:53.825943  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:53.826464  9951 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:53.757782  9984 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.822315  9588 raft_consensus.cc:3055] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:53.829487  9588 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:53.830549  9984 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:53.830338  9588 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.764215  9776 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.749497  9835 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:53.831421  9776 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:53.831656  9943 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 1.303s	user 0.015s	sys 0.002s
I20260709 12:26:53.832059  9943 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15.
I20260709 12:26:53.821557  9955 raft_consensus.cc:3055] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:53.832777  9955 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:53.833670  9955 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:53.834940  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 128633us
I20260709 12:26:53.821244  5314 leader_election.cc:400] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15.
I20260709 12:26:53.836249  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:53.828007  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2665) took 1056 ms. Trace:
I20260709 12:26:53.783486  9742 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.837236  9757 raft_consensus.cc:3055] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:53.837647  9757 raft_consensus.cc:740] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:53.838356  9946 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:53.840544  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3226) took 1152 ms. Trace:
I20260709 12:26:53.840657  9931 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:53.842147  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 2052) took 1434 ms. Trace:
I20260709 12:26:53.842747  4675 rpcz_store.cc:276] 0709 12:26:52.407245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.407852 (+   607us) service_pool.cc:224] Handling call
0709 12:26:53.841962 (+1434110us) spinlock_profiling.cc:243] Waited 59.5 ms on lock 0x7b540017eb54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:53.842078 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136763136}
I20260709 12:26:53.844260  4679 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:53.844895  5343 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:26:53.836894  4603 rpcz_store.cc:276] 0709 12:26:52.771269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.140960 (+369691us) service_pool.cc:224] Handling call
0709 12:26:53.141355 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.628347 (+486992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.827698 (+199351us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b5400140854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:53.827930 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091926784}
I20260709 12:26:53.845690  5314 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:26:53.787717 10016 consensus_meta.cc:287] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.659s	user 0.014s	sys 0.003s
I20260709 12:26:53.840993  4755 rpcz_store.cc:276] 0709 12:26:52.687880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.815295 (+1127415us) service_pool.cc:224] Handling call
0709 12:26:53.840047 (+ 24752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.840227 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.840253 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:53.840296 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:53.840335 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.840386 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.840486 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:53.810616  4596 rpcz_store.cc:276] 0709 12:26:52.444371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.758331 (+313960us) service_pool.cc:224] Handling call
0709 12:26:52.802357 (+ 44026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.804436 (+  2079us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.804495 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.814893 (+ 10398us) log.cc:844] Serialized 26 byte log entry
0709 12:26:52.929024 (+114131us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:52.929062 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.929135 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.457966 (+528831us) raft_consensus.cc:1685] finished
0709 12:26:53.457985 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.809867 (+351882us) spinlock_profiling.cc:243] Waited 43.7 ms on lock 0x7b54003552d0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:53.810101 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"spinlock_wait_cycles":100510976,"thread_start_us":113768,"threads_started":1,"wal-append.queue_time_us":274312}
I20260709 12:26:53.853199  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:53.853111  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:53.860702  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:53.834818  9955 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:53.864410  9835 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:53.865016  9835 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.865427  9776 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.867648  9776 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:53.868875  4678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3261) took 1207 ms. Trace:
I20260709 12:26:53.869375  4680 raft_consensus.cc:2368] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:26:53.869302  4678 rpcz_store.cc:276] 0709 12:26:52.660969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.457200 (+796231us) service_pool.cc:224] Handling call
0709 12:26:53.457542 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.482693 (+ 25151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.482729 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:53.483316 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.745887 (+262571us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:53.745944 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.746038 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.868634 (+122596us) raft_consensus.cc:1685] finished
0709 12:26:53.868655 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.868803 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":739,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":12576,"thread_start_us":248001,"threads_started":1,"wal-append.queue_time_us":355167}
I20260709 12:26:53.870222  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:53.863637  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:53.873687  4755 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:53.852130  9850 raft_consensus.cc:3055] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:53.876965  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:53.773988  9966 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:53.856150  9772 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:53.882274  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:53.860172 10016 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:53.867666  9835 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:53.887125  9772 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:53.892091  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:53.855769  9963 consensus_queue.cc:1059] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:26:53.895560  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3221) took 1310 ms. Trace:
I20260709 12:26:53.899772  4756 rpcz_store.cc:276] 0709 12:26:52.585171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.703502 (+1118331us) service_pool.cc:224] Handling call
0709 12:26:53.703873 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.704232 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.704259 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:53.704688 (+   429us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.779686 (+ 74998us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:53.779759 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.779895 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.895321 (+115426us) raft_consensus.cc:1685] finished
0709 12:26:53.895345 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.895497 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12573,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":74535,"threads_started":1,"wal-append.queue_time_us":167291}
I20260709 12:26:53.886121 10025 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:53.884078  9588 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:53.898041  9772 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:53.880831  9850 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:53.886310  9931 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:53.892858 10066 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:53.898788 10022 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:53.902097  9765 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:53.907349  9931 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:53.917613  9966 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:53.922006  9965 consensus_meta.cc:287] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.587s	user 0.009s	sys 0.009s
I20260709 12:26:53.922435  9965 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:53.924029  9965 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:53.960182  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3228) took 1243 ms. Trace:
I20260709 12:26:53.961364  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:53.961102  4751 rpcz_store.cc:276] 0709 12:26:52.716721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.818022 (+1101301us) service_pool.cc:224] Handling call
0709 12:26:53.818298 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.818493 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:53.894594 (+ 76101us) log.cc:844] Serialized 14 byte log entry
0709 12:26:53.959033 (+ 64439us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:53.959175 (+   142us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:53.959334 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.959493 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.959789 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":102306,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"thread_start_us":136135,"threads_started":2,"wal-append.queue_time_us":19663}
I20260709 12:26:53.963194  4603 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:53.965399  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:53.966259  4751 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:26:53.967741  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3253) took 1537 ms. Trace:
I20260709 12:26:53.899459  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:53.968257  4672 rpcz_store.cc:276] 0709 12:26:52.430472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.740116 (+309644us) service_pool.cc:224] Handling call
0709 12:26:52.789671 (+ 49555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.848261 (+ 58590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.967154 (+1118893us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b54001bc950. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:53.967454 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113325952}
W20260709 12:26:53.970031  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 70508.9us
I20260709 12:26:53.974948  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3225) took 1322 ms. Trace:
I20260709 12:26:53.976187  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 853) took 1346 ms. Trace:
I20260709 12:26:53.977546  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 2062) took 1553 ms. Trace:
I20260709 12:26:53.978099  4679 rpcz_store.cc:276] 0709 12:26:52.424087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.537837 (+113750us) service_pool.cc:224] Handling call
0709 12:26:53.844079 (+1306242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.852377 (+  8298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.977071 (+124694us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540017eb50. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:53.977330 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3003281152}
I20260709 12:26:53.975575  4749 rpcz_store.cc:276] 0709 12:26:52.652008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.770842 (+1118834us) service_pool.cc:224] Handling call
0709 12:26:53.835529 (+ 64687us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.856160 (+ 20631us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:53.856206 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:53.856753 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.895837 (+ 39084us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:53.895876 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.895937 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.974193 (+ 78256us) raft_consensus.cc:1685] finished
0709 12:26:53.974252 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.974672 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21434,"raft.queue_time_us":21962,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":14616,"thread_start_us":38784,"threads_started":1,"wal-append.queue_time_us":56753}
I20260709 12:26:53.980645  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:53.976660  4677 rpcz_store.cc:276] 0709 12:26:52.629907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.335627 (+705720us) service_pool.cc:224] Handling call
0709 12:26:53.335875 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.532862 (+196987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.975609 (+442747us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b540007c954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:53.975979 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":451731328}
W20260709 12:26:53.984861  4750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2654) took 1705 ms (client timeout 1723 ms). Trace:
I20260709 12:26:53.985236  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:53.985550  4750 rpcz_store.cc:269] 0709 12:26:52.279314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.419625 (+140311us) service_pool.cc:224] Handling call
0709 12:26:53.984318 (+1564693us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b5400263254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:53.984592 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":542742784}
I20260709 12:26:53.986446  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:53.987260  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:53.987859  4755 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:26:53.988541  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:53.988932  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:53.986464  4749 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:54.043298  4603 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:54.117239  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2663) took 1655 ms. Trace:
I20260709 12:26:53.989285  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:54.001113  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 848) took 2121 ms. Trace:
I20260709 12:26:54.034561  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:53.992249  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3216) took 1913 ms. Trace:
I20260709 12:26:54.007432  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 2052) took 1598 ms. Trace:
W20260709 12:26:54.017712  4747 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2652) took 1738 ms (client timeout 2183 ms). Trace:
W20260709 12:26:54.039975  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.046657  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:53.999943  4604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2656) took 1764 ms (client timeout 2130 ms). Trace:
I20260709 12:26:54.009274  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:54.081725  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:54.179224  4754 rpcz_store.cc:276] 0709 12:26:52.079105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.045118 (+966013us) service_pool.cc:224] Handling call
0709 12:26:53.989677 (+944559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.990244 (+   567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.990286 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:53.990362 (+    76us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:53.990408 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.990514 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.991836 (+  1322us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b5400263250. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:53.992007 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2171131264}
I20260709 12:26:54.177064  4676 rpcz_store.cc:276] 0709 12:26:51.879465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.879585 (+   120us) service_pool.cc:224] Handling call
0709 12:26:51.879829 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.923029 (+1043200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.000519 (+1077490us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b5400085f54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:54.000878 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2054025600}
W20260709 12:26:54.142282  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.091666  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:54.025673  4571 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:54.075161  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:54.053580  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:53.990132  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:53.990303  4720 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:54.179780  4601 rpcz_store.cc:276] 0709 12:26:52.409280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.724065 (+314785us) service_pool.cc:224] Handling call
0709 12:26:54.007040 (+1282975us) spinlock_profiling.cc:243] Waited 955 ms on lock 0x7b540004f454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:54.007254 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2197086976}
W20260709 12:26:54.006115  4647 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.723s (SENT) [suppressed 10 similar messages]
W20260709 12:26:54.184897  4604 rpcz_store.cc:269] 0709 12:26:52.235392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.354282 (+118890us) service_pool.cc:224] Handling call
0709 12:26:53.999345 (+1645063us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b5400353ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:53.999712 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1634774272}
I20260709 12:26:54.208644  9742 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:54.183022  4747 rpcz_store.cc:269] 0709 12:26:52.278799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.556721 (+277922us) service_pool.cc:224] Handling call
0709 12:26:54.017235 (+1460514us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b540004cc54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:54.017488 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401169664}
I20260709 12:26:54.184376  9984 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:54.215817  4756 raft_consensus.cc:2393] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 8: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:54.123929  4568 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:54.219843  9207 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:26:54.220005  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 78744.1us
W20260709 12:26:54.220758  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.220994  9207 consensus_queue.cc:1059] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:54.222186  9853 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:54.134132  9667 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.915365  9931 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:54.159478  9899 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:54.218047  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:54.229995 10066 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:54.242605  9976 raft_consensus.cc:3055] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:54.243323  9976 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:54.244798  9801 consensus_meta.cc:287] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.000s
I20260709 12:26:54.209481  9963 raft_consensus.cc:3055] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:54.237905  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:54.248111  4598 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:54.226168  9664 consensus_meta.cc:287] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.991s	user 0.011s	sys 0.011s
I20260709 12:26:54.225244  9984 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.223031  9207 raft_consensus.cc:3055] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:54.247159  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:54.248610  4599 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:54.262583  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:54.175159  4748 rpcz_store.cc:276] 0709 12:26:52.461201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.676881 (+1215680us) service_pool.cc:224] Handling call
0709 12:26:53.685538 (+  8657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.828201 (+142663us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:53.984072 (+155871us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b18001555c0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c6455 00007efff1eae3c7 00007effee4c09a7 00007effee4d00bf 00007effee4d0038
0709 12:26:54.101986 (+117914us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.106335 (+  4349us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:54.107417 (+  1082us) log.cc:844] Serialized 26 byte log entry
0709 12:26:54.107625 (+   208us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:54.107684 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.108430 (+   746us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540004db54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:54.108553 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:54.116723 (+  8170us) raft_consensus.cc:1685] finished
0709 12:26:54.116745 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.117020 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13084,"prepare.queue_time_us":92815,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":887,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":15841,"spinlock_wait_cycles":338071680,"thread_start_us":121196,"threads_started":2,"wal-append.queue_time_us":4194}
I20260709 12:26:54.236979  9850 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.263154  4602 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:26:54.267935  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:54.278496  4597 raft_consensus.cc:2368] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:26:54.279750  4645 leader_election.cc:400] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17.
I20260709 12:26:54.280123  9876 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:26:54.281675  9966 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:54.289139  9667 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:54.290268  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 300217us
I20260709 12:26:54.292308  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:54.280153  4645 leader_election.cc:403] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:26:54.293174  9730 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:54.265452  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:54.297986  4596 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:54.260344  9963 raft_consensus.cc:740] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:54.244941  9976 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.297538  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:54.265998  9984 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:54.296834  9972 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.268110 10012 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:54.268222  9892 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:54.307224  4747 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:54.307792  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:54.267244  9207 raft_consensus.cc:740] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:54.306059  4601 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:54.305114  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:54.267887  9955 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.314788 10025 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.315254  9776 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:54.318470  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49627.3us
I20260709 12:26:54.319089  9740 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:54.320489  9990 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:54.269026  9664 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:54.267582  9835 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.093s
I20260709 12:26:54.305107  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:54.322517  9977 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:54.327229  9999 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.020s	sys 0.009s
I20260709 12:26:54.352159 10009 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:54.352989  4604 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:26:54.354415  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 230894us
W20260709 12:26:54.356068  9661 consensus_meta.cc:287] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.561s	user 0.011s	sys 0.009s
I20260709 12:26:54.356623  9661 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:54.324411  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:54.341125  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 44581.3us
I20260709 12:26:54.321481  5344 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:26:54.339962  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.345554  4751 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
W20260709 12:26:54.341517  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 316252us
I20260709 12:26:54.322288  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:54.349678  9966 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.350934  4605 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:54.351073  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:54.351475  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:54.302279  9899 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.305970 10012 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:54.352483  9999 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.021s	sys 0.014s
I20260709 12:26:54.341149  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:54.363057  9947 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.383087  4749 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 8.
I20260709 12:26:54.363801 10046 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:54.364223  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:54.364461  9765 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:54.365929  9740 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.399295  9207 consensus_queue.cc:260] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.400764  9740 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:54.402184  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.340350  9892 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:54.368110  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:54.368752  9952 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.322925  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:54.370141  9757 consensus_queue.cc:260] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:54.374761  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:54.386983  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:54.391991  9765 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.398631  5342 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:54.408948  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 402973us
I20260709 12:26:54.384950 10012 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:54.398018  9976 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:54.401401  9207 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:54.407944  4602 raft_consensus.cc:2393] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:54.310225  9972 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:54.414156  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.395777  4747 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:54.415867  9661 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:54.416671  9757 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:54.439788  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:26:54.440943  9951 consensus_meta.cc:287] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.614s	user 0.005s	sys 0.009s
I20260709 12:26:54.397497  9947 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.441195  9952 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.443557  9947 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:54.446144  9850 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:54.411753  9765 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:54.454212  4672 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:54.456570  4677 pending_rounds.cc:85] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 9
I20260709 12:26:54.456962  4677 pending_rounds.cc:107] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:26:54.457252  4677 raft_consensus.cc:2889] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:54.424158  4597 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
W20260709 12:26:54.423671  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 48439.5us
I20260709 12:26:54.349454  9664 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:54.460565  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:54.461824  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:54.462146  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:54.464147  4681 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:26:54.464500  4676 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:54.413522  4754 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
W20260709 12:26:54.470381  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.472487  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:54.474674  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:54.429970  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 55332.2us
I20260709 12:26:54.394804  9667 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:54.311326  9730 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.437028  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:54.438364  4753 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:54.439152 10046 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:54.446409  9772 consensus_meta.cc:287] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.003s
I20260709 12:26:54.398142  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:54.450407  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:54.432013  4598 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:26:54.481900  4601 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:26:54.481954  4596 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:54.483088  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:54.353087 10009 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.489382  4681 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:26:54.437213  9740 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:54.504547  4600 raft_consensus.cc:1240] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:54.507409  4604 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:54.509130  4568 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:54.505271  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:54.456635  9882 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:54.389320  9955 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:54.501461  9735 consensus_meta.cc:287] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.616s	user 0.015s	sys 0.003s
I20260709 12:26:54.494473  9835 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:54.517326  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3227) took 1655 ms. Trace:
I20260709 12:26:54.411468 10025 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:54.413314  9776 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.530335  4603 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:54.524546  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:54.521241  5341 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:54.526736  9740 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.530705  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:54.507160  9963 consensus_queue.cc:260] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.534582  9733 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:54.565796  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:54.566535  9963 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:54.566972  4748 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:54.510113  9772 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.522178  4752 rpcz_store.cc:276] 0709 12:26:52.688225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.816110 (+1127885us) service_pool.cc:224] Handling call
0709 12:26:53.816379 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.816596 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:53.898602 (+ 82006us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.262999 (+364397us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.295039 (+ 32040us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:54.299873 (+  4834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.306413 (+  6540us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.343231 (+ 36818us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9516,"prepare.queue_time_us":96475,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":553,"raft.queue_time_us":15175,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":31903,"thread_start_us":394417,"threads_started":2,"wal-append.queue_time_us":300012}
I20260709 12:26:54.570777  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:54.571828  9772 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:54.572897 10008 consensus_meta.cc:287] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.782s	user 0.004s	sys 0.009s
I20260709 12:26:54.534976  5341 raft_consensus.cc:606] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:54.537766  4749 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:54.513721  9951 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:54.552071  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:54.549088  5348 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:54.559723  4673 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:54.560578  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 48959.6us
I20260709 12:26:54.582293  4753 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:54.595958  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:54.625692  4752 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:54.625907  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 116872us
I20260709 12:26:54.536619  9961 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:54.561800  9661 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:54.550359  9603 raft_consensus.cc:3055] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:54.539893  9955 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:54.551852  9740 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:54.628561  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:54.559137  9966 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:54.544843 10064 log.cc:927] Time spent T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.676s	user 0.002s	sys 0.000s
I20260709 12:26:54.546326  9892 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:54.536727  9972 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:54.586176  9730 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:54.544696 10009 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:54.640895  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:54.618724  9951 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:26:54.641880  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 114311us
I20260709 12:26:54.643617  5349 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:54.559285  9776 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:54.646186  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:54.677199  5342 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:26:54.686743  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:54.689731  4680 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:54.689628  9871 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.176s
I20260709 12:26:54.558218  5343 raft_consensus.cc:2955] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index 3 to 10, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:54.635707  9603 raft_consensus.cc:740] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:54.649874  9735 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:54.698230  9588 consensus_meta.cc:287] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.790s	user 0.013s	sys 0.000s
I20260709 12:26:54.697721  9931 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:54.645897  9733 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:54.699163 10016 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:54.650715  9984 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:54.705791  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:54.544847  9797 consensus_queue.cc:1059] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.117s
I20260709 12:26:54.552215  9882 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.730873  4602 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:54.579728 10008 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:54.733924  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.742471  4748 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
W20260709 12:26:54.746367  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.758662  4599 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
W20260709 12:26:54.766914  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 137882us
I20260709 12:26:54.542775 10046 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:54.794836  4755 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 17: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:54.783499  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:54.693245 10012 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:54.792809  9984 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:54.802189  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.803157  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 56824.7us
I20260709 12:26:54.792574  9765 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:54.797434  9589 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.812004  4756 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 16: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:54.776229  9733 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.819257  4750 raft_consensus.cc:2410] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:26:54.821274  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:54.817301  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:54.835491  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2676) took 1279 ms. Trace:
I20260709 12:26:54.836150  4605 rpcz_store.cc:276] 0709 12:26:53.556004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.556124 (+   120us) service_pool.cc:224] Handling call
0709 12:26:53.556380 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.405557 (+849177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.812295 (+406738us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b54002e59d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:54.835365 (+ 23070us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1826261888}
I20260709 12:26:54.838055  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:54.837747  9735 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.839742  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:54.824615  9733 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:54.843804 10081 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:26:54.845744 10008 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:54.846446 10008 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:54.846655  9772 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.847730  4571 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:54.847471  9772 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.825767 10016 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:54.802526 10066 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:54.849547  4571 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:54.851850  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:54.851886  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:54.853107  4674 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:54.859946  4680 raft_consensus.cc:2410] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:26:54.841144  9735 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:54.855873  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:54.872974  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:54.906339  4751 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 17: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:26:54.860365 10079 log.cc:927] Time spent T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:26:54.859694  9931 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:54.926646  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 85186.4us
I20260709 12:26:54.860632 10066 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:54.919512  4756 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:26:54.857812  9899 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:54.850960  9882 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:54.920459  4678 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:26:54.931381  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:54.938905  4749 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:26:54.851092  9805 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:54.852430 10006 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:54.935832  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 86395.8us
I20260709 12:26:54.853466  9936 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:54.941113  9735 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:54.914727 10012 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:54.917547  9984 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:54.855023 10016 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:54.932667  4680 pending_rounds.cc:85] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 6
I20260709 12:26:54.953047  4680 pending_rounds.cc:107] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:26:54.858407  9733 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.953450  4680 raft_consensus.cc:2889] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:54.948303  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:54.933138 10066 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:54.954747  9871 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:54.930382  9588 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:54.860096  9589 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.954814  9735 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:54.958652  9805 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:54.959272  9984 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:54.966059  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:54.960979  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 2074) took 1377 ms. Trace:
I20260709 12:26:54.943377  9661 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:54.967008  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:54.967713  4674 rpcz_store.cc:276] 0709 12:26:53.583253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.793135 (+209882us) service_pool.cc:224] Handling call
0709 12:26:53.793450 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.923710 (+1130260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.960676 (+ 36966us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540038e5d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:54.960915 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2358019968}
I20260709 12:26:54.849910  9951 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:54.957770  9871 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:54.969538  9990 consensus_meta.cc:287] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.603s	user 0.012s	sys 0.000s
I20260709 12:26:54.979171  4677 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:54.982028  9805 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:54.986984  9984 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:54.989138  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 46020us
I20260709 12:26:54.990005  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:54.994074  9772 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:54.993780  9735 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.000852  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:55.004269  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 148548us
I20260709 12:26:54.962306  9963 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.967054 10016 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:55.007972  4749 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 15: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:54.977138 10025 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:55.011057  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 3234) took 1152 ms. Trace:
I20260709 12:26:55.013924  4749 rpcz_store.cc:276] 0709 12:26:53.859081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.966675 (+1107594us) service_pool.cc:224] Handling call
0709 12:26:55.010978 (+ 44303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.010291  9805 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:26:54.982645  9871 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:55.016491  9733 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:55.018270 10077 log.cc:927] Time spent T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.504s	user 0.004s	sys 0.001s
W20260709 12:26:55.019873 10080 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:26:54.977591  9588 raft_consensus.cc:697] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:55.023289  5345 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:55.014483  4678 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:55.003916  9990 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17.
I20260709 12:26:55.027905  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:54.963946  9603 consensus_queue.cc:260] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:55.038236  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:55.056818  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.027309  9936 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:26:55.061827  9946 consensus_meta.cc:287] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.112s	user 0.003s	sys 0.015s
I20260709 12:26:55.065155  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:55.595996  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 573352us
W20260709 12:26:55.597661  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.598349  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.087817  4672 raft_consensus.cc:2410] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:26:55.077800  4674 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:55.083153  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:55.607944  4749 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:55.062885  4752 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:26:55.765235 10103 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:55.081182  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.064476  5346 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:55.272125  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 3235) took 1412 ms. Trace:
W20260709 12:26:55.769356  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.770488  4751 rpcz_store.cc:276] 0709 12:26:53.859440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.397626 (+538186us) service_pool.cc:224] Handling call
0709 12:26:55.271911 (+874285us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54000bf4d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:55.272059 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":945751808}
I20260709 12:26:55.765352  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 3249) took 1313 ms. Trace:
I20260709 12:26:55.771710  4752 rpcz_store.cc:276] 0709 12:26:54.451491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.570430 (+118939us) service_pool.cc:224] Handling call
0709 12:26:55.765128 (+1194698us) spinlock_profiling.cc:243] Waited 54.3 ms on lock 0x7b540004d654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:55.765284 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124907904}
W20260709 12:26:55.383042  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.466522  4681 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:26:55.500897  5345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 718) took 1178 ms. Trace:
I20260709 12:26:55.513181 10097 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:55.508312  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 493146us
I20260709 12:26:55.777245 10097 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:55.778162  4677 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.698s	user 0.012s	sys 0.000s
I20260709 12:26:55.772356  4751 raft_consensus.cc:2955] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index 3 to 10, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:26:55.779610 10097 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:55.568881  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.535385  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 861) took 1113 ms. Trace:
I20260709 12:26:55.550211  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3265) took 1090 ms. Trace:
I20260709 12:26:55.776922  5345 rpcz_store.cc:276] 0709 12:26:54.322489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.450137 (+127648us) service_pool.cc:224] Handling call
0709 12:26:55.500581 (+1050444us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b5400782dd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:55.500762 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1085607168}
I20260709 12:26:55.781714  4673 rpcz_store.cc:276] 0709 12:26:54.459898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.460125 (+   227us) service_pool.cc:224] Handling call
0709 12:26:55.549910 (+1089785us) spinlock_profiling.cc:243] Waited 98.1 ms on lock 0x7b54000b0254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:55.550111 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225737472}
I20260709 12:26:55.782301  4677 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:26:55.785008  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:55.583081  4680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.009s	sys 0.002s
W20260709 12:26:55.785630  4680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.011s	sys 0.002s
I20260709 12:26:55.785661  4677 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:55.786450  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3274) took 1150 ms. Trace:
I20260709 12:26:55.787037  9740 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.787727  9740 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:55.786835  4680 rpcz_store.cc:276] 0709 12:26:54.640202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.917069 (+276867us) service_pool.cc:224] Handling call
0709 12:26:54.929695 (+ 12626us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:54.998602 (+ 68907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.998635 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:55.011017 (+ 12382us) log.cc:844] Serialized 52 byte log entry
0709 12:26:55.786142 (+775125us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:55.786180 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.786242 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:55.786280 (+    38us) raft_consensus.cc:1685] finished
0709 12:26:55.786295 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.786400 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32656,"raft.queue_time_us":20402,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":6577,"thread_start_us":725113,"threads_started":1,"wal-append.queue_time_us":232227}
I20260709 12:26:55.255103  9661 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:55.798723  5346 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 715) took 1477 ms. Trace:
I20260709 12:26:55.085969  9936 raft_consensus.cc:697] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:55.058939  9947 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:55.366878  9952 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:55.494218  9977 raft_consensus.cc:993] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:26:55.383558  9984 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.831007  9984 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.072212  9588 consensus_queue.cc:237] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:55.072438  9589 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:55.383188  9735 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.843453  9735 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:55.022815  9963 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:55.025489 10016 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.849212 10016 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:55.065766 10025 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:55.854473  9850 raft_consensus.cc:1081] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:26:55.261792 10066 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.080775  9765 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:55.255196  9871 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:55.857765 10066 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:55.859347  9871 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:55.531697 10091 log.cc:927] Time spent T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.509s	user 0.002s	sys 0.000s
I20260709 12:26:55.600361 10098 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:55.617626  4749 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:55.862912  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:55.864765  4749 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:55.618757  4755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 3233) took 1760 ms (client timeout 1892 ms). Trace:
I20260709 12:26:55.620368 10099 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:55.620620  5347 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 12:26:55.866115 10099 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:55.865388  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 989) took 2036 ms. Trace:
I20260709 12:26:55.866966  4749 rpcz_store.cc:276] 0709 12:26:53.829062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.864216 (+2035154us) service_pool.cc:224] Handling call
0709 12:26:55.864577 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.865158 (+   581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.865317 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.867775  9735 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:55.868733 10099 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:55.621079  4569 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:55.631317  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:55.632131  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:55.869958  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 877) took 1296 ms. Trace:
I20260709 12:26:55.870421  4605 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:26:55.870704  9984 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:55.870481  4601 rpcz_store.cc:276] 0709 12:26:54.573881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.624102 (+1050221us) service_pool.cc:224] Handling call
0709 12:26:55.624491 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.867253 (+242762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:55.867296 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:55.867340 (+    44us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:55.867377 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.869637 (+  2260us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54002e54d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:55.869698 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.869844 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557802368}
I20260709 12:26:55.871640  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:55.872284  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:55.872781  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:55.634666  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 871) took 1160 ms. Trace:
I20260709 12:26:55.869745  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 990) took 2040 ms. Trace:
I20260709 12:26:55.873558  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:55.636721  4750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 3238) took 1776 ms (client timeout 1871 ms). Trace:
I20260709 12:26:55.873819  4749 rpcz_store.cc:276] 0709 12:26:53.829333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.868658 (+2039325us) service_pool.cc:224] Handling call
0709 12:26:55.869086 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.869324 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:55.869355 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:55.869421 (+    66us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:55.869492 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.869548 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.869684 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:55.874694  4750 rpcz_store.cc:269] 0709 12:26:53.860421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.988525 (+128104us) service_pool.cc:224] Handling call
0709 12:26:55.636517 (+1647992us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b54000555d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:55.636660 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1429273856}
I20260709 12:26:55.873710  4602 rpcz_store.cc:276] 0709 12:26:54.474236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.474375 (+   139us) service_pool.cc:224] Handling call
0709 12:26:54.507008 (+ 32633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.794043 (+287035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.633362 (+839319us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b5400033754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:55.634394 (+  1032us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b5400033750. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:55.634543 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530868736}
I20260709 12:26:55.876439  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:55.877045  4602 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:55.877111  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 991) took 2047 ms. Trace:
I20260709 12:26:55.650144  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:55.865763  4755 rpcz_store.cc:269] 0709 12:26:53.858721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.988880 (+130159us) service_pool.cc:224] Handling call
0709 12:26:55.618594 (+1629714us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b5400263754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:55.618707 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422848896}
I20260709 12:26:55.651768  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:55.878798  4755 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:55.879514  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 993) took 2049 ms. Trace:
I20260709 12:26:55.879771  4601 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:55.879913  4755 rpcz_store.cc:276] 0709 12:26:53.830145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.878341 (+2048196us) service_pool.cc:224] Handling call
0709 12:26:55.878647 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.879272 (+   625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.879454 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.654345  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:55.881078  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:55.680692 10009 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:55.882073 10009 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:55.685242  4676 consensus_meta.cc:287] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.941s	user 0.001s	sys 0.016s
I20260709 12:26:55.884151  4676 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:26:55.884336 10009 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:55.884653  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48198 (request call id 865) took 1263 ms. Trace:
I20260709 12:26:55.885002  4676 rpcz_store.cc:276] 0709 12:26:54.620783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.686360 (+ 65577us) service_pool.cc:224] Handling call
0709 12:26:55.884594 (+1198234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.693035  4678 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:55.882447  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:55.869961  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 248974us
I20260709 12:26:55.887050  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:55.887831  4678 raft_consensus.cc:2393] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:55.887923  9735 consensus_queue.cc:1059] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:55.888737  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:55.888890 10100 consensus_queue.cc:1059] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:55.696859  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3262) took 1238 ms. Trace:
I20260709 12:26:55.701828  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:55.701164  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:55.890025 10100 raft_consensus.cc:3055] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:55.890184  4754 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:55.704731  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 873) took 1229 ms. Trace:
I20260709 12:26:55.890972  9871 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:55.892032  4596 raft_consensus.cc:2393] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 14: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:55.719285  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:55.889824  4672 rpcz_store.cc:276] 0709 12:26:54.458811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.491894 (+ 33083us) service_pool.cc:224] Handling call
0709 12:26:55.696611 (+1204717us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b54002df854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:55.696782 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1303860096}
I20260709 12:26:55.890723  4600 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:55.729059  9603 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:55.891016 10098 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:55.893702 10098 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:55.739672  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.897962  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.743350  4747 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:26:55.898501 10098 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.769789  9772 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:55.900034  9850 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.900592  9850 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:55.902000 10098 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:55.778112  4647 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:55.514580  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 485223us
I20260709 12:26:55.905045  4720 leader_election.cc:400] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:26:55.905318  4720 leader_election.cc:403] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:55.906497 10025 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:55.781174  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.907094 10025 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:55.909334  9765 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
W20260709 12:26:55.571537  5349 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53808 (request call id 633) took 1968 ms (client timeout 2237 ms). Trace:
I20260709 12:26:55.382993  9882 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:55.911214  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:55.911474  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:55.911330  9882 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.789734  9955 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.790331  9983 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.781553 10097 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.791687  4677 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:26:55.791690  9740 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:55.781462  4679 rpcz_store.cc:276] 0709 12:26:54.422298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.432981 (+ 10683us) service_pool.cc:224] Handling call
0709 12:26:54.455741 (+ 22760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.572989 (+117248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.573016 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.573078 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:54.573113 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.535072 (+961959us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400085554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:55.535130 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.535308 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":269079296}
I20260709 12:26:55.793730 10008 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:55.794816  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 133140us
I20260709 12:26:55.917584 10008 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.919868 10008 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:55.920291  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:55.058466  9892 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.936204  9892 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.803172  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3272) took 1282 ms. Trace:
I20260709 12:26:55.797868  9961 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:55.811702  9990 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:55.814949 10092 log.cc:927] Time spent T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.820s	user 0.000s	sys 0.008s
I20260709 12:26:55.943341  9990 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.797788  4644 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:55.820389  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:55.822806  9936 consensus_queue.cc:237] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:55.945508  9990 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:55.948128  9990 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:55.821913  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:55.949419  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:55.949860  4680 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:55.950829  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:55.819558  5346 rpcz_store.cc:276] 0709 12:26:54.321037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.321822 (+   785us) service_pool.cc:224] Handling call
0709 12:26:55.798540 (+1476718us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54004fb0d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:55.798655 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407893376}
I20260709 12:26:55.819782  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2674) took 2894 ms. Trace:
I20260709 12:26:55.954103  4751 rpcz_store.cc:276] 0709 12:26:52.921547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.771606 (+2850059us) service_pool.cc:224] Handling call
0709 12:26:55.771973 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.772237 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:55.788106 (+ 15869us) log.cc:844] Serialized 14 byte log entry
0709 12:26:55.815777 (+ 27671us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:55.815829 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:55.815868 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.815932 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.816058 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":917,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":109,"thread_start_us":27059,"threads_started":1,"wal-append.queue_time_us":31306}
I20260709 12:26:55.955864  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:55.956485  4751 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:26:55.073875 10021 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:55.851161  9963 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:55.869861  9977 consensus_queue.cc:1059] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:26:55.974550 10021 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.635643  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:55.877763  4749 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:55.975354  9892 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:55.975745  5340 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:55.975996  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 992) took 2146 ms. Trace:
I20260709 12:26:55.976419  4749 rpcz_store.cc:276] 0709 12:26:53.829861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.877148 (+2047287us) service_pool.cc:224] Handling call
0709 12:26:55.877494 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.975717 (+ 98223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.975905 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.976827  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:55.977080 10021 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:55.977535  5340 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:55.977815  4749 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 13. Current term is 15. Ops: []
I20260709 12:26:55.978544  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 996) took 2147 ms. Trace:
I20260709 12:26:55.978963  4680 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:55.978876  4749 rpcz_store.cc:276] 0709 12:26:53.830896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.977298 (+2146402us) service_pool.cc:224] Handling call
0709 12:26:55.977665 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.978315 (+   650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.978495 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.878830  4753 raft_consensus.cc:2393] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 15: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:55.660460  9946 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:55.890612 10100 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:55.890791  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2682) took 2008 ms. Trace:
I20260709 12:26:55.984510  4754 rpcz_store.cc:276] 0709 12:26:53.882369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.701540 (+1819171us) service_pool.cc:224] Handling call
0709 12:26:55.890734 (+189194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.703246  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 868) took 1318 ms. Trace:
I20260709 12:26:55.891145  4598 rpcz_store.cc:276] 0709 12:26:54.474862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.507741 (+ 32879us) service_pool.cc:224] Handling call
0709 12:26:54.514194 (+  6453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.524961 (+ 10767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.525010 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.532036 (+  7026us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:54.532106 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.704445 (+1172339us) spinlock_profiling.cc:243] Waited 4.92 ms on lock 0x7b5400343854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:55.704508 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.704654 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11322624}
W20260709 12:26:55.707696  5343 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.005s	sys 0.009s
W20260709 12:26:55.985594  5343 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.006s	sys 0.009s
I20260709 12:26:55.985302  4604 rpcz_store.cc:276] 0709 12:26:54.392851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.405697 (+ 12846us) service_pool.cc:224] Handling call
0709 12:26:54.503689 (+ 97992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.514504 (+ 10815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.703032 (+1188528us) spinlock_profiling.cc:243] Waited 92.3 ms on lock 0x7b54003520d0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:55.703180 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212241792}
I20260709 12:26:55.986532  5343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 717) took 1664 ms. Trace:
I20260709 12:26:55.986826  5343 rpcz_store.cc:276] 0709 12:26:54.322018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.513531 (+191513us) service_pool.cc:224] Handling call
0709 12:26:54.527133 (+ 13602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.557918 (+ 30785us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:55.024080 (+466162us) log.cc:844] Serialized 14 byte log entry
0709 12:26:55.986180 (+962100us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:55.986231 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:55.986279 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.986344 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.986458 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":921682,"threads_started":1,"wal-append.queue_time_us":217218}
I20260709 12:26:55.988420  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:55.989017  5343 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:55.892675  4599 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:55.723145  4571 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:55.989401  4674 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:55.990490  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 267477us
I20260709 12:26:55.991078  4568 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:55.991883  9961 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:55.992585  9961 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:55.730625 10102 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:55.992937  9961 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:55.894096  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:55.894901  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:55.993335 10102 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.993362  9961 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:55.975405  9977 raft_consensus.cc:3055] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:55.996155  4500 catalog_manager.cc:5697] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:55.997164  9977 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:55.879272  4750 rpcz_store.cc:276] 0709 12:26:53.829605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.876055 (+2046450us) service_pool.cc:224] Handling call
0709 12:26:55.876466 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.876762 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:55.876792 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:55.876843 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:55.876877 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.876934 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.877054 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.998090  9977 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:55.999372  9977 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:55.994890  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:55.895869  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 994) took 2065 ms. Trace:
I20260709 12:26:56.001354  4755 rpcz_store.cc:276] 0709 12:26:53.830415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.892846 (+2062431us) service_pool.cc:224] Handling call
0709 12:26:55.893178 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.895523 (+  2345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:55.895550 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:55.895594 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:55.895631 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.895690 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.895788 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.899428  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2671) took 3139 ms. Trace:
I20260709 12:26:56.002395  4747 rpcz_store.cc:276] 0709 12:26:52.759536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.742564 (+2983028us) service_pool.cc:224] Handling call
0709 12:26:55.743057 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:55.899157 (+156100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.899374 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.900190  9772 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:55.910244  9765 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:26:55.910171  5349 rpcz_store.cc:269] 0709 12:26:53.603214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.603652 (+   438us) service_pool.cc:224] Handling call
0709 12:26:55.571304 (+1967652us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b54007116d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:55.571464 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2198685568}
I20260709 12:26:55.858438  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:55.906801  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:56.007500  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 100821us
I20260709 12:26:56.008495  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:55.766155 10103 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.009501 10104 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:56.010396 10104 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:56.011588 10103 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:55.915045  4677 pending_rounds.cc:85] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 9
I20260709 12:26:56.012408  4677 pending_rounds.cc:107] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:26:56.012712  4677 raft_consensus.cc:2889] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:56.013998  4751 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:56.015219  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 995) took 2184 ms. Trace:
I20260709 12:26:56.015556  4751 rpcz_store.cc:276] 0709 12:26:53.830664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.961326 (+2130662us) service_pool.cc:224] Handling call
0709 12:26:55.981276 (+ 19950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.014406 (+ 33130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.014982 (+   576us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b540030aa54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:56.015164 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74417408}
I20260709 12:26:55.922046  9776 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:56.017776  9776 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.018214  9776 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:55.920347 10008 consensus_queue.cc:1059] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:26:56.022171 10008 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:56.022590 10008 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:56.023276 10008 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.024336 10008 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:55.923657  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:26:56.026293  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 104776us
I20260709 12:26:56.026911  9892 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:56.027390  9892 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.027915  9892 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:55.924649  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:56.029608  5348 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:55.924420 10097 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.924916  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:55.925216  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:26:55.925962  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 147932us
I20260709 12:26:56.031612  5347 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:56.017422  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3240) took 2156 ms. Trace:
I20260709 12:26:56.032207  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:56.032430  4751 rpcz_store.cc:276] 0709 12:26:53.860970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.016519 (+2155549us) service_pool.cc:224] Handling call
0709 12:26:56.016889 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.017089 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.017115 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.017157 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:56.017197 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.017250 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.017346 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.926448 10066 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:56.034229 10066 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:56.034690 10066 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:56.035808 10066 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.926687  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:55.931216 10054 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.902835  9850 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:55.924666  9983 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:55.915186 10025 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:55.934831  4675 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:55.817807  4756 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:55.938143  4681 rpcz_store.cc:276] 0709 12:26:54.520839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.520977 (+   138us) service_pool.cc:224] Handling call
0709 12:26:55.802892 (+1281915us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b540038f4d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:55.803081 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1141866240}
W20260709 12:26:55.938287  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49343.1us
I20260709 12:26:55.940629  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3230) took 2641 ms. Trace:
I20260709 12:26:55.927973 10016 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:55.927742  9955 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:55.957679  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.947369  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 149671us
I20260709 12:26:56.040057  9588 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:56.038264 10054 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.042922  4644 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:56.043548  5344 raft_consensus.cc:1813] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Handling vote request from an unknown peer 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:56.044175  4644 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:56.044489  9990 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:55.947288  9882 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:56.044905 10054 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:55.967346  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:56.045120  9951 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:55.974004  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:56.041415  9983 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:55.873356  4605 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:56.045859  9951 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:56.046761  4676 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:56.047122  5346 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 15: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:56.047358  9951 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.048077  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:56.048856  5346 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:26:56.041966  4752 rpcz_store.cc:276] 0709 12:26:53.299023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.772608 (+2473585us) service_pool.cc:224] Handling call
0709 12:26:55.773004 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.773330 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:55.916156 (+142826us) log.cc:844] Serialized 14 byte log entry
0709 12:26:55.939833 (+ 23677us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:55.939918 (+    85us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:55.939995 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.940332 (+   337us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.940547 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":142579,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":169,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":162506,"threads_started":2,"wal-append.queue_time_us":20339}
I20260709 12:26:55.997934  4500 catalog_manager.cc:5697] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 7 to 9, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 9 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:55.770984  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.050190  4752 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:26:56.050580  4752 pending_rounds.cc:85] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 8
I20260709 12:26:56.046295  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:56.050976  4752 pending_rounds.cc:107] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:26:56.051174  9955 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:55.960850  9603 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.040779  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2691) took 1416 ms. Trace:
I20260709 12:26:56.037979  5345 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
W20260709 12:26:56.045995  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 87910.5us
I20260709 12:26:56.044222  5344 raft_consensus.cc:2410] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:26:56.053880  9946 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.055601  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.056123  9946 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:56.057077  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.057494  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.049814  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:56.057943  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.058140  4604 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:56.058573  4597 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:56.059329  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.059543  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.056385  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.061122  9882 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.042233 10097 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:56.061596  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:56.061748  9735 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:56.062310  9735 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.062716  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.063306  4753 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:56.064704  9735 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:56.064925  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3244) took 2195 ms. Trace:
I20260709 12:26:56.065063  4605 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:56.065717  9946 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.066308  9588 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.037528 10066 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:56.066895  9588 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.066267  9946 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.061626  9882 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.070783  4644 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:56.071357  9882 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:56.072220  4644 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:56.073107  9990 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:56.073706 10009 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:56.069137  9588 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:56.074910 10100 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.039829  4751 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:56.076195  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.076633  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3241) took 2215 ms. Trace:
I20260709 12:26:56.077018  4751 rpcz_store.cc:276] 0709 12:26:53.861257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.033527 (+2172270us) service_pool.cc:224] Handling call
0709 12:26:56.039469 (+  5942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.076304 (+ 36835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.076574 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.077447  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.078267  4751 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:56.076216 10100 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:56.079025  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 997) took 1935 ms. Trace:
I20260709 12:26:56.079411  4751 rpcz_store.cc:276] 0709 12:26:54.144381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.077807 (+1933426us) service_pool.cc:224] Handling call
0709 12:26:56.078105 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.078792 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.078975 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.080523  4751 raft_consensus.cc:1240] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:56.074653  9946 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:56.078238  4598 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:56.051236  4752 raft_consensus.cc:2889] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:56.086058  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3243) took 2224 ms. Trace:
I20260709 12:26:56.086470  4752 rpcz_store.cc:276] 0709 12:26:53.862021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.049482 (+2187461us) service_pool.cc:224] Handling call
0709 12:26:56.049879 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.085815 (+ 35936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.086007 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.095295  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 998) took 1876 ms. Trace:
W20260709 12:26:56.081358  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.094218  9765 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:56.109861  4673 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 15: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:56.109622  4751 rpcz_store.cc:276] 0709 12:26:54.213264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.080092 (+1866828us) service_pool.cc:224] Handling call
0709 12:26:56.080366 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.080830 (+   464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.080993 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.041812  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:56.052235  4756 rpcz_store.cc:276] 0709 12:26:54.624709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.953266 (+328557us) service_pool.cc:224] Handling call
0709 12:26:56.040621 (+1087355us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b54001f2554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:56.040725 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1957173632}
I20260709 12:26:56.111054  4645 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:56.100328  5340 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:56.111904  4645 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:56.113504  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:56.042233  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:56.113950  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2672) took 3193 ms. Trace:
I20260709 12:26:56.114502  4748 rpcz_store.cc:276] 0709 12:26:52.920946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.766098 (+2845152us) service_pool.cc:224] Handling call
0709 12:26:55.766524 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.093648 (+327124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.093685 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.093761 (+    76us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:56.093792 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.113689 (+ 19897us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54001f1154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:56.113751 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.113890 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538990848}
I20260709 12:26:56.065367  4753 rpcz_store.cc:276] 0709 12:26:53.869141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.064070 (+2194929us) service_pool.cc:224] Handling call
0709 12:26:56.064377 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.064632 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.064657 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.064697 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:56.064733 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.064782 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.064877 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.119426  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:56.122031  5342 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:26:56.126566  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.130931  4644 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:56.134341  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 101497us
I20260709 12:26:56.136443  4568 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:56.137498  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.139567  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.140435  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.141011  4602 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:56.136631  4756 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:55.996170 10102 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:56.042302 10016 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:56.142825  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.146169  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:56.138806  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 82129.2us
I20260709 12:26:56.140096  4754 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:56.147322  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:56.108335  9963 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:56.156826  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.149507  4645 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:56.110495  9765 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:56.163731  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.164651  4747 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:56.153223  4748 raft_consensus.cc:2410] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:26:56.165346  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.153203  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:56.169269  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.151809  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.172621  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:56.170192  4756 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:56.128340  9990 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:56.175976  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:56.173560  5344 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:56.162616  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:56.129464  9946 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:56.180142  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 138405us
I20260709 12:26:56.161468  4647 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:56.135720  9955 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:56.174667  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3237) took 2314 ms. Trace:
I20260709 12:26:56.185716  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.175401  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:56.186024  4755 rpcz_store.cc:276] 0709 12:26:53.860143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.002261 (+2142118us) service_pool.cc:224] Handling call
0709 12:26:56.146327 (+144066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.174313 (+ 27986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.174341 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.174391 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:56.174440 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.174508 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.174614 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.117659  9984 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:26:56.180086  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:56.187391  5340 raft_consensus.cc:2393] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 15: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:56.156889  9882 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:56.108664  9603 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.163940  9963 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.183562  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.146812  9952 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:56.218282  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3236) took 2358 ms. Trace:
I20260709 12:26:56.137676 10021 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:56.219240 10021 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.219767 10021 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:56.218838  4750 rpcz_store.cc:276] 0709 12:26:53.859781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.999060 (+2139279us) service_pool.cc:224] Handling call
0709 12:26:55.999489 (+   429us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:55.999946 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:55.999972 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:56.000541 (+   569us) log.cc:844] Serialized 52 byte log entry
0709 12:26:56.141789 (+141248us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:56.141845 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.141935 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:56.218028 (+ 76093us) raft_consensus.cc:1685] finished
0709 12:26:56.218049 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.218219 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5406,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":42376,"thread_start_us":140851,"threads_started":1,"wal-append.queue_time_us":208361}
I20260709 12:26:56.220772  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.183845  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:56.149997  9961 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.184886  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.222599  4747 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:56.194761  9882 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.223492  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.180030  9740 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:56.223945  9882 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:56.224324  9740 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.224496  4747 raft_consensus.cc:2393] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 15: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:56.224787  9740 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:56.185693  4753 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:26:56.226104  4753 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:56.226701  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3253) took 1652 ms. Trace:
I20260709 12:26:56.227022  4753 rpcz_store.cc:276] 0709 12:26:54.574301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.225587 (+1651286us) service_pool.cc:224] Handling call
0709 12:26:56.225915 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.226474 (+   559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.226646 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.227404  9955 raft_consensus.cc:697] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:56.228775  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3254) took 1654 ms. Trace:
I20260709 12:26:56.207809 10089 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:26:56.173756  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 122037us
I20260709 12:26:56.188207  4748 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:56.182399  4751 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: [15.8-17.9]
W20260709 12:26:56.230106  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 175219us
I20260709 12:26:56.230367  4601 raft_consensus.cc:2410] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:26:56.230542  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3250) took 1778 ms. Trace:
I20260709 12:26:56.129185  9892 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.151218 10112 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:56.230883  4751 rpcz_store.cc:276] 0709 12:26:54.451756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.166602 (+1714846us) service_pool.cc:224] Handling call
0709 12:26:56.182148 (+ 15546us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:56.230288 (+ 48140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.230493 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.162324  9871 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:56.211186  4755 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:26:56.211074  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 168925us
I20260709 12:26:56.212298  9776 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.213061  9603 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:56.212648  9765 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.283651  4751 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:56.214535  9772 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:56.180902  9946 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.182512  9735 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:56.214586  9963 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:56.218406  4752 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:56.184315  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:56.223250 10016 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.228480  9955 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.186319  4754 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:56.203504  4500 catalog_manager.cc:5697] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:56.181775  9990 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.230129  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3248) took 1779 ms. Trace:
I20260709 12:26:56.298879  4748 rpcz_store.cc:276] 0709 12:26:54.451165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.166039 (+1714874us) service_pool.cc:224] Handling call
0709 12:26:56.188027 (+ 21988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.229905 (+ 41878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.230079 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.229209  4753 rpcz_store.cc:276] 0709 12:26:54.574615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.227793 (+1653178us) service_pool.cc:224] Handling call
0709 12:26:56.228106 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.228421 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.228451 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.228499 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:56.228541 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.228597 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.228720 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.253531  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.269627 10089 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:56.270406  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:56.271095  9892 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:56.271314  9871 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.271814  4719 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:56.221395  4750 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:56.272526  9951 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:56.308303  9951 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.308569  9952 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:56.308739  9951 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:56.308997  9952 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.309438  9952 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:56.311319  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:56.272490  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.273813  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3252) took 1699 ms. Trace:
I20260709 12:26:56.202608  9947 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.314857  9947 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.315902  4645 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:56.317029  9947 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:56.284392  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3255) took 1709 ms. Trace:
I20260709 12:26:56.317720  4751 rpcz_store.cc:276] 0709 12:26:54.574888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.231707 (+1656819us) service_pool.cc:224] Handling call
0709 12:26:56.283463 (+ 51756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.284146 (+   683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.284338 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.318549  9951 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.223541  9961 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:56.296053  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:56.300770  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.286064 10112 raft_consensus.cc:697] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:56.290865  9946 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:56.307301  5312 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:26:56.347112  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 56750.9us
I20260709 12:26:56.239305 10120 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 12:26:56.319550  4602 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:26:56.342846  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 164541us
I20260709 12:26:56.351400  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.328023  9947 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:56.376363  9947 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.376965  9947 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:56.343410 10098 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:56.320768  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:56.340741  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 69179.4us
I20260709 12:26:56.350380  4753 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:56.314687  4755 rpcz_store.cc:276] 0709 12:26:54.573986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.202279 (+1628293us) service_pool.cc:224] Handling call
0709 12:26:56.210806 (+  8527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.239228 (+ 28422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.273722 (+ 34494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.345234  9952 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.368279  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:56.365952  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:56.370764  4750 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:56.304579  9735 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:56.398418  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2687) took 1822 ms. Trace:
I20260709 12:26:56.398921  4750 rpcz_store.cc:276] 0709 12:26:54.575761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.307103 (+1731342us) service_pool.cc:224] Handling call
0709 12:26:56.370541 (+ 63438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.398140 (+ 27599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.398344 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:56.384353  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 223001us
I20260709 12:26:56.367111  4747 raft_consensus.cc:2410] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:26:56.388767  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:56.409039  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 139266us
I20260709 12:26:56.412307  4755 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:26:56.412969  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 47122.2us
I20260709 12:26:56.413139  4605 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:56.389425  4752 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:56.390413  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:56.376255 10128 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:56.427627 10128 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:26:56.428136  4496 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52812 (request call id 292) took 25412 ms (client timeout 15000 ms). Trace:
W20260709 12:26:56.430562 10119 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:26:56.430662 10097 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:56.432804  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:56.433494  4598 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:56.434248  4750 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:56.434895  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3259) took 1481 ms. Trace:
I20260709 12:26:56.435209  4750 rpcz_store.cc:276] 0709 12:26:54.953597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.411458 (+1457861us) service_pool.cc:224] Handling call
0709 12:26:56.433917 (+ 22459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.434613 (+   696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.434817 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.436326  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:56.438143  9882 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.439483  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:56.439918  9603 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:56.440093  4673 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:56.440291  9882 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:56.441100  4676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:48198
I20260709 12:26:56.441807  4676 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:56.408337  9588 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:26:56.302312  9990 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:56.442255  4676 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:56.444089  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:56.386452  9765 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:56.444360  9892 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:56.444695  4681 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:26:56.444880  9892 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.445650  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:56.447001  9892 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:56.428099 10128 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:56.423095  9961 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:56.452163  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.452435  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.452991  4596 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:26:56.453896  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.453728 10089 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.455091  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:56.455693  4597 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:26:56.423671  9735 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.426587  9951 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:56.426949  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:56.457612  4748 raft_consensus.cc:2393] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 18: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:56.459429  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3256) took 1685 ms. Trace:
I20260709 12:26:56.461022  4755 rpcz_store.cc:276] 0709 12:26:54.728163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.388061 (+1659898us) service_pool.cc:224] Handling call
0709 12:26:56.388818 (+   757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.412796 (+ 23978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.412997 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:56.463845  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.478477  9871 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:56.428681  4496 rpcz_store.cc:269] 0709 12:26:31.035395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.047248 (+ 11853us) service_pool.cc:224] Handling call
0709 12:26:56.427894 (+25380646us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:50.444058 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:50.456887 (+ 12829us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:50.456957 (+    70us) write_op.cc:435] Acquired schema lock
0709 12:26:50.456985 (+    28us) tablet.cc:662] Decoding operations
0709 12:26:50.459374 (+  2389us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:50.459683 (+   309us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:50.459727 (+    44us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:26:50.469878 (+ 10151us) tablet.cc:701] Row locks acquired
0709 12:26:50.470049 (+   171us) write_op.cc:260] PREPARE: finished
0709 12:26:50.470932 (+   883us) write_op.cc:270] Start()
0709 12:26:50.471509 (+   577us) write_op.cc:276] Timestamp: P: 1783600010470565 usec, L: 0
0709 12:26:50.471635 (+   126us) op_driver.cc:348] REPLICATION: starting
0709 12:26:50.501468 (+ 29833us) log.cc:844] Serialized 7454 byte log entry
0709 12:26:50.820045 (+318577us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effeb3ac0ac 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b
0709 12:26:51.399911 (+579866us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b1e28 00007efff1eec581 00007efff1eebd58 00007efff1ef09f1 00007efff1ef09a9 00007efff1ef0939 00007efff1ef0901 00007efff1eefb2d 0000000000388634 0000000000388479
0709 12:26:51.782408 (+382497us) op_driver.cc:464] REPLICATION: finished
0709 12:26:52.021796 (+239388us) write_op.cc:301] APPLY: starting
0709 12:26:52.072431 (+ 50635us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:52.520244 (+447813us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:52.520273 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:53.403120 (+882847us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:53.403547 (+   427us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 12:26:53.403617 (+    70us) write_op.cc:312] APPLY: finished
0709 12:26:53.404167 (+   550us) log.cc:844] Serialized 172 byte log entry
0709 12:26:54.396091 (+991924us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:54.396806 (+   715us) write_op.cc:454] Released schema lock
0709 12:26:54.396954 (+   148us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:54.397560 (+   606us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:54.397604 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2350777,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":178687,"apply.run_cpu_time_us":64430,"apply.run_wall_time_us":2386962,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83433,"num_ops":19,"prepare.queue_time_us":586371,"prepare.run_cpu_time_us":58849,"prepare.run_wall_time_us":982695,"raft.queue_time_us":317489,"raft.run_cpu_time_us":23845,"raft.run_wall_time_us":190388,"replication_time_us":1310488,"spinlock_wait_cycles":106227072,"thread_start_us":2271618,"threads_started":5,"wal-append.queue_time_us":1274960,"wal-append.run_cpu_time_us":38684,"wal-append.run_wall_time_us":2422701}]]}
I20260709 12:26:56.480571  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:56.483430  4598 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:56.475152  4676 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.484774  4645 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:56.515646  4752 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:26:56.521023  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 95652.9us
W20260709 12:26:56.513931  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47613.8us
W20260709 12:26:56.546773  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 62062.6us
I20260709 12:26:56.520573  4719 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:56.552776  5347 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
W20260709 12:26:56.555375  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 94146.8us
I20260709 12:26:56.556787  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:56.559291  4676 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:56.557647  5314 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:56.567323  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:56.567631  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 51031.4us
W20260709 12:26:56.568005  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 142235us
I20260709 12:26:56.569715  5312 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:56.554561  4568 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:56.574225  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2688) took 1998 ms. Trace:
I20260709 12:26:56.574285  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3251) took 2122 ms. Trace:
W20260709 12:26:56.577414  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 56904.4us
W20260709 12:26:56.559111  9977 consensus_meta.cc:287] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.559s	user 0.011s	sys 0.001s
I20260709 12:26:56.598989  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:56.599633  4598 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:56.608366  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 51657.2us
I20260709 12:26:56.558961 10104 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:56.615656  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:56.615914  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2689) took 2039 ms. Trace:
I20260709 12:26:56.549749  9850 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:56.617403  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:56.616266  4751 rpcz_store.cc:276] 0709 12:26:54.576301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.318520 (+1742219us) service_pool.cc:224] Handling call
0709 12:26:56.318844 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.319075 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:56.468685 (+149610us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.615576 (+146891us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.615634 (+    58us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:56.615680 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.615746 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.615860 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":218051,"threads_started":2}
I20260709 12:26:56.578900  4756 rpcz_store.cc:276] 0709 12:26:54.452017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.174295 (+1722278us) service_pool.cc:224] Handling call
0709 12:26:56.178867 (+  4572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.208484 (+ 29617us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:56.269679 (+ 61195us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.566995 (+297316us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.574008 (+  7013us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.574055 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:56.574099 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.574151 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.574237 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6728,"prepare.queue_time_us":1232,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":28250,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":21332,"thread_start_us":274948,"threads_started":2}
W20260709 12:26:56.618171  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.618949  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:56.619520  4750 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:56.621603  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3258) took 1673 ms. Trace:
I20260709 12:26:56.622110  4753 rpcz_store.cc:276] 0709 12:26:54.947872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.395194 (+1447322us) service_pool.cc:224] Handling call
0709 12:26:56.408192 (+ 12998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.413343 (+  5151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.413415 (+    72us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.413486 (+    71us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:56.413543 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.413663 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.621381 (+207718us) spinlock_profiling.cc:243] Waited 6.51 ms on lock 0x7b54000e56d0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:56.621510 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14976768}
W20260709 12:26:56.623795  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:56.624531  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 97253.6us
I20260709 12:26:56.624871  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:56.483026  9963 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.616660  4500 catalog_manager.cc:5697] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:56.627219  4605 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:56.576794  9936 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:56.627626  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:56.576856  9984 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:56.577442  9983 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.574862  9772 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:26:56.634806  5342 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:56.636934  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:56.633800  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 206927us
I20260709 12:26:56.639933  4752 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:56.612398  9882 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:56.652922  5349 raft_consensus.cc:2393] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 18: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:56.650864  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:56.628456  9661 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:56.633811 10126 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:56.634099 10129 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:56.683759  9882 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:56.624604 10009 consensus_queue.cc:1059] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:26:56.682422  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.685189  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.693274  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:56.685385  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:56.694152  4752 raft_consensus.cc:2410] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:26:56.630287 10099 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:26:56.695181  5340 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:26:56.658401 10104 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.634271  9850 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.691310  4598 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:56.670235 10126 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.682313  9776 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:56.700538  9936 raft_consensus.cc:697] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:56.670226 10098 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:56.650372  9983 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.721787  5343 raft_consensus.cc:3055] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:56.722257  5343 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:56.723160  5343 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.696063 10129 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:56.724429 10145 raft_consensus.cc:3055] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:56.724629 10132 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.724869 10129 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.725171 10132 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.712625  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.715893  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:56.662520  9871 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:26:56.634829 10112 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.719537  9963 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:56.720386 10124 log.cc:927] Time spent T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.338s	user 0.004s	sys 0.000s
I20260709 12:26:56.720690 10122 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:56.721025  5370 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5820 after lost signal to thread 3434
I20260709 12:26:56.788870  5347 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:56.789855  5370 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5180 after lost signal to thread 4536
W20260709 12:26:56.791580  5370 debug-util.cc:398] Leaking SignalData structure 0x7b08003a84a0 after lost signal to thread 4630
W20260709 12:26:56.791950  5370 debug-util.cc:398] Leaking SignalData structure 0x7b08003a8380 after lost signal to thread 4631
W20260709 12:26:56.792984  5370 debug-util.cc:398] Leaking SignalData structure 0x7b0800388920 after lost signal to thread 4706
W20260709 12:26:56.721901 10131 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:26:56.726032  9882 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:56.691164  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 61044.9us
I20260709 12:26:56.727147  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:56.711551  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:56.729635  9653 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:56.733327  5343 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:56.708722 10104 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:56.738011 10127 log.cc:927] Time spent T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:26:56.702540 10016 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:56.745904 10146 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:56.747480 10144 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:26:56.824998  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 113557us
W20260709 12:26:56.759482  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.761106  4750 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:56.771433  9850 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:56.722002  9776 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:56.772310  4752 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
W20260709 12:26:56.797103  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 160237us
I20260709 12:26:56.832281  4647 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:56.799614  5370 debug-util.cc:398] Leaking SignalData structure 0x7b0800383560 after lost signal to thread 4780
I20260709 12:26:56.712280 10126 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:56.812721  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 127460us
I20260709 12:26:56.753501  9984 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:56.729682  9952 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:56.822460  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 244219us
I20260709 12:26:56.724889 10145 raft_consensus.cc:740] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:56.826809  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:56.828536  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:56.837226  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 110145us
I20260709 12:26:56.837762  4647 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:56.838462  5370 debug-util.cc:398] Leaking SignalData structure 0x7b08002c15e0 after lost signal to thread 4781
I20260709 12:26:56.839828  5343 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:56.843251  4751 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:56.844597  5346 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:56.865103  5347 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
W20260709 12:26:56.867110  5370 debug-util.cc:398] Leaking SignalData structure 0x7b08002ddec0 after lost signal to thread 5373
W20260709 12:26:56.867609  5370 debug-util.cc:398] Leaking SignalData structure 0x7b080037d880 after lost signal to thread 5374
I20260709 12:26:56.873067  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:56.840348  9765 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:26:56.885826  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 59084.6us
W20260709 12:26:56.886850  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 49166.4us
I20260709 12:26:56.886742  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:56.887873  5314 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:56.845907 10137 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:56.894726  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:56.952199  4676 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:56.850512  9984 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:56.829356 10016 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:56.894174  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 266498us
W20260709 12:26:56.950546  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 62783.2us
I20260709 12:26:56.896433  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:56.829558 10128 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.896701  4647 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:56.965978  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3232) took 3107 ms. Trace:
I20260709 12:26:56.966434  4749 rpcz_store.cc:276] 0709 12:26:53.858369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.979669 (+2121300us) service_pool.cc:224] Handling call
0709 12:26:55.980043 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:56.118619 (+138576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.118667 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:56.135358 (+ 16691us) log.cc:844] Serialized 26 byte log entry
0709 12:26:56.201888 (+ 66530us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:56.201945 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.202046 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:56.719710 (+517664us) spinlock_profiling.cc:243] Waited 3.92 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:56.932225 (+212515us) raft_consensus.cc:1685] finished
0709 12:26:56.932250 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.965908 (+ 33658us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35681,"raft.queue_time_us":58090,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":13474,"spinlock_wait_cycles":9005952,"thread_start_us":21518,"threads_started":1,"wal-append.queue_time_us":125259}
W20260709 12:26:56.961747  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 65432.3us
I20260709 12:26:56.853085 10149 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:56.977881 10143 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:56.978546 10143 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:56.979368 10009 consensus_queue.cc:1059] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:56.978578  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3287) took 1186 ms. Trace:
I20260709 12:26:56.981178  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:56.992830 10138 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 12:26:56.995378 10138 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:56.996447 10151 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:56.997102 10151 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:56.997704 10153 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:56.998176 10153 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:56.998557 10153 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:56.999547 10151 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.005406  4676 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:26:56.979658 10143 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.009361  5342 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:56.954941 10137 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.972846  9871 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:56.980557  4672 rpcz_store.cc:276] 0709 12:26:55.792593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.895847 (+103254us) service_pool.cc:224] Handling call
0709 12:26:55.896168 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.121777 (+225609us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:56.150466 (+ 28689us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.290117 (+139651us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.290169 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:56.290218 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.978326 (+688108us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54001bd354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:56.978371 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.978518 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28673,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":4056,"raft.queue_time_us":84,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3464,"spinlock_wait_cycles":459165568,"thread_start_us":110056,"threads_started":1,"wal-append.queue_time_us":233372}
W20260709 12:26:57.021183  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 124545us
W20260709 12:26:57.019618  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 43843.6us
W20260709 12:26:57.023677  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 130295us
I20260709 12:26:57.025142  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:57.032003  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 100417us
I20260709 12:26:57.037418  9961 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:26:57.037933  9990 consensus_meta.cc:287] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.595s	user 0.010s	sys 0.000s
I20260709 12:26:57.032682 10099 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:26:56.896732  9776 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.126s
I20260709 12:26:57.012848 10148 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:57.038528  9990 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.065181  4720 leader_election.cc:400] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:26:57.020735  9871 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:26:57.033317  9740 consensus_meta.cc:287] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.808s	user 0.011s	sys 0.000s
I20260709 12:26:57.020013 10100 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:57.021726 10137 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:57.060801  4647 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:57.014273  9850 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:56.858349  9947 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.037294 10104 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:57.075747  4647 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:57.076160  9740 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.079084 10008 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:57.074095 10100 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:57.080446 10159 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:26:57.083137  9990 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.089736  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:57.061630 10152 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:57.140038  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.092661  4644 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:57.094202  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:57.094799  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:57.140878  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 48363.2us
I20260709 12:26:57.095299 10158 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.928793  9936 consensus_queue.cc:237] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:57.141774  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.041679 10054 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:57.117842  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 52574.1us
W20260709 12:26:57.103986  9946 consensus_meta.cc:287] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.756s	user 0.009s	sys 0.003s
I20260709 12:26:57.074733  9871 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:57.039999 10009 consensus_queue.cc:1059] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 12:26:57.126560  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:57.136077  9603 consensus_meta.cc:287] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.696s	user 0.011s	sys 0.000s
I20260709 12:26:57.124532 10008 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.126171 10100 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:57.139845  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.092073 10016 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.140972  4600 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:57.122834  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47186.7us
W20260709 12:26:57.145140  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 120519us
I20260709 12:26:57.124421  9892 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:57.161530  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:57.150100 10089 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.150225  9990 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:57.145587 10152 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:57.145248 10054 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.201524  4647 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:57.207232  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:57.154943  9588 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.116410 10137 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:57.208245  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 187007us
I20260709 12:26:57.208474  4599 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:26:57.159587 10008 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:57.159775  9603 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:57.211505  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:57.172511  9892 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:57.215504  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 64069.9us
I20260709 12:26:57.154125  9946 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.220135  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:57.205919 10016 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:57.210115 10152 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:57.227461  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:57.218547 10021 consensus_meta.cc:287] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.998s	user 0.011s	sys 0.002s
I20260709 12:26:57.152827 10158 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.215210  9776 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:57.213040  9990 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:57.254438  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:57.242726  4603 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 12:26:57.255023  4603 pending_rounds.cc:85] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 8
I20260709 12:26:57.255295  4603 pending_rounds.cc:107] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:26:57.223786  9892 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:57.255715  4603 raft_consensus.cc:2889] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:57.234030  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 347422us
I20260709 12:26:57.212503 10163 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.248648  4751 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:26:57.259259  4751 pending_rounds.cc:85] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 7
I20260709 12:26:57.259541  4751 pending_rounds.cc:107] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:26:57.259783  4751 raft_consensus.cc:2889] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:57.212809 10054 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:57.222863 10144 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:57.263988  4599 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:57.264228  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:57.264524  4749 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:26:57.206377 10126 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.245884  9740 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.270538  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:57.270644  9740 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:57.271135 10126 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.271299  9740 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.271852  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:57.273447 10161 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:57.206037 10089 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.245024 10154 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:57.275869  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:57.220956  9588 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.251827  9776 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:57.251308 10021 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:56.574857  4754 rpcz_store.cc:276] 0709 12:26:54.576038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.312378 (+1736340us) service_pool.cc:224] Handling call
0709 12:26:56.312691 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.312953 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:56.313300 (+   347us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.573868 (+260568us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.573920 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:56.573964 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.574055 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.574168 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23602,"prepare.queue_time_us":4599,"raft.queue_time_us":23737,"spinlock_wait_cycles":5200128,"thread_start_us":230107,"threads_started":1}
I20260709 12:26:57.286230  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:57.261222 10164 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:57.269189 10137 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:57.253487  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 91636.5us
I20260709 12:26:57.280099  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:57.253746  4748 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 16. (term mismatch)
I20260709 12:26:57.256263 10102 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:57.284950  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 70442.9us
I20260709 12:26:57.287796  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:57.306689  4749 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:57.274228 10161 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:57.291661  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:57.292034  4748 pending_rounds.cc:85] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 15
I20260709 12:26:57.292048  4569 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:57.292357  4750 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:57.293013  9990 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.295733  9735 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:57.277086 10016 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:57.299578  9588 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.299327 10145 consensus_queue.cc:260] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.301364  9661 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:26:57.303933 10141 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.555s	user 0.008s	sys 0.001s
I20260709 12:26:57.374244  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:57.306805  4754 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:57.278262 10154 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:57.309484  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:57.311247  5312 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:26:57.378861  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 67720.8us
I20260709 12:26:57.323706  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:57.315402  4748 pending_rounds.cc:107] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 18.16
I20260709 12:26:57.327518  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:57.381906  4748 raft_consensus.cc:2889] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:57.382237  4605 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 17: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:57.321938  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:57.274585 10169 consensus_queue.cc:1059] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:26:57.322618 10098 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:57.328594  4602 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:57.331733 10158 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:57.328622  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:57.386466  5343 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:57.324308 10089 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.330317  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:57.326319  9946 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:57.329221  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:57.389508  4596 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:57.390098  4673 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:57.328886  9947 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.322929 10021 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:57.286446  9776 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:57.393370  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:57.417389  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:57.418280  4602 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:57.318892  4600 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:57.262465  9882 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:26:57.356751  4749 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:57.374886  5347 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:57.373122  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 58736.9us
I20260709 12:26:57.316246 10161 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:26:57.376834  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 84940.7us
I20260709 12:26:57.265352 10163 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.335632 10122 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.454355  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:57.387512  4750 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:57.368063  9735 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:57.402699  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:57.456619  4754 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:57.405838  4720 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:57.367363  9740 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.406224  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:57.374040  9661 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:57.383044  4603 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:57.402518 10169 raft_consensus.cc:3055] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:57.448486  5343 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:57.367413  9984 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.465407  5347 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:57.466369  4596 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:26:57.369176 10153 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.405746 10170 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:57.401909 10145 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:57.467272  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:57.389382  9603 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:57.472929  4748 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 17: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:57.476737  4644 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:57.374433 10016 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.478979  4750 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:26:57.383258  9871 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.401484  9892 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.482462  4597 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
W20260709 12:26:57.487185  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.471381 10122 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:57.477656 10102 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:57.454304 10161 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:57.454871  9947 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:57.490579  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:57.449137  9882 consensus_queue.cc:1243] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:57.400588 10100 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.495311  4644 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:57.496675  4749 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:57.475777 10169 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:57.466897 10112 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:26:57.502315  4601 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
W20260709 12:26:57.498646  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 123508us
I20260709 12:26:57.507050  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:57.507716  4748 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:57.498934  9990 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:57.521337  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:57.521922  4748 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:57.523201  4605 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:57.526747  4749 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:57.484548  9653 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:26:57.497694  9603 raft_consensus.cc:697] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:57.501904  9947 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.530851  9892 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.427897 10154 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.530447 10169 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.520987  9735 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.534790 10008 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.505141 10144 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:57.540925  4603 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:57.506784 10102 raft_consensus.cc:697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:26:57.555696  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 100473us
I20260709 12:26:57.552522  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:57.582537  4749 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 15: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:57.547827  4748 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:26:57.527916 10104 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:57.592805 10140 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.851s	user 0.000s	sys 0.008s
W20260709 12:26:57.594959 10181 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:57.597585  5348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 766) took 1660 ms. Trace:
I20260709 12:26:57.598703 10054 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.597997  5348 rpcz_store.cc:276] 0709 12:26:55.936928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.051583 (+114655us) service_pool.cc:224] Handling call
0709 12:26:56.073250 (+ 21667us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:56.307116 (+233866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.307162 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:56.376000 (+ 68838us) log.cc:844] Serialized 26 byte log entry
0709 12:26:56.623388 (+247388us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:56.623428 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.623498 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:57.592163 (+968665us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:26:57.595920 (+  3757us) raft_consensus.cc:1685] finished
0709 12:26:57.595947 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:57.597375 (+  1428us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b54008d7cd0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:57.597499 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55644,"raft.queue_time_us":217873,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":3179,"spinlock_wait_cycles":123670016,"thread_start_us":348373,"threads_started":2,"wal-append.queue_time_us":296223}
I20260709 12:26:57.601069 10054 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:57.602341 10183 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 12:26:57.602532 10142 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.850s	user 0.008s	sys 0.000s
I20260709 12:26:57.604074  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:57.583768 10180 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.604903  4750 raft_consensus.cc:2393] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 18: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:57.605165 10180 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:57.606410  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 64043.9us
I20260709 12:26:57.607200  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:57.607350 10180 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:57.583511 10167 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.154s
W20260709 12:26:57.584717 10179 log.cc:927] Time spent T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:26:57.615480  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:57.594547  9871 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.541441  9990 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:57.610831  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:57.622702  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:57.620021  4602 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:57.587656  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:57.629029  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:57.634195  4751 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:26:57.635241  4754 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:57.610075 10148 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.588671 10144 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:57.636736  9735 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:57.620682 10154 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:57.663545  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:57.664414  4601 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:57.636337  9871 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.637509 10169 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:57.674891  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:57.675668  4754 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 17: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:57.655084  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:57.669533  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 54142.6us
I20260709 12:26:57.619666 10104 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:57.672104 10143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.002s	sys 0.048s
I20260709 12:26:57.686677  4569 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:26:57.670889  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63784.5us
I20260709 12:26:57.647612 10180 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:57.615571 10152 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.692631  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:57.663020  9892 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:57.696240  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:57.698848  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:26:57.690349 10143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.002s	sys 0.048s
I20260709 12:26:57.712241  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:57.712900  4601 raft_consensus.cc:2468] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:26:57.688208 10163 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.719645  4679 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:57.720739  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:57.721873  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:57.724040 10185 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:26:57.696066 10180 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:57.724865  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:57.725263 10180 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.726845  4673 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:26:57.729442  4674 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:57.643582  9603 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.731572  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:57.732300  4678 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:57.684093  9871 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.733903  4680 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:57.734946  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:57.700546 10100 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:57.720937 10190 raft_consensus.cc:2764] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:26:57.753859 10199 raft_consensus.cc:2764] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
W20260709 12:26:57.754511  5349 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 730) took 1621 ms (client timeout 2082 ms). Trace:
I20260709 12:26:57.718278  9984 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:57.754901  5349 rpcz_store.cc:269] 0709 12:26:56.132926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.138269 (+  5343us) service_pool.cc:224] Handling call
0709 12:26:57.754299 (+1616030us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b54005e2754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:57.754454 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093995264}
I20260709 12:26:57.739014 10104 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:57.755993  9947 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.714938 10112 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:57.757422 10008 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:57.720970 10144 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.641784 10148 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.747965 10198 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:57.750021 10097 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.750824  9661 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.674737 10102 consensus_queue.cc:237] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.701638  9892 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.752884 10128 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.748010 10152 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.714392 10192 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.774259  4680 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:26:57.775957  4678 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
I20260709 12:26:57.798449  4678 pending_rounds.cc:85] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 8
I20260709 12:26:57.798826  4678 pending_rounds.cc:107] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 12:26:57.799178  4678 raft_consensus.cc:2889] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:57.776856  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:57.744881 10200 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.780251  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:57.784111  4599 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:26:57.786058  4749 pending_rounds.cc:85] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 8
I20260709 12:26:57.803534  4752 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:57.804193  4681 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:57.807322  4749 pending_rounds.cc:107] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:26:57.807643  4749 raft_consensus.cc:2889] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:57.805905  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:57.814821  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:57.799997 10198 raft_consensus.cc:697] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:57.783363 10161 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:57.822369  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.729650  9990 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.781492 10097 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.854579  4748 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:57.855043  4752 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:57.805963 10192 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:57.808161 10200 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.865089 10148 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.866784 10161 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:57.874770  9661 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:57.878443  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:57.842801 10204 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.898226  4752 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:57.901283  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:57.906019  4756 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:26:57.906951  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:57.907500  4756 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:26:57.908239  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:57.909139  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:26:57.910496 10197 log.cc:927] Time spent T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:26:57.910862 10132 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:57.912762 10134 log.cc:927] Time spent T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 1.167s	user 0.003s	sys 0.006s
I20260709 12:26:57.914017  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:57.914682  4601 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:57.918843 10016 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:57.921154  4756 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:26:57.857411  9892 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:57.941004  4752 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:57.942145  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:57.939441 10104 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:57.950937 10200 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:57.953859  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:57.954439  5340 raft_consensus.cc:1813] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Handling vote request from an unknown peer 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:57.954888  5340 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 13: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:57.956993  4569 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:57.964213  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:57.972783  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:57.974121  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:57.975128  4677 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:26:57.978387  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:57.951571 10104 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:57.965077  9735 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:57.976025 10098 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:57.968634 10154 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:26:57.976740 10180 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:26:58.010329 10192 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:57.986310 10191 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:57.971300 10145 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:26:58.012393 10104 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:58.010025  5342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 737) took 1166 ms. Trace:
I20260709 12:26:58.022217  5342 rpcz_store.cc:276] 0709 12:26:56.843793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.844953 (+  1160us) service_pool.cc:224] Handling call
0709 12:26:56.857047 (+ 12094us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:57.014700 (+157653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.009734 (+995034us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b540084ce54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:58.009964 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":349783936}
I20260709 12:26:57.971707 10198 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:58.026414  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:57.989970  9882 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:26:58.019271 10191 raft_consensus.cc:697] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:57.999063  9871 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:58.020617 10154 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:57.983960 10122 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:58.030740  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:57.989720  9961 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:26:58.033286  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:58.035917  5343 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:57.948309 10204 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:58.103312  5347 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 16. Ops: []
I20260709 12:26:58.032970 10154 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:26:58.113471  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 126978us
I20260709 12:26:58.032191  9990 consensus_queue.cc:1059] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
W20260709 12:26:58.110149 10149 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.104s	user 0.015s	sys 0.000s
W20260709 12:26:58.128844 10149 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.016s	sys 0.000s
W20260709 12:26:58.135099  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 196539us
I20260709 12:26:58.135986  4673 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 12:26:58.139712  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 138277us
W20260709 12:26:58.135182  9776 consensus_meta.cc:287] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.685s	user 0.010s	sys 0.003s
I20260709 12:26:58.135545 10149 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:58.152280  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:58.154646  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:58.157621  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:58.146104 10191 consensus_queue.cc:237] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:58.163059 10139 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.414s	user 0.003s	sys 0.009s
I20260709 12:26:58.160396 10221 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:58.163550 10149 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:58.164901  4571 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:58.156185 10021 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:58.145956  9776 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.188079  9776 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:58.189035 10169 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:58.189837 10210 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 12:26:58.190291 10216 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:26:58.196873  4681 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:58.197420  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:58.202420  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:58.203119 10225 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:58.203351  4754 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:58.174124  9936 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:58.184525  9983 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:58.185779 10211 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:26:58.204676  4754 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:26:58.207204  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:58.208390  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.220418  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.219455  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.183375  4675 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:26:58.203351  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:58.229090  4596 raft_consensus.cc:2393] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:58.229350  4603 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:26:58.206807  4680 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:58.233664  4571 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:58.165174 10171 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:26:58.236090  4602 raft_consensus.cc:2368] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14.
I20260709 12:26:58.239398  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.239883  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:58.239423  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.243379  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:58.244760  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:58.244845  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.235917 10225 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:58.267890  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:58.268983  4597 raft_consensus.cc:2368] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
I20260709 12:26:58.269871  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:58.272176  4754 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:58.275576  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:58.266531 10217 log.cc:927] Time spent T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:26:58.277267  9990 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:26:58.266884  9936 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:26:58.232810 10221 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:26:58.254462 10122 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:58.288717 10220 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:26:58.289940 10144 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 12:26:58.290536 10204 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:58.278249  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.279093  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.281734  4600 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:58.282518  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:26:58.282979 10223 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:26:58.292632  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:26:58.279166  4647 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:58.298849  4647 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:58.268764 10112 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:58.301445  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.252635  9776 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:26:58.304297  9936 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:58.304651 10104 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.319550 10104 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.319846  9947 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:58.306636 10229 consensus_queue.cc:1059] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:26:58.322065 10229 raft_consensus.cc:3055] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:58.322438 10229 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:58.323339 10229 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:58.325014 10229 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:58.319818  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:58.272866  9882 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:58.327029  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:58.303730 10221 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:58.306149  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:58.329792 10104 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:58.328331  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:58.329381  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.326012  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:58.317577  4603 raft_consensus.cc:2393] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:58.318836  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:58.337837  4602 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 13: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:58.339160  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:26:58.340765  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.347267  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.348660  4596 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:58.350723  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:58.327118  9882 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.355129  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.356640  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:58.373064  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.336916  9936 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:58.337617 10222 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:58.344720 10221 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:58.405001  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.583424  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.594139  4597 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:26:58.594861  4678 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:26:58.405151  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.589190  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.386482  4601 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:26:58.582909  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.584110  4604 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:58.394845  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:58.451960 10162 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 1.365s	user 0.007s	sys 0.003s
W20260709 12:26:58.599248  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 204468us
I20260709 12:26:58.600778  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 894) took 1199 ms. Trace:
I20260709 12:26:58.601280  4605 rpcz_store.cc:276] 0709 12:26:57.401841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.415670 (+ 13829us) service_pool.cc:224] Handling call
0709 12:26:57.468198 (+ 52528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:57.523594 (+ 55396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.600484 (+1076890us) spinlock_profiling.cc:243] Waited 54.8 ms on lock 0x7b54001400d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:58.600706 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126030720}
W20260709 12:26:58.452122 10214 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:26:58.604233  5342 raft_consensus.cc:1240] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:58.372550  9882 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:58.405653 10016 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:58.375334  9776 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:58.603536  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:58.612238  4647 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:58.401930 10230 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:58.495745 10104 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:58.605314 10149 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:58.618093  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.622298  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:58.622776  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:58.623976  4753 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:58.624161  4754 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:58.625778 10198 consensus_queue.cc:1059] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:58.626703 10198 raft_consensus.cc:3055] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:58.627028 10198 raft_consensus.cc:740] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:58.619732 10016 raft_consensus.cc:697] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:58.627831 10198 consensus_queue.cc:260] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:58.631165 10198 raft_consensus.cc:3060] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:58.619370 10104 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.632288  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:58.609305 10238 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:58.634022 10238 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.636742 10238 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:58.633471  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:58.611547  9776 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.596202 10222 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.638239  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:58.637636  9947 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:58.638868  4749 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
I20260709 12:26:58.631027 10016 consensus_queue.cc:237] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.639972  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.616336  9740 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:58.662021  4753 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:26:58.655172 10104 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:58.668300  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:58.663403  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:58.637687  9776 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:58.673195  4747 raft_consensus.cc:2393] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:26:58.669696  4753 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:58.673035  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.646413 10149 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:58.642310 10122 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:58.642630  9947 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:58.680119  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:58.658139 10132 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.683403  4753 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:58.655447  9740 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:58.661016  9772 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:26:58.667157 10215 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 12:26:58.684546  9947 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:58.686453  9936 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.688647 10222 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:58.693079  5342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:26:58.641788 10230 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:58.693972  5342 raft_consensus.cc:606] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:58.694455  5342 raft_consensus.cc:614] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:58.656257  9990 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 12:26:58.699698  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 52615.8us
I20260709 12:26:58.720594 10152 raft_consensus.cc:3055] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:58.721120 10152 raft_consensus.cc:740] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:58.724049  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.724943  4675 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 15: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:58.716859  9772 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:58.687484  9740 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:58.687686 10132 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:58.735934 10132 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:26:58.738183 10243 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:26:58.748551  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:58.749558  4747 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:26:58.763276 10230 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:58.764278  9772 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:58.773052  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.687934 10122 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:58.770092 10236 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:58.739933 10149 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.769871  9882 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.678866 10221 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.705638  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:58.800153  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:58.803332  4673 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 17: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:26:58.804754  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 85490.6us
I20260709 12:26:58.805833  5314 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:58.806807  9772 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:58.807303  9772 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:58.807742  9772 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:58.808007  9882 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:58.821372  5314 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:58.841676  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:26:58.843680  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.844795  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:58.846392  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:58.852213  4752 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:26:58.842027  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.845144  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.828997  9882 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:58.828275  9776 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.856987  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:58.863041  9882 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.864568  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.865157  9776 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:58.854561  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.856614 10132 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:58.866168 10132 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:58.866570  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.868209  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.869192  4678 raft_consensus.cc:2393] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 18: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:58.866600 10132 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:58.863111  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:58.874534  4754 raft_consensus.cc:2410] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:58.875862  9882 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:58.817528  9740 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.858471  4674 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 17: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:26:58.820394 10122 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:58.858208 10097 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:58.844792  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:58.879958  9740 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:58.857754  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.881598  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:58.884088  4673 raft_consensus.cc:2393] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 18: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:58.883505  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:58.818296  9772 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:58.890707  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.891217  9772 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:58.892239  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.893678  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.893288  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.894502  4602 raft_consensus.cc:2410] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:26:58.895880  4752 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:26:58.895900  4676 raft_consensus.cc:2368] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12.
I20260709 12:26:58.896870  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:26:58.912334  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 142765us
I20260709 12:26:58.922353  4597 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:58.897518  4752 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:26:58.929107  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.936076  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.937120  4674 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:58.937929  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.937981  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.938669  4674 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:58.939146  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.940764  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:58.942145  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.943356  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.944433  4604 raft_consensus.cc:1240] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:58.928171  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:58.945114  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.950466  4605 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:58.953610  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:58.884325 10198 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:58.955428  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:58.956306  4752 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 17: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:58.949956  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.958262  4598 raft_consensus.cc:2410] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 19 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:26:58.959503  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:58.959933  5312 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:58.961838  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 3284) took 1202 ms. Trace:
I20260709 12:26:58.962251  4750 rpcz_store.cc:276] 0709 12:26:57.759729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.814607 (+ 54878us) service_pool.cc:224] Handling call
0709 12:26:58.961650 (+1147043us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b5400117954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:58.961775 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1369944448}
I20260709 12:26:58.963413  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:58.964015  4750 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:26:58.975296  5314 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:58.976193  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:58.978791  4750 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 15: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:58.977873  5312 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:58.847023  9947 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:58.989118  4605 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:26:58.992640  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:58.992976  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3311) took 1260 ms. Trace:
I20260709 12:26:58.993319  4600 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:58.995151  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.998324  4605 raft_consensus.cc:3055] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:58.998941  4605 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:58.991032  4568 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:58.998072  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:58.993506  4679 rpcz_store.cc:276] 0709 12:26:57.733031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.733625 (+   594us) service_pool.cc:224] Handling call
0709 12:26:57.767796 (+ 34171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:57.768244 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:57.783617 (+ 15373us) log.cc:844] Serialized 14 byte log entry
0709 12:26:58.723340 (+939723us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b1800310d80. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:26:58.990721 (+267381us) log.cc:844] Serialized 14 byte log entry
0709 12:26:58.992593 (+  1872us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:58.992658 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:58.992716 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.992782 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:58.992914 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1564,"prepare.queue_time_us":16354,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":3870,"spinlock_wait_cycles":6473600,"thread_start_us":266816,"threads_started":1}
I20260709 12:26:59.004076  4675 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:58.969187 10132 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.000054  4605 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:58.972105 10248 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:59.005786  4568 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:59.008260  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:59.009920  4597 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:26:59.010704 10259 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:59.008504  4605 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:59.011394  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:58.989789 10122 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:59.018450  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:59.020004  4605 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:26:59.020134  4600 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:59.022719  4605 pending_rounds.cc:85] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 11
I20260709 12:26:59.023033  4605 pending_rounds.cc:107] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:26:59.023342  4605 raft_consensus.cc:2889] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:58.973639 10198 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:26:59.030275  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:59.031384  5314 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.724s (SENT) [suppressed 5 similar messages]
I20260709 12:26:59.009442 10009 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:59.034423  4645 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.034514  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:59.036345  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:26:59.036859  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:59.037464  5349 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 17: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:26:59.040040  4645 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:26:59.027619 10246 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.270s	user 0.005s	sys 0.000s
I20260709 12:26:59.021162 10122 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.043111 10132 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:59.044984  4645 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:59.031458 10190 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:59.049851  4645 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.050088  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:59.055991  4720 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:59.039372 10230 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.040746 10008 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:59.046471 10182 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:59.048807 10122 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:59.077997  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.043370 10144 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:26:59.079602  4645 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.047998 10097 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:59.080228  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 62031.7us
I20260709 12:26:59.052428 10190 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.078505  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:59.051739 10166 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:26:59.074812 10257 log.cc:927] Time spent T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:26:59.075197 10259 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:59.073267 10222 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:59.087842  5312 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:59.084177  9776 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:59.105786  5312 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:26:59.092549  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:59.079679 10200 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:59.108958  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:59.085316 10143 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:59.112021  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.077696 10230 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.091658 10190 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:59.113165  9984 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:59.113270 10261 raft_consensus.cc:2804] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:59.114014 10132 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:26:59.114096 10104 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:59.125124 10230 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:59.090363 10008 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:26:59.128576  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:59.094143 10099 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:26:59.130738  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:59.131546  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:59.131803  4677 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:59.168614  4681 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:59.132756  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:59.114157 10248 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:59.095901 10238 consensus_queue.cc:1059] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:26:59.091310 10169 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:26:59.135291  4749 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:59.138221 10198 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:59.096001 10192 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:59.185547 10192 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:59.186074 10192 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:59.168068  9984 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:59.186743  9984 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.140242  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:59.188045  4754 raft_consensus.cc:2410] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:59.143908  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:59.143426  4647 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:26:59.189121  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 45763.2us
I20260709 12:26:59.190050  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:59.146381  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 64367.3us
I20260709 12:26:59.147264  9946 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:59.147464  9947 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:59.129326  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:59.170585  4750 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:59.198191  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 53558.3us
I20260709 12:26:59.201613 10132 raft_consensus.cc:3055] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:59.218427  4681 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:26:59.219864  9946 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.197324  9740 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:59.210268 10138 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:59.225975 10198 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:59.226799  4645 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.211218  4571 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:59.194620 10166 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.231599  4750 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
W20260709 12:26:59.232131  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 88315.2us
I20260709 12:26:59.239905  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:59.221004 10144 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:59.237039  4571 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:59.172536 10248 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.264977  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:59.275753  4754 raft_consensus.cc:1240] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:26:59.278865  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.227524 10132 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:59.234025 10262 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:59.283607  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.234360 10138 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.262132 10100 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:59.272399 10248 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:59.298888  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.301231  4645 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.302637  4645 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:59.272190 10104 raft_consensus.cc:2764] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:26:59.279721  9740 raft_consensus.cc:697] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:59.277892 10099 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:59.277415 10182 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:59.278679 10232 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:59.233108  9947 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:26:59.289829 10138 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:59.287117 10200 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:59.321774  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.305778 10264 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:59.273139 10166 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.235877 10259 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.326566  4681 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:26:59.287518 10122 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:59.336373  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 46490.7us
I20260709 12:26:59.314890 10099 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:59.319684 10008 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:59.317587 10100 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:59.291535 10262 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.317211 10016 consensus_queue.cc:1059] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:26:59.365727  5340 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:59.337008 10104 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:59.339569 10200 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:59.341396 10099 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:59.371300  5314 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:59.335220 10275 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:59.373922  9984 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:26:59.374459 10265 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:26:59.374657  9772 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.374387 10182 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:59.375263 10008 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:59.376091 10278 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:59.344183 10149 raft_consensus.cc:3055] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:59.379947 10166 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:59.381413  9946 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:59.335676 10232 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.372740 10261 raft_consensus.cc:697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:59.424624 10261 consensus_queue.cc:237] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:26:59.368248 10267 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 12:26:59.404337  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 82641us
I20260709 12:26:59.430193  4720 leader_election.cc:400] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
W20260709 12:26:59.418928  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48420.6us
I20260709 12:26:59.432011  4674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53908897ce974579947af90acaf0aed7"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:26:59.432637  4674 raft_consensus.cc:606] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:59.434335  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:59.394873 10264 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:26:59.438376  4604 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:59.413894  4677 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:26:59.368415 10279 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:59.396528  9740 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.415597 10149 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:59.396580 10144 raft_consensus.cc:3055] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:59.386130 10099 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.425872 10224 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.429999 10274 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:59.444703  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:59.422626 10122 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:59.343590 10132 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.465279  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:59.468372  4720 leader_election.cc:403] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:26:59.418172 10262 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:59.420671 10278 raft_consensus.cc:697] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:26:59.479916  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49872.4us
I20260709 12:26:59.499550  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.500411  4603 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:59.415419  9772 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.513499  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.464207 10144 raft_consensus.cc:740] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:59.516310  9984 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:59.518218 10153 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:59.530706  4720 leader_election.cc:400] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14.
I20260709 12:26:59.531301  4720 leader_election.cc:403] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:59.510411 10200 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.535060  4675 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abf423ac214b42acbe1696a11279567c"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
W20260709 12:26:59.536026  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.438735 10154 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.439774 10100 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.541929  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.500099 10279 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
I20260709 12:26:59.443735 10275 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 12:26:59.554631  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:59.471640 10224 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.556568 10099 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:59.561122  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:59.556386 10145 raft_consensus.cc:3055] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:59.501035 10182 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:26:59.564445  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:59.526366 10232 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:59.570648  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:59.573493  4750 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:59.576047 10281 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:59.576553 10178 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.577139 10178 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.579684 10178 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:59.472818 10274 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.581626  9772 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:59.582445 10274 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:59.565680 10278 consensus_queue.cc:237] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.592754  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:59.593065  4677 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:26:59.596443  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:59.597318  4678 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
I20260709 12:26:59.600664  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:59.527295 10190 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.575356 10154 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:59.548324 10200 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.573114 10145 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:26:59.574910 10144 consensus_queue.cc:260] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.603286 10281 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:59.613852  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:26:59.613071  9776 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:59.566071 10138 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.602957 10192 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.616254 10279 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:59.562925 10149 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.617344 10281 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:26:59.634361  4675 raft_consensus.cc:606] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:26:59.634761  4675 raft_consensus.cc:614] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:59.635109  4750 raft_consensus.cc:2468] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:26:59.636693  5314 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:59.637698  4568 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:59.642794  4750 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:59.629976  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:59.630928 10145 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.645527  9772 raft_consensus.cc:2804] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:59.646025 10145 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:59.646107  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:59.646333  9772 raft_consensus.cc:697] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:26:59.623199  9984 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:59.646787  4603 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:26:59.647564  9772 consensus_queue.cc:237] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.647871 10152 consensus_queue.cc:260] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.649185 10152 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:59.650471  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:26:59.651538 10008 consensus_queue.cc:1059] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:59.651727  9776 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:59.652246 10206 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:59.652410  9776 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:26:59.653048  9776 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:59.652410 10008 raft_consensus.cc:3055] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:59.627213 10144 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:59.662429  4569 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:59.669703  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:59.639901 10281 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:59.644021 10099 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:59.655565 10008 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:26:59.656718 10198 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:59.690852  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 70050.5us
I20260709 12:26:59.695865  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.697240  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2748) took 1042 ms. Trace:
I20260709 12:26:59.697690  4749 rpcz_store.cc:276] 0709 12:26:58.654366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.184070 (+529704us) service_pool.cc:224] Handling call
0709 12:26:59.184589 (+   519us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:59.186412 (+  1823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.186447 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:59.187048 (+   601us) log.cc:844] Serialized 52 byte log entry
0709 12:26:59.464582 (+277534us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:59.464622 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.464689 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:59.696991 (+232302us) raft_consensus.cc:1685] finished
0709 12:26:59.697017 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:59.697176 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":959,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":277144,"threads_started":1,"wal-append.queue_time_us":388127}
I20260709 12:26:59.682545 10180 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:59.702509  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.671471 10169 consensus_queue.cc:1059] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:26:59.711921  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:59.716579  4599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2733) took 2340 ms (client timeout 2323 ms). Trace:
I20260709 12:26:59.699858 10222 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:59.726656 10198 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:59.738526  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 73743.8us
I20260709 12:26:59.713653 10206 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:59.715256  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:26:59.709028 10100 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:26:59.748749  4749 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:59.788981  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 2086) took 1106 ms. Trace:
I20260709 12:26:59.870731  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1019) took 1003 ms. Trace:
I20260709 12:26:59.792490  4755 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:59.871234  4749 rpcz_store.cc:276] 0709 12:26:58.867576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.733499 (+865923us) service_pool.cc:224] Handling call
0709 12:26:59.746703 (+ 13204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:59.870460 (+123757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.870667 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:59.872797  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:59.873545  4749 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:59.870366  4676 rpcz_store.cc:276] 0709 12:26:58.682809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.683030 (+   221us) service_pool.cc:224] Handling call
0709 12:26:59.788707 (+1105677us) spinlock_profiling.cc:243] Waited 95.4 ms on lock 0x7b54001d29d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:59.788904 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219532160}
I20260709 12:26:59.786718  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2740) took 1464 ms. Trace:
W20260709 12:26:59.869397  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 154255us
W20260709 12:26:59.790843  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:59.747771  4599 rpcz_store.cc:269] 0709 12:26:57.373092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.384472 (+ 11380us) service_pool.cc:224] Handling call
0709 12:26:59.712901 (+2328429us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b54003543d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:59.713055 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":847088896}
W20260709 12:26:59.791261  4748 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 3270) took 2204 ms (client timeout 1990 ms). Trace:
I20260709 12:26:59.879545  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:26:59.879959  4748 rpcz_store.cc:269] 0709 12:26:57.586714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.587408 (+   694us) service_pool.cc:224] Handling call
0709 12:26:59.790955 (+2203547us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54000a7654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:26:59.791133 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503322368}
I20260709 12:26:59.880614  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:59.880779  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.881371  4680 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:26:59.881827  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:26:59.884291  4748 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:26:59.886029  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:59.886894  4748 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:59.888940  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.877722  4596 rpcz_store.cc:276] 0709 12:26:58.322757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.349712 (+ 26955us) service_pool.cc:224] Handling call
0709 12:26:58.402468 (+ 52756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:58.696088 (+293620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:58.696122 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:58.696168 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:58.696203 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.786351 (+1090148us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b540032c3d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:26:59.786413 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:59.786603 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674760064}
I20260709 12:26:59.737911 10138 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:59.898456  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:26:59.765471 10234 log.cc:927] Time spent T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 1.024s	user 0.001s	sys 0.005s
W20260709 12:26:59.790690 10293 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:26:59.745340 10276 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:59.910627  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:59.789772  9735 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.862210 10200 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:59.915652  4673 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 15: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:26:59.782353 10222 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:59.910938  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:26:59.744264 10190 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:26:59.747929 10224 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:26:59.930567 10294 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 12:26:59.931137  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:59.931813  4602 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:59.863127 10192 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:59.907236 10206 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:59.921721 10198 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:59.895664  9772 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:26:59.922324 10276 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.945076 10276 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:59.748571 10008 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.954571 10008 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:59.957626 10248 consensus_meta.cc:287] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.644s	user 0.008s	sys 0.004s
I20260709 12:26:59.958132 10248 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:26:59.958860  4749 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:26:59.883654  9984 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:26:59.960666  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:59.961282  4749 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:26:59.961532  9984 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.962369  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:59.964442  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:26:59.960660 10248 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:26:59.966276 10248 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:59.927923  9735 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.934509 10200 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:59.923530 10297 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:59.967490 10297 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.929952  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:26:59.969762 10222 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:26:59.970290 10297 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:59.970386  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:26:59.966774 10248 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:26:59.971076  4596 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:59.971349 10248 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:59.954413  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:26:59.974403  4647 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:26:59.976548  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 284169us
I20260709 12:26:59.977371  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:26:59.979929  4749 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:59.982098  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:26:59.982914  4749 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:59.984071  4748 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 20.
I20260709 12:26:59.988868  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:26:59.950591  4602 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:26:59.982520  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:26:59.999221  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:00.003120  4647 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:27:00.000926  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.972016 10281 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:00.006364  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.972035 10222 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:26:59.953177  9776 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.009075  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.003080  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:00.011114  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:00.020825 10292 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.401s	user 0.001s	sys 0.000s
I20260709 12:27:00.025246  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.026557  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:27:00.007144  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.986683 10276 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:00.030067 10276 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.030488 10276 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:00.030679 10171 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:00.031337  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.031406 10171 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.032693  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:00.033185  5349 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:59.986898 10284 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:00.034813 10230 raft_consensus.cc:3055] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:00.035293 10230 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:00.035753 10281 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:00.016037  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.996120 10279 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:00.036098 10230 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.036818  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:00.037261 10279 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.038453 10171 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:00.033926  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:00.040432  5347 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:27:00.034703  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:00.042043  5343 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 15: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:00.042234  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3297) took 1198 ms. Trace:
I20260709 12:27:00.042631  4750 rpcz_store.cc:276] 0709 12:26:58.843536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.637998 (+794462us) service_pool.cc:224] Handling call
0709 12:26:59.638257 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:59.643262 (+  5005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.041985 (+398723us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b540004c754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:00.042174 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9998080}
I20260709 12:27:00.043859  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:00.044981  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:00.047631 10149 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:00.051061  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:00.051723  4748 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:27:00.003602 10251 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:00.053762  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:00.055456  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:00.006726 10138 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:00.058744  5349 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:00.027540  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.011831  9990 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:00.062973  4600 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 16: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:00.016027  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:00.061357  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 180642us
I20260709 12:27:00.062407  4679 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:00.064445  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.057238  9653 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.051261 10145 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:00.037285 10230 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.072690  5349 raft_consensus.cc:1240] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:00.060338  9983 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:00.061018  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:00.076436  4748 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:00.009966  4596 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:27:00.060303  9882 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.077344  4677 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:00.060021 10170 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.077817  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:00.124537  4602 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 15: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:00.067445 10171 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:00.084787  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.088027  5343 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:27:00.069875 10284 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:00.085939  4749 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:27:00.075045  9990 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 20 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:00.069693 10224 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:27:00.128993  9882 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.129969 10190 raft_consensus.cc:3055] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:00.130386 10190 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:00.130447  9990 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.092476  9776 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:00.073352 10138 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:00.132684  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2743) took 1481 ms. Trace:
I20260709 12:27:00.133196  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:00.132709 10138 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.118702 10099 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:00.126932 10170 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.078764  9653 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.133525  5340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50644 (request call id 781) took 1110 ms. Trace:
I20260709 12:27:00.135550  5340 rpcz_store.cc:276] 0709 12:26:59.022733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.022917 (+   184us) service_pool.cc:224] Handling call
0709 12:26:59.023222 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:59.368320 (+345098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.133259 (+764939us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b54007116d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:00.133465 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":787163008}
I20260709 12:27:00.136529 10170 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:00.136734 10290 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.619s	user 0.003s	sys 0.000s
I20260709 12:27:00.078117 10145 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.138087  9653 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:00.138335  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2746) took 1484 ms. Trace:
I20260709 12:27:00.138728  4752 rpcz_store.cc:276] 0709 12:26:58.653802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.987026 (+333224us) service_pool.cc:224] Handling call
0709 12:26:58.987363 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:58.989717 (+  2354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:58.989743 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:58.994395 (+  4652us) log.cc:844] Serialized 26 byte log entry
0709 12:26:59.038783 (+ 44388us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:00.137816 (+1099033us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.138025 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.138104 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.138145 (+    41us) raft_consensus.cc:1685] finished
0709 12:27:00.138157 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.138284 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1100333,"raft.queue_time_us":1844,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":16874,"thread_start_us":43993,"threads_started":1,"wal-append.queue_time_us":91014}
I20260709 12:27:00.139654  4571 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:00.140522  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:00.142066 10204 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:00.131457 10190 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.143342 10190 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:00.145713 10300 log.cc:927] Time spent T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.383s	user 0.003s	sys 0.000s
I20260709 12:27:00.147902 10191 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:00.148687 10191 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:00.149418 10191 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.150076 10097 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:00.124780  5349 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:27:00.134397  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:00.151937  4600 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:27:00.074738  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:00.128430  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:00.153357  4749 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:27:00.154207  5340 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:00.154783  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:00.155648  4752 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:00.155367  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:00.156569  4749 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:00.131449  9882 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:00.133090  4747 rpcz_store.cc:276] 0709 12:26:58.651786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.767003 (+115217us) service_pool.cc:224] Handling call
0709 12:26:58.773683 (+  6680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:58.796036 (+ 22353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:58.796085 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:58.839227 (+ 43142us) log.cc:844] Serialized 26 byte log entry
0709 12:26:58.926508 (+ 87281us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:00.132131 (+1205623us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.132347 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.132431 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.132471 (+    40us) raft_consensus.cc:1685] finished
0709 12:27:00.132482 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.132600 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1205271,"prepare.queue_time_us":1208068,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":8520,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":19347,"thread_start_us":86798,"threads_started":1,"wal-append.queue_time_us":155273}
I20260709 12:27:00.139780 10145 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:00.152084  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.152886  4753 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 13: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:00.075169  9983 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:00.128253  4647 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:00.156752  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:00.123874  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:00.158051  4571 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:00.159787  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:00.160234  4678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8e62ff81eb34853811314530131292b"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:27:00.124565  4680 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:27:00.125859 10171 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:00.164743  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2744) took 1511 ms. Trace:
W20260709 12:27:00.175570  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47413.1us
I20260709 12:27:00.174685  4751 rpcz_store.cc:276] 0709 12:26:58.653161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.775417 (+122256us) service_pool.cc:224] Handling call
0709 12:26:58.775710 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:58.776056 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:58.776081 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:58.783500 (+  7419us) log.cc:844] Serialized 26 byte log entry
0709 12:26:59.655525 (+872025us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b180041b6c0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:26:59.996975 (+341450us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:59.997021 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.997099 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.164340 (+167241us) raft_consensus.cc:1685] finished
0709 12:27:00.164373 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.164661 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3350,"raft.queue_time_us":4217,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":5566,"spinlock_wait_cycles":8143744,"thread_start_us":341218,"threads_started":1,"wal-append.queue_time_us":1378706}
I20260709 12:27:00.176096  5349 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:00.167774  4752 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:27:00.173995  9983 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:00.177640  4571 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:00.178170  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:00.178432 10149 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:00.178903  4747 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:27:00.180624 10145 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:00.181094 10145 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.181493 10145 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:00.162434  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 91671.2us
I20260709 12:27:00.183332  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:00.166838  4602 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:27:00.186100  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.185668  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:00.153714 10099 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.186302  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:00.177673 10170 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:00.198302  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:00.164973 10281 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:00.214767  4601 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:27:00.181931 10171 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:00.219058  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.221971  4601 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:27:00.218801  4602 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:00.216674  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:00.225500  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:00.210944  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.227501  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:00.212002  9740 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.027s	sys 0.031s
I20260709 12:27:00.231756 10309 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:00.199460 10204 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:00.203948 10104 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.199893 10301 raft_consensus.cc:2764] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:27:00.234514  5349 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:27:00.219275 10229 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:00.269658  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:00.260756  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:00.262917  4568 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:00.218147 10222 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:00.263888  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.277812  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:00.204385 10149 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:00.231781  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.267995  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:00.257930  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:00.203393 10170 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:00.254716  9740 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.028s	sys 0.031s
I20260709 12:27:00.268436 10297 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:00.282056  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:00.292796  4748 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:27:00.267568 10200 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:00.315651  4720 leader_election.cc:400] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12.
I20260709 12:27:00.319820  4720 leader_election.cc:403] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:27:00.321451  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:00.276254 10299 log.cc:927] Time spent T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:27:00.290901 10009 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:00.269776 10229 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.322405 10198 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:00.218212 10284 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.267357 10309 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.334311 10309 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:00.340021  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:00.340606  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.281669 10298 consensus_queue.cc:1059] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:27:00.314206 10191 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:00.343452 10021 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.311859  9882 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:27:00.291832 10206 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:00.326501 10248 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.270205 10104 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.350431 10192 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 11. Current term is 12.
I20260709 12:27:00.350600 10132 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:00.344885  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:00.346431  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:00.356884 10229 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.277532 10222 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:00.360358  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.329334 10297 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.267297 10276 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.276237 10204 raft_consensus.cc:697] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:00.327572 10200 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.218943 10099 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.426553  9983 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.428920 10099 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:00.429059  9983 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:00.368851  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.368857  5312 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:00.372545  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:00.459429  4603 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:00.373322  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:00.311666 10154 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:27:00.373749  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.375604  4605 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:00.468961  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:00.469722  4676 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:00.331369  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:00.470690  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 139429us
I20260709 12:27:00.314658 10170 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.350648 10149 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.383510  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1012) took 2770 ms. Trace:
I20260709 12:27:00.380303 10298 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.380503 10301 raft_consensus.cc:3055] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:00.345822 10251 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:00.380381 10021 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.395814  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1022) took 1527 ms. Trace:
I20260709 12:27:00.484973  4750 rpcz_store.cc:276] 0709 12:26:58.868429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.072699 (+1204270us) service_pool.cc:224] Handling call
0709 12:27:00.074298 (+  1599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.074512 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:00.147822 (+ 73310us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.395433 (+247611us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:00.395486 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:00.395538 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.395609 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.395748 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72741,"prepare.queue_time_us":75266,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":15628,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":16204,"thread_start_us":235411,"threads_started":1}
I20260709 12:27:00.401410  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3339) took 1125 ms. Trace:
I20260709 12:27:00.486956  4681 rpcz_store.cc:276] 0709 12:26:59.276166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.276318 (+   152us) service_pool.cc:224] Handling call
0709 12:26:59.284510 (+  8192us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:59.335624 (+ 51114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.401163 (+1065539us) spinlock_profiling.cc:243] Waited 41.7 ms on lock 0x7b5400099f54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:00.401355 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95923456}
I20260709 12:27:00.488384  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:00.489037  4681 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:00.403122  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3301) took 1521 ms. Trace:
I20260709 12:27:00.404852  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:00.406062  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:00.408893  4597 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:00.385090 10206 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:00.489851  4747 rpcz_store.cc:276] 0709 12:26:58.881844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.179652 (+1297808us) service_pool.cc:224] Handling call
0709 12:27:00.179980 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.180195 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:00.180609 (+   414us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.402688 (+222079us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:00.402739 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:00.402801 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.402891 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.403050 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6620,"prepare.queue_time_us":701,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":6824,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":26664,"thread_start_us":214877,"threads_started":1}
I20260709 12:27:00.491139 10191 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:00.492007  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:00.492544 10021 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:00.492625  4747 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:00.484607 10099 raft_consensus.cc:3055] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:00.493515 10281 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:00.493315 10099 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:00.483022 10301 raft_consensus.cc:740] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:00.430527  4749 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:00.494982 10099 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.495205 10301 consensus_queue.cc:260] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.495760  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3302) took 1601 ms. Trace:
I20260709 12:27:00.496527 10301 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:00.496272  4749 rpcz_store.cc:276] 0709 12:26:58.894022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.430030 (+1536008us) service_pool.cc:224] Handling call
0709 12:27:00.430342 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.495450 (+ 65108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.495692 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:00.425820 10204 consensus_queue.cc:237] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.498208  5343 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:00.432353  4678 raft_consensus.cc:606] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:00.328159 10009 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.499001  4678 raft_consensus.cc:614] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:00.440047  4752 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:00.499207 10009 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:00.499791  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3304) took 1380 ms. Trace:
I20260709 12:27:00.500159  4752 rpcz_store.cc:276] 0709 12:26:59.119846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.439360 (+1319514us) service_pool.cc:224] Handling call
0709 12:27:00.439656 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.499547 (+ 59891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.499738 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:00.446600  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:00.461000  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.432696 10200 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:00.405181 10222 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:00.469641  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:00.512737  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.512955  4675 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 12: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:00.378305  4748 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 8.
I20260709 12:27:00.399291 10277 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14.
W20260709 12:27:00.520641  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.524788  9735 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:00.474306  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:00.531121  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:00.531430  4748 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 11. Current term is 12. Ops: []
W20260709 12:27:00.532860  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.532948  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:00.476313  4602 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
W20260709 12:27:00.476888  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 108131us
I20260709 12:27:00.476400  4755 rpcz_store.cc:276] 0709 12:26:57.611198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.611305 (+   107us) service_pool.cc:224] Handling call
0709 12:26:59.792198 (+2180893us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:59.871815 (+ 79617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.381893 (+510078us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b54000a7650. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:00.382027 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4972494080}
I20260709 12:27:00.477942  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:00.535019  5342 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:00.491115 10206 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:00.481488 10129 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:00.482854 10298 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.438424  4751 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:00.445767  5340 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:00.429170  4753 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:00.461282  9951 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.501467  4752 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:00.547084  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1023) took 1660 ms. Trace:
W20260709 12:27:00.503979  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 47464.7us
W20260709 12:27:00.504082  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47184.4us
I20260709 12:27:00.549266  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:00.496356 10099 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:00.507450  4597 raft_consensus.cc:1240] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:00.476325  9984 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:00.478838 10279 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.476040 10100 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.536991  4748 raft_consensus.cc:1240] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:00.536633  4603 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:27:00.506178  4750 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:00.547704  4753 rpcz_store.cc:276] 0709 12:26:58.886960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.428617 (+1541657us) service_pool.cc:224] Handling call
0709 12:27:00.428971 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.546802 (+117831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.547022 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:00.571949  4753 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:00.554145  4752 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:27:00.560667  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.574944  4597 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:00.563721  4681 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:00.563908  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3303) took 1533 ms. Trace:
I20260709 12:27:00.576131  4751 rpcz_store.cc:276] 0709 12:26:59.030786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.437867 (+1407081us) service_pool.cc:224] Handling call
0709 12:27:00.438226 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.563606 (+125380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.563843 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:00.596271  4747 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:00.597395  4749 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:00.597862  9946 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:27:00.562644  9984 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:00.597703 10221 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:00.598712 10143 raft_consensus.cc:3055] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:00.645046  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:00.645833  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:00.648409  5349 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 12:27:00.655232  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.573939 10152 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 12:27:00.654531  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:00.658846  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:00.640497  9984 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:00.568930  9951 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.599124 10100 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.636920 10129 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:27:00.651614 10097 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:00.630113 10021 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:27:00.577880 10279 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.597985 10182 consensus_queue.cc:1059] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:27:00.646790 10143 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:00.689868  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3290) took 2393 ms. Trace:
I20260709 12:27:00.646530 10221 raft_consensus.cc:697] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:00.691516  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:00.691941  4674 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:00.629814  9892 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:27:00.684432 10309 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:00.721338  4571 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:00.716451  4756 rpcz_store.cc:276] 0709 12:26:58.296394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.296604 (+   210us) service_pool.cc:224] Handling call
0709 12:26:58.296932 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:58.297191 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:58.663411 (+366220us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.125489 (+1462078us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b1800426c40. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007effeb3cba40 00007effeb3c64e4 00007effeb3c6455 00007efff1eae3c7 00007effee4c09a7 00007effee4d00bf 00007effee4d0038
0709 12:27:00.126884 (+  1395us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.689275 (+562391us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:00.689349 (+    74us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:00.689421 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.689636 (+   215us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.689789 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41886,"prepare.queue_time_us":1749343,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":77,"raft.queue_time_us":28774,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":15880,"spinlock_wait_cycles":13397376,"thread_start_us":867971,"threads_started":3,"wal-append.queue_time_us":499371}
I20260709 12:27:00.724370  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:00.725961  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:00.729146  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:00.729871  4748 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:00.692093 10287 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:00.732565 10154 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:00.734709 10302 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:00.734862  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:00.736807 10295 log.cc:927] Time spent T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 1.040s	user 0.005s	sys 0.000s
I20260709 12:27:00.724309  9653 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.719719 10104 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:00.746974  5347 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:00.717047 10276 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:00.751304 10145 consensus_meta.cc:287] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.561s	user 0.010s	sys 0.001s
I20260709 12:27:00.751786 10145 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.728050 10248 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:00.754626 10143 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.755492  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2752) took 1934 ms. Trace:
I20260709 12:27:00.755678 10143 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.755896  4754 rpcz_store.cc:276] 0709 12:26:58.821411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.287903 (+466492us) service_pool.cc:224] Handling call
0709 12:26:59.309247 (+ 21344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:59.333093 (+ 23846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.333636 (+   543us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:59.338599 (+  4963us) log.cc:844] Serialized 26 byte log entry
0709 12:27:00.039100 (+700501us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b180039a5a0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:27:00.395488 (+356388us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:00.395532 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.395605 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.755201 (+359596us) raft_consensus.cc:1685] finished
0709 12:27:00.755221 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.755431 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34619,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":18086,"spinlock_wait_cycles":33851776,"thread_start_us":371468,"threads_started":2,"wal-append.queue_time_us":1326241}
I20260709 12:27:00.757910  4604 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:00.758145  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:00.758773  4754 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:27:00.731079  9882 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:00.741412 10154 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.739679 10309 raft_consensus.cc:697] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:00.738190 10297 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:00.722146 10190 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:27:00.773130 10221 consensus_queue.cc:237] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.782625 10218 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:00.783074 10218 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:00.783824 10218 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.785138 10218 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:00.787916  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:00.773463 10154 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:00.789240 10298 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:00.769716 10323 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:00.797246  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 903) took 1001 ms. Trace:
I20260709 12:27:00.799059  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:00.785477 10315 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:27:00.778196 10190 consensus_queue.cc:1243] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:27:00.797735  4600 rpcz_store.cc:276] 0709 12:26:59.795372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.071569 (+276197us) service_pool.cc:224] Handling call
0709 12:27:00.797076 (+725507us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b5400140854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:00.797185 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57006336}
W20260709 12:27:00.817626  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 245476us
I20260709 12:27:00.821936  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1027) took 1430 ms. Trace:
I20260709 12:27:00.822386  4749 rpcz_store.cc:276] 0709 12:26:59.391373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.497540 (+1106167us) service_pool.cc:224] Handling call
0709 12:27:00.597117 (+ 99577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.597941 (+   824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.821707 (+223766us) spinlock_profiling.cc:243] Waited 99.1 ms on lock 0x7b5400117e50. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:00.821867 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227975552}
I20260709 12:27:00.823997  4749 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:00.803490 10229 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.827643 10229 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:00.803370 10145 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:00.797531 10323 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:00.828840 10323 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:00.828899  4752 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:00.817059 10190 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:00.829890 10319 log.cc:927] Time spent T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.117s	user 0.002s	sys 0.000s
I20260709 12:27:00.776357  9735 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:27:00.831099  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.834100  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.834408  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.834995  4597 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:27:00.779487  9961 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:27:00.831823  4598 raft_consensus.cc:2393] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:00.836666  4720 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:00.802125 10104 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:00.830039 10190 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.865959 10104 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:00.867384 10190 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:00.867969  9951 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:00.868391  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:00.838824 10318 log.cc:927] Time spent T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:27:00.838719  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:00.869004  4598 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:00.869501  4600 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:00.837615  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:00.870117  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:00.870445 10230 raft_consensus.cc:3055] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:00.870872 10230 raft_consensus.cc:740] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:00.839946  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:00.876309 10287 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:00.876924 10287 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.877158  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:27:00.852792 10321 log.cc:927] Time spent T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:27:00.878247 10287 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:00.857079  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:00.879568  4747 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.880167  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:00.841171  9984 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.858425 10309 consensus_queue.cc:237] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.881880  4756 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:00.883009  9984 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:00.883544  4756 raft_consensus.cc:1240] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:00.858052 10279 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:00.860303 10112 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.884989 10132 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:00.886119 10138 raft_consensus.cc:3055] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:00.886590 10138 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:00.886780 10009 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.887429 10138 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.888392 10138 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:00.889103  4747 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:27:00.889673 10297 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:00.866852 10104 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:00.839385 10316 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 12:27:00.840538 10192 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:00.891597  4752 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:00.888396 10301 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.894371  4752 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:00.894225 10301 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.896517 10301 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:00.874265  4603 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:00.858670 10154 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.898522  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:00.898970  4599 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:27:00.899708 10154 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:00.900327  4599 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:00.858045  9653 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:00.875226  4598 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 12:27:00.875720  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:00.902287  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.873060 10170 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.903328  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:00.903650  4601 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:00.905045 10261 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:00.907963 10322 raft_consensus.cc:3055] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:00.908579 10322 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:00.909858  4678 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:00.909729 10322 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.910645  4568 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:00.878407 10200 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:00.911854  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:00.913017 10100 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:00.878765 10320 consensus_queue.cc:1059] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:00.913647 10100 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.914376 10320 raft_consensus.cc:3055] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:00.884930 10112 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.914793 10320 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:00.915745 10191 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:00.916235 10191 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.916436 10112 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:00.916625 10191 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:00.917781  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:00.918038  4679 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:27:00.918376  4750 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:00.886221 10132 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:00.907905  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:00.907927  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:00.923341  5340 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:00.869081 10171 consensus_meta.cc:287] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.617s	user 0.012s	sys 0.000s
I20260709 12:27:00.915695 10320 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.901512  4599 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: []
W20260709 12:27:00.935928  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 364133us
W20260709 12:27:00.941293  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 122144us
I20260709 12:27:00.861140  9961 raft_consensus.cc:3055] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:00.896557  4752 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 12:27:00.907253  4603 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:27:00.947535  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:00.943608  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2768) took 1004 ms. Trace:
I20260709 12:27:00.948874  4601 rpcz_store.cc:276] 0709 12:26:59.939336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.431412 (+492076us) service_pool.cc:224] Handling call
0709 12:27:00.431715 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.904059 (+472344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.943279 (+ 39220us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b54003d0254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:00.943525 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1084977024}
I20260709 12:27:00.892902 10206 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.871635 10230 consensus_queue.cc:260] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:00.969801  4678 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:27:00.874234  4597 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:27:00.969429  5340 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:00.921008  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:00.921479  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:00.972189  4680 raft_consensus.cc:2410] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:00.844666 10222 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:00.914121 10100 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:00.990260  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 64080.3us
I20260709 12:27:00.947413  4752 pending_rounds.cc:85] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 7
I20260709 12:27:00.991848  4752 pending_rounds.cc:107] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:27:00.992187  4752 raft_consensus.cc:2889] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:00.925994  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:27:00.995054  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 69140.7us
I20260709 12:27:00.996061  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:00.880556  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:00.859999 10099 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:00.950770  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:01.011981  5346 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:00.951570  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:00.958446  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:01.014515  5341 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:00.960924  4569 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:00.968153  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:01.023492  5347 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:00.974541  4568 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:01.024384  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 49929us
I20260709 12:27:00.975041  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:00.979480  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2760) took 1162 ms. Trace:
I20260709 12:27:01.026026  4750 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:00.985466  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:01.051725  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 43539.7us
W20260709 12:27:00.999733  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.002156 10279 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.957671  9961 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:01.054332 10279 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.054768  9961 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.056016  9961 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:00.987655  9951 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:27:01.057219 10279 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:01.003670  4571 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:01.004051  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.069077  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 65492.1us
I20260709 12:27:01.029844  4748 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:01.004415  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:01.006002  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:00.950949 10171 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.007115 10100 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:00.989737 10192 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:27:01.008244  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:01.008347  5340 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:27:01.083434  5340 raft_consensus.cc:606] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:01.025074  4755 rpcz_store.cc:276] 0709 12:26:59.817400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.534295 (+716895us) service_pool.cc:224] Handling call
0709 12:27:00.546160 (+ 11865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.546384 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:00.571074 (+ 24690us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.951204 (+380130us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.979007 (+ 27803us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:00.979078 (+    71us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:00.979147 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.979239 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.979396 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54814,"prepare.queue_time_us":33503,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":82,"raft.queue_time_us":77880,"spinlock_wait_cycles":90962816,"thread_start_us":364001,"threads_started":1}
I20260709 12:27:00.985414  4674 raft_consensus.cc:2393] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 15: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:01.085130  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:00.922354 10322 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:00.922187 10132 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:00.922415 10009 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:01.026819  4719 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:01.042373 10218 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:27:01.086987  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 60252.8us
I20260709 12:27:00.998428 10170 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:01.048514  4568 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:01.052610  4645 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:01.090204  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:01.052852  4749 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:01.003492  5348 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: [13.13-13.13]
W20260709 12:27:01.057825  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 84375.3us
I20260709 12:27:01.090958  5340 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:01.058347 10152 consensus_queue.cc:1059] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:01.061721  9983 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.061734  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:01.088850  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:01.094949  4673 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:01.095938  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:01.097012  4596 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 15: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:01.062757  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:01.097900  4680 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:27:01.098213  4753 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:27:01.004503 10261 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:01.105453  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:01.079754 10224 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:01.078117  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:01.011515 10099 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:01.080232  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:01.112031  4752 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:01.084173  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:01.112920  4678 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:00.942682 10200 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:01.045754  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 84936.4us
W20260709 12:27:01.115821  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63303.3us
I20260709 12:27:01.116420  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:01.028446  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 53496.3us
I20260709 12:27:01.085745  4674 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:01.118432  4571 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:01.042613 10154 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:01.093986 10279 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:01.090687  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.050066  9653 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:01.092726  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:27:01.092847  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 44463.1us
I20260709 12:27:01.051297 10320 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:01.094983  9735 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:27:01.105408 10218 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:01.130359  4755 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:01.131170 10218 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.134378  4753 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:01.135764  4753 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:01.097216  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:01.094422  4719 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:01.139232  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 44890.1us
I20260709 12:27:01.088455 10190 consensus_queue.cc:1059] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:27:01.107281 10009 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:01.114954  9882 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:01.125854  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:01.110997 10224 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.143206  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3309) took 1047 ms. Trace:
I20260709 12:27:01.143965  4753 raft_consensus.cc:1275] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:27:01.131803  4755 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:27:01.158159  4748 rpcz_store.cc:276] 0709 12:27:00.095660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.866466 (+770806us) service_pool.cc:224] Handling call
0709 12:27:00.866760 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.071901 (+205141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.142979 (+ 71078us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400184054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:01.143152 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374443392}
W20260709 12:27:01.159232  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 55513.7us
I20260709 12:27:01.002450  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:01.046653  9984 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:01.160440  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 158058us
I20260709 12:27:01.160738  4752 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:01.161196  5312 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:01.164898  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:01.122114  9936 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:01.163785  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:01.121816 10100 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:01.158214  4753 pending_rounds.cc:85] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 8
I20260709 12:27:01.126214 10279 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.110127  9983 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.223850  4681 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:27:01.224357  4753 pending_rounds.cc:107] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:27:01.224737  4753 raft_consensus.cc:2889] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:01.223918  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 60208.4us
I20260709 12:27:01.118352 10302 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:27:01.227310  4755 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:27:01.212841  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.233300  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 107524us
I20260709 12:27:01.158439 10009 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.169342 10261 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:01.163559 10224 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:01.121964 10200 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:01.288609  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 168167us
I20260709 12:27:01.171721  9882 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:01.172394 10248 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:01.150411 10221 consensus_queue.cc:1059] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 12:27:01.169679 10145 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:27:01.298595  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 59000.9us
W20260709 12:27:01.293888  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 175524us
I20260709 12:27:01.226747 10097 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:01.172276 10281 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.122s
I20260709 12:27:01.159950 10325 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.151s
I20260709 12:27:01.290505 10143 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:01.169893 10008 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.147s
I20260709 12:27:01.323467  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:01.324096  4755 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:27:01.236857 10284 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:01.325824  4747 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:01.325938  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:01.326682  4753 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:01.157913 10132 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.136962 10154 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.241957 10009 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:01.337296  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 2771) took 1217 ms. Trace:
I20260709 12:27:01.341188  4596 rpcz_store.cc:276] 0709 12:27:00.119684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.123633 (+  3949us) service_pool.cc:224] Handling call
0709 12:27:01.337018 (+1213385us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b54003d0754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:01.337215 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2135934208}
I20260709 12:27:01.303570 10182 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.338030  4605 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:01.341678 10152 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
W20260709 12:27:01.345371 10332 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
W20260709 12:27:01.346356 10334 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:27:01.231693 10192 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:01.447108 10192 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:01.448153 10192 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.335927 10222 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:01.353585 10132 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:01.231232 10099 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:01.454178 10287 consensus_meta.cc:287] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.561s	user 0.004s	sys 0.006s
I20260709 12:27:01.454694 10287 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.456756 10287 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:01.457825 10099 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:01.458144 10138 consensus_meta.cc:287] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.569s	user 0.003s	sys 0.007s
I20260709 12:27:01.458690 10138 raft_consensus.cc:3055] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:01.459055 10138 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:01.459810 10138 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.460952 10138 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:01.461769  4756 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 14. Ops: []
I20260709 12:27:01.463404  9882 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.465854 10347 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:01.466506 10302 consensus_queue.cc:1059] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:01.466374 10347 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.467619 10324 raft_consensus.cc:3055] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:01.468007 10324 raft_consensus.cc:740] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:01.468595 10347 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:01.468752 10324 consensus_queue.cc:260] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:01.438030 10337 log.cc:927] Time spent T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:27:01.472517  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1030) took 1522 ms. Trace:
I20260709 12:27:01.473455 10229 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:01.342818 10284 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.475781 10200 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:01.417124 10182 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.478571 10182 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:01.416563 10328 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.367s	user 0.000s	sys 0.001s
I20260709 12:27:01.408149 10251 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:01.479470  4680 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:01.324903 10206 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:01.227926 10279 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:01.472864  4754 rpcz_store.cc:276] 0709 12:26:59.950326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.759339 (+809013us) service_pool.cc:224] Handling call
0709 12:27:00.759640 (+   301us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:00.759989 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.760013 (+    24us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:00.775945 (+ 15932us) log.cc:844] Serialized 52 byte log entry
0709 12:27:01.088342 (+312397us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:01.111516 (+ 23174us) log.cc:844] Serialized 14 byte log entry
0709 12:27:01.116301 (+  4785us) log.cc:844] Serialized 14 byte log entry
0709 12:27:01.121204 (+  4903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.121275 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:01.471892 (+350617us) raft_consensus.cc:1685] finished
0709 12:27:01.471912 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.472454 (+   542us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4344,"prepare.queue_time_us":23012,"prepare.run_wall_time_us":442,"raft.queue_time_us":17978,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":16604,"spinlock_wait_cycles":13552640,"thread_start_us":269207,"threads_started":1,"wal-append.queue_time_us":499039}
I20260709 12:27:01.334280 10275 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.111s
I20260709 12:27:01.473992 10229 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:27:01.478835  4677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51858 (request call id 2114) took 1270 ms (client timeout 1575 ms). Trace:
W20260709 12:27:01.490859  4677 rpcz_store.cc:269] 0709 12:27:00.208319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.208533 (+   214us) service_pool.cc:224] Handling call
0709 12:27:01.478612 (+1270079us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b540019f854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:01.478770 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2047923456}
I20260709 12:27:01.444643 10143 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:01.485783  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3311) took 1389 ms. Trace:
I20260709 12:27:01.340466 10261 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:01.291858  9951 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:27:01.320926 10145 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:01.517403  4756 rpcz_store.cc:276] 0709 12:27:00.096241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.884234 (+787993us) service_pool.cc:224] Handling call
0709 12:27:00.884486 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.462211 (+577725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.464042 (+  1831us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b54001f1654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:01.485695 (+ 21653us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1327176064}
W20260709 12:27:01.542106  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.518040  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 347464us
I20260709 12:27:01.523399  4753 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
W20260709 12:27:01.528453  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 208948us
W20260709 12:27:01.528963  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.530267  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 211020us
W20260709 12:27:01.531293  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 230849us
I20260709 12:27:01.532809  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3323) took 1421 ms. Trace:
I20260709 12:27:01.531371  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 914) took 1487 ms. Trace:
I20260709 12:27:01.556351  4605 rpcz_store.cc:276] 0709 12:27:00.043917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.475871 (+431954us) service_pool.cc:224] Handling call
0709 12:27:01.337789 (+861918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.358297 (+ 20508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.531159 (+172862us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b54003d0750. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:01.531310 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1981367680}
I20260709 12:27:01.475781 10284 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:01.557332  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:01.556283  4747 rpcz_store.cc:276] 0709 12:27:00.111646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.075455 (+963809us) service_pool.cc:224] Handling call
0709 12:27:01.075750 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.326348 (+250598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.532583 (+206235us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54000e56d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:01.532751 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":524012288}
I20260709 12:27:01.566705  4752 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:01.567015  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:01.565457  4754 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:01.569342  4747 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:01.573935  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:01.576347  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:01.577033  4756 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:01.539732 10274 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:01.582726  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:01.540536 10171 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:01.581732  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:01.583628  4751 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:01.598278  4569 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:01.597682  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 423710us
I20260709 12:27:01.588009  4755 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:01.642226  9983 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:01.617398 10229 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.654160 10229 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:01.654592 10170 consensus_queue.cc:1175] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3a5aeedd11fe49938079fe86cb01232c" responder_term: 17 status { last_received { term: 17 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:01.662770 10261 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.646757 10143 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:01.666940 10320 consensus_meta.cc:287] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.537s	user 0.008s	sys 0.000s
I20260709 12:27:01.655681 10347 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:01.642477 10341 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
I20260709 12:27:01.668808 10347 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.669227 10347 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:01.672039 10322 consensus_meta.cc:287] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.578s	user 0.007s	sys 0.001s
W20260709 12:27:01.655963 10323 consensus_meta.cc:287] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.827s	user 0.012s	sys 0.001s
I20260709 12:27:01.678164 10323 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.676519 10224 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.680660 10323 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:01.656577 10154 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:01.619113  9740 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:01.682106  9740 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.685344  9740 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:01.660303 10191 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.000s
W20260709 12:27:01.662185  9653 consensus_meta.cc:287] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.525s	user 0.003s	sys 0.007s
I20260709 12:27:01.641876 10145 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.618533  4569 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:27:01.715024  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 96549.5us
I20260709 12:27:01.717381  4569 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:01.587394 10360 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:27:01.687520  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.701025  5349 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.001s	sys 0.024s
W20260709 12:27:01.720840  5349 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.001s	sys 0.025s
W20260709 12:27:01.708683  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 130496us
I20260709 12:27:01.725205  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:01.728840  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:01.677188 10347 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.743556 10009 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.594170 10284 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.138s
I20260709 12:27:01.731603 10320 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:01.745682 10360 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:01.746160  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:01.746760 10360 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:01.747694  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 59936.6us
I20260709 12:27:01.731714 10323 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:01.748512 10009 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:01.717057  9653 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:01.741216 10349 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:01.716291 10154 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:01.744103 10347 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:01.747635 10351 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:01.748615 10323 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.749763  4645 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:01.750320 10355 consensus_queue.cc:1059] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:01.740677  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:01.702991 10309 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.581185 10221 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 12:27:01.775705 10221 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:01.776074 10221 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:01.776909 10221 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.778077 10221 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:01.705461 10218 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:01.778748 10218 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.779552  9990 raft_consensus.cc:3055] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:01.779932  9990 raft_consensus.cc:740] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:01.780820 10218 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:01.739374  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:01.781554  4605 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:01.752944  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:01.782052  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:27:01.757843  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 45189.2us
I20260709 12:27:01.741317  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:01.597128 10274 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.772013 10009 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.783914 10009 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.785135 10274 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:01.785943 10009 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:01.786270 10343 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.342s	user 0.004s	sys 0.000s
I20260709 12:27:01.786520  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:01.787278  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:01.787905  4605 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:01.788285  4603 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:01.599248  9961 consensus_queue.cc:1059] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.167s
W20260709 12:27:01.774919 10361 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:27:01.790628  9961 raft_consensus.cc:3055] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:01.791059  9961 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:01.777987  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:01.774451 10309 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.782707  4596 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:01.792748  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:01.793207  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 43527.2us
I20260709 12:27:01.792801  5347 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:01.766086 10154 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.717712 10191 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.808620  4603 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:27:01.809607  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 228822us
I20260709 12:27:01.798681 10206 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.818784  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:01.773241 10323 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:01.815598  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:01.827183  4605 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
W20260709 12:27:01.828514  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 96157.3us
I20260709 12:27:01.829819  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:01.810070  9951 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:01.830978  4603 raft_consensus.cc:1240] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:01.800238 10298 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:01.832913  4571 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:27:01.873019  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57543.2us
I20260709 12:27:01.810568 10204 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:01.814481 10154 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:01.839558  5347 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
W20260709 12:27:01.882570  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 52835.8us
I20260709 12:27:01.803472 10274 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:27:01.842600  4596 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:27:01.831600 10206 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.828039 10218 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:01.959745 10218 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.962548 10218 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:01.875186  9653 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:01.974757  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2771) took 2000 ms. Trace:
I20260709 12:27:01.975191  4749 rpcz_store.cc:276] 0709 12:26:59.974384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.824689 (+850305us) service_pool.cc:224] Handling call
0709 12:27:00.824936 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.092180 (+267244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.974479 (+882299us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b540004e054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:01.974689 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":523596032}
I20260709 12:27:01.880728  9961 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:01.902470  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 69628.9us
I20260709 12:27:01.945714  9984 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:01.893196 10298 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:01.982414 10367 log.cc:927] Time spent T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:27:01.983789 10298 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:01.888723  9951 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:01.895771  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.984612  9951 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:01.978453  9653 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.985312  9653 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:01.987439  9961 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:01.949935  4598 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:01.988695  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:01.989894  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:01.991994  4674 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:27:01.951028  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:01.813150 10200 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 12:27:01.894629  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:01.828755 10097 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.104s
I20260709 12:27:01.957875  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:01.966504  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:01.999682  4603 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:01.818756 10363 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.167s
I20260709 12:27:01.991842  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:01.891952 10301 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:01.965257 10297 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:01.994377  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 99807.8us
I20260709 12:27:02.017871 10297 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.020146 10297 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:02.020340  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:01.995450  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:02.021327 10364 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.129s
I20260709 12:27:02.022763 10206 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:02.023424 10206 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:02.023959 10206 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:02.001457 10104 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:02.024811 10206 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:01.876999 10224 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:01.832326 10347 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:01.975694 10192 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.020s	sys 0.049s
I20260709 12:27:02.001734  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:01.973165 10145 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:02.035837 10145 raft_consensus.cc:697] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:01.878618 10143 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:27:02.037923 10344 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.591s	user 0.005s	sys 0.000s
I20260709 12:27:01.874951 10279 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:01.979383 10129 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:27:02.009254 10309 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:02.009773 10301 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:02.011538 10191 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:01.978686 10369 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:01.963250  4495 catalog_manager.cc:5697] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 11 to 20, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 20 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:02.026641  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:01.826556 10284 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.147s
W20260709 12:27:02.030483  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 156914us
W20260709 12:27:02.097551  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 71018.8us
I20260709 12:27:02.034104 10298 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:27:02.037778  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:02.100994  4596 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:01.882376 10204 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:01.829524 10182 consensus_queue.cc:1059] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.172s
I20260709 12:27:02.005762  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:02.070539 10323 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.053016 10363 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:02.075858  4597 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:27:02.039219 10192 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.020s	sys 0.049s
I20260709 12:27:02.074954  9951 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.083078  9990 consensus_queue.cc:260] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.119721  9951 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:02.092170 10301 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.120913  4680 raft_consensus.cc:2393] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 18: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:02.052047 10143 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:02.093397 10145 consensus_queue.cc:237] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.089315  9653 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.071210 10279 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:02.052618 10097 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:02.108273  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:02.132649  4596 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:27:02.073376 10230 raft_consensus.cc:3060] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:02.100868  9740 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:27:02.114796  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 109162us
I20260709 12:27:02.137768  4673 raft_consensus.cc:3055] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:02.138171  4673 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:02.117875 10221 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:02.088763 10324 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:01.945681 10353 consensus_queue.cc:1059] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.200s
I20260709 12:27:02.103907 10309 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:02.131268  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 404147us
I20260709 12:27:02.133735  4604 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:27:02.107134 10204 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:02.139060  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.140496  4675 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:02.141001  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:02.100633 10369 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.057919 10347 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.149403  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.144176  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:02.156214  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.127105 10192 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:02.159623  4596 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:02.166136  4598 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:02.138928  4673 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.109150 10351 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:27:02.122702 10363 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:02.202564  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.167963 10297 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:27:02.182133  4598 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
W20260709 12:27:02.183616 10371 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:27:02.149511  9740 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.186116  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:02.226833  4681 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:02.190099  4673 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:02.189930 10359 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:02.168507 10251 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:27:02.168715 10322 consensus_queue.cc:1048] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
I20260709 12:27:02.175115  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:02.278370  4596 raft_consensus.cc:2368] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17.
I20260709 12:27:02.140318 10097 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.279217  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:02.280368 10097 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:02.280650  4596 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:02.165082 10373 consensus_queue.cc:1059] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:27:02.215873  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:02.282477 10373 raft_consensus.cc:3055] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:02.282927 10373 raft_consensus.cc:740] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:02.283377  4598 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:02.283818 10373 consensus_queue.cc:260] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.285256 10373 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:02.275125 10359 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.130940 10143 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.287729  4720 leader_election.cc:400] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17.
I20260709 12:27:02.292760  4720 leader_election.cc:403] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:02.293794  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:27:02.212635 10342 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:02.162757 10349 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.125s
I20260709 12:27:02.300225  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:02.162473 10129 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.125s
I20260709 12:27:02.306252 10248 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:02.306865 10248 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:02.308107 10248 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.309875  9740 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:02.275967  9653 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:02.313099 10097 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.287122 10347 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:02.313841 10097 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.296604 10298 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:27:02.315600  4602 raft_consensus.cc:1275] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:27:02.315999  4602 pending_rounds.cc:85] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 11
I20260709 12:27:02.316211  4602 pending_rounds.cc:107] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:27:02.316262  9984 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.317368 10200 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.317026  9984 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.317909 10200 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.319664  9984 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:02.319895 10200 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:02.321918 10248 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:02.324420  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:02.325294  4680 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 15: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:02.316448  4602 raft_consensus.cc:2889] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:02.328466  4681 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:02.200225 10320 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.106s
I20260709 12:27:02.369066  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:02.288582 10309 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:02.174674 10192 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.334981  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.369803  4681 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:02.394904  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 56534.1us
W20260709 12:27:02.373071  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 208231us
I20260709 12:27:02.347738 10200 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:02.329455 10206 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:02.411214  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:02.397714  4647 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:02.415174  4681 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:02.407011  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:02.415627  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:02.415894  4680 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:02.416265  4605 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:27:02.417657  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:02.419417  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.423311  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.426512  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.399242  9740 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.418712  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:02.431196  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:02.397010  9984 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.437793  9776 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:02.439859  4680 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:02.434190  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:02.429203  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:02.441138  4605 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:02.442657  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.453320  4680 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:02.392287 10325 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:02.430215 10229 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:02.444202  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:02.444797  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:02.458411  4756 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:02.459318  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:02.459800  4756 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:02.447207  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:27:02.447508 10374 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:02.448251  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:02.461808  4755 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:02.448526  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:02.449077  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.449769  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:02.422358 10192 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:02.450197  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:02.435822  9653 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.456329  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.457327  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:02.474028  4747 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:02.435221 10129 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.463420  4751 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:02.449925  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:02.476163  4750 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:02.441601 10206 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:02.432575 10330 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:02.454437  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.470873  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:02.452023 10284 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:02.465857  4673 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:02.414007 10309 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.457038 10297 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.450855  9984 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.477721 10200 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:02.499191  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:02.528776  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:02.529374  4681 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:02.509205  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:02.453730 10323 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:02.512437  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.509418  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:02.451551  9951 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:27:02.481341  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.505162 10284 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:02.548244  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:02.443171  9740 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.476976 10229 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:02.481789 10192 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:02.559844  4681 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
W20260709 12:27:02.528905 10154 consensus_meta.cc:287] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.597s	user 0.008s	sys 0.002s
I20260709 12:27:02.533789 10297 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.524104  9961 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.554908  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.555980  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.572886  4677 raft_consensus.cc:2376] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13. Re-sending same reply.
I20260709 12:27:02.518630 10347 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.560876  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:02.477942 10301 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:02.534265 10389 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.564101  9776 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:02.600900  9776 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.480887  9653 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.493280 10129 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.551590 10284 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:02.574270  4681 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:27:02.469419 10325 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.554893 10330 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:02.511886 10206 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.562908 10192 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.563426  9951 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:02.616631 10389 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.578086 10363 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:02.625257  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.625283  9776 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:02.589761 10359 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:02.570851 10154 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.671556 10154 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:02.673283 10345 log.cc:927] Time spent T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.229s	user 0.006s	sys 0.008s
I20260709 12:27:02.675374 10297 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:02.675490  4601 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:27:02.678604  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3368) took 1617 ms. Trace:
I20260709 12:27:02.679152  4676 rpcz_store.cc:276] 0709 12:27:01.060768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.062645 (+  1877us) service_pool.cc:224] Handling call
0709 12:27:01.062942 (+   297us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:01.063314 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:01.063341 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:01.063926 (+   585us) log.cc:844] Serialized 52 byte log entry
0709 12:27:01.169766 (+105840us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:01.239280 (+ 69514us) log.cc:844] Serialized 14 byte log entry
0709 12:27:01.239496 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.239579 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:02.036694 (+797115us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b1800246fa0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c9522 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:27:02.672698 (+636004us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:27:02.678279 (+  5581us) raft_consensus.cc:1685] finished
0709 12:27:02.678311 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.678520 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54076,"prepare.queue_time_us":115258,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":52868,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":7560,"spinlock_wait_cycles":62755712,"thread_start_us":105426,"threads_started":1,"wal-append.queue_time_us":375551}
I20260709 12:27:02.681910 10363 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:02.682368  9653 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:02.682461 10363 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.684739 10363 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:02.576860  9961 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.686908  9961 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:02.688977  4601 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:02.720276  4601 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 13. Current term is 14. Ops: [11.9-13.10]
I20260709 12:27:02.668918 10389 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:02.691033  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:02.725745  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:02.637665 10229 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.726593  4751 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:27:02.727798  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:02.728461  4751 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:02.730002 10182 raft_consensus.cc:3055] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:02.730412 10182 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:02.731238 10182 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.732216 10182 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:02.734272 10309 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:02.734683 10358 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.989s	user 0.005s	sys 0.007s
I20260709 12:27:02.735164 10129 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:02.702075  9776 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:02.735982  9776 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.702207  9984 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:02.703619  9740 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:02.708375  4568 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:02.709141 10351 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.709770 10322 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.640738 10330 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:02.710592  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:02.772636  5345 raft_consensus.cc:2368] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17.
I20260709 12:27:02.711522  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:02.712015 10391 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:02.712399  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:27:02.713003 10392 log.cc:927] Time spent T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:27:02.713349  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:02.714936  9772 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:02.662941 10192 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:02.715052 10325 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:02.717744 10353 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:27:02.667969 10384 log.cc:927] Time spent T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:27:02.721557  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:02.722677 10368 log.cc:927] Time spent T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.978s	user 0.006s	sys 0.000s
I20260709 12:27:02.737362  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:02.613430 10143 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:02.639941 10386 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:27:02.757252  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:02.550304 10200 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.742105  5341 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:02.577198 10347 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:02.637967 10377 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 12:27:02.737371  5349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 766) took 1788 ms. Trace:
I20260709 12:27:02.769945  9776 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:02.710152  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:02.659376 10331 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:02.760471 10349 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:27:02.783645  4755 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:02.656334 10388 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:02.836978 10388 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.785666  5342 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:02.839152 10388 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:02.840003  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:02.841197  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:02.798763  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:02.841830  5345 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:02.844097  4755 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 13. Current term is 14. Ops: [11.9-13.10]
I20260709 12:27:02.846858  5341 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:27:02.615882 10301 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.800932  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:02.756565  9983 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:02.849447 10301 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:02.849246  9983 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.760546 10363 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.805401  4751 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:27:02.806456  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:02.852403  5347 raft_consensus.cc:2393] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 15: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:02.808120  4677 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16. Ops: [14.9-15.10]
W20260709 12:27:02.813941  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 304622us
I20260709 12:27:02.738015  9951 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.817945  5349 rpcz_store.cc:276] 0709 12:27:00.948810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.968869 (+ 20059us) service_pool.cc:224] Handling call
0709 12:27:00.969235 (+   366us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:00.970153 (+   918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.970180 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:00.970707 (+   527us) log.cc:844] Serialized 52 byte log entry
0709 12:27:01.874894 (+904187us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b480027c258. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff33dfaa1 00007efff33d6b37 00007efff33d69ca 00007efff34ce26f 00007efff34cdf37 00007efff34e255c 00007efff34e2508 00007efff34e2488 00007efff34e2443 00007efff34e16ec 00007efff240ba96
0709 12:27:02.707497 (+832603us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b1800080920. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effee05e4bc 00007effeb3ac48b 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1
0709 12:27:02.707729 (+   232us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:02.707783 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.707897 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:02.733977 (+ 26080us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:27:02.737082 (+  3105us) raft_consensus.cc:1685] finished
0709 12:27:02.737104 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.737293 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43768,"raft.queue_time_us":15773,"raft.run_cpu_time_us":5084,"raft.run_wall_time_us":892180,"spinlock_wait_cycles":63046272,"thread_start_us":656052,"threads_started":1,"wal-append.queue_time_us":701228}
W20260709 12:27:02.805913 10373 consensus_meta.cc:287] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.000s
I20260709 12:27:02.776948 10391 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:02.830953  4756 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:02.867019  4748 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:27:02.775941 10351 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.867933  4756 raft_consensus.cc:1240] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:02.861354  5345 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:02.868228  4754 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:27:02.868642  4748 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:27:02.817307 10298 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17.
I20260709 12:27:02.882032  4754 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:02.873973  4748 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:27:02.776152 10322 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.883939  4754 raft_consensus.cc:1240] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:02.806893 10353 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.890552  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:02.891381  4754 raft_consensus.cc:3060] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:02.862488 10388 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.918895  4751 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:27:02.862291 10301 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:02.868965 10391 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:02.928208  5348 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:02.936522  4754 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:02.943918  4752 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:02.951717  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:02.874219 10330 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.961098 10330 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:02.963008  4719 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:27:02.963927  9946 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:27:02.964509  9946 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:02.964881  9946 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.965225  9946 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:02.965942  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:02.966820 10132 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.967206  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:02.967336 10132 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.967792  4750 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 12.
I20260709 12:27:02.968590  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:02.979619 10104 raft_consensus.cc:2764] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 12:27:02.981057 10385 log.cc:927] Time spent T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.487s	user 0.000s	sys 0.000s
I20260709 12:27:02.969413 10132 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:02.986534 10132 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:02.986716 10352 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 1.255s	user 0.000s	sys 0.008s
I20260709 12:27:02.988396  5344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 769) took 1887 ms. Trace:
I20260709 12:27:02.988749  5344 rpcz_store.cc:276] 0709 12:27:01.101034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.101177 (+   143us) service_pool.cc:224] Handling call
0709 12:27:01.114524 (+ 13347us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:01.120068 (+  5544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:01.120112 (+    44us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:01.137365 (+ 17253us) log.cc:844] Serialized 78 byte log entry
0709 12:27:01.492107 (+354742us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:01.492145 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.492212 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:02.986137 (+1493925us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:27:02.988080 (+  1943us) raft_consensus.cc:1685] finished
0709 12:27:02.988101 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.988334 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4932,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2787,"spinlock_wait_cycles":39384832,"thread_start_us":254786,"threads_started":1,"wal-append.queue_time_us":522973}
I20260709 12:27:02.990381  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 2128) took 1205 ms. Trace:
I20260709 12:27:02.990826  4604 rpcz_store.cc:276] 0709 12:27:01.784960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.789280 (+  4320us) service_pool.cc:224] Handling call
0709 12:27:01.789618 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.155468 (+365850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.990122 (+834654us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b5400141ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:02.990313 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":790771584}
I20260709 12:27:02.991076 10132 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:02.992692  4604 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:27:02.993186  4749 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:02.994017  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2780) took 1116 ms. Trace:
I20260709 12:27:02.994506  4597 rpcz_store.cc:276] 0709 12:27:01.877355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.879678 (+  2323us) service_pool.cc:224] Handling call
0709 12:27:01.893931 (+ 14253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.107718 (+213787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.993765 (+886047us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b540013ecd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:02.993951 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417227648}
I20260709 12:27:02.996169  4597 raft_consensus.cc:1240] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:27:02.851483 10331 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.866370 10363 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.002705  4719 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:03.009083  4719 leader_election.cc:400] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17.
I20260709 12:27:03.010251  4719 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:02.921808 10206 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:03.014873  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:03.014988  5348 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
W20260709 12:27:03.016435  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 214849us
I20260709 12:27:03.025254  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:03.026485  4750 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:03.027025  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:03.027704  5345 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:27:03.017477  4755 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:27:03.122068  4597 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:27:03.135277 10132 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:03.136191  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2792) took 1253 ms. Trace:
I20260709 12:27:03.136597  4596 rpcz_store.cc:276] 0709 12:27:01.882922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.551120 (+668198us) service_pool.cc:224] Handling call
0709 12:27:02.568275 (+ 17155us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.568597 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.568638 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:02.569286 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 12:27:02.875485 (+306199us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:02.875549 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.875653 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.135910 (+260257us) raft_consensus.cc:1685] finished
0709 12:27:03.135943 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.136122 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":284488,"threads_started":1,"wal-append.queue_time_us":559162}
I20260709 12:27:03.028781 10276 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.109s
I20260709 12:27:03.138070  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2795) took 1001 ms. Trace:
I20260709 12:27:03.138633  4601 rpcz_store.cc:276] 0709 12:27:02.136648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.738798 (+602150us) service_pool.cc:224] Handling call
0709 12:27:02.776498 (+ 37700us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.776801 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.776841 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:02.777557 (+   716us) log.cc:844] Serialized 26 byte log entry
0709 12:27:02.945012 (+167455us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:02.945076 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.945180 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.137877 (+192697us) raft_consensus.cc:1685] finished
0709 12:27:03.137895 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.138015 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":167054,"threads_started":1,"wal-append.queue_time_us":357613}
I20260709 12:27:02.934060 10301 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.145468 10104 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.146185 10104 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.151093 10200 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:03.015604 10325 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:02.924321 10388 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:03.166484  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.179237  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:03.194162  4597 raft_consensus.cc:1240] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:27:03.026839  9740 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.029070 10206 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:27:03.201087  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.030848  9983 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:03.204368 10363 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:03.202494  9740 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:03.209376  9740 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:03.210134 10401 log.cc:927] Time spent T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 12:27:03.211323  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 2142) took 1041 ms. Trace:
I20260709 12:27:03.211707  4680 rpcz_store.cc:276] 0709 12:27:02.170090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.602956 (+432866us) service_pool.cc:224] Handling call
0709 12:27:02.642514 (+ 39558us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.642996 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.643036 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:02.655128 (+ 12092us) log.cc:844] Serialized 26 byte log entry
0709 12:27:02.851866 (+196738us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:02.851906 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.851977 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.211127 (+359150us) raft_consensus.cc:1685] finished
0709 12:27:03.211146 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.211267 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10241,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":9558,"thread_start_us":196388,"threads_started":1,"wal-append.queue_time_us":293853}
W20260709 12:27:03.215534  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 478238us
I20260709 12:27:03.216908  4645 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:03.213002 10391 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:02.004534  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:03.303705  4500 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 164) took 49349 ms (client timeout 15000 ms). Trace:
I20260709 12:27:03.283226  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:03.326260  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:03.327672  4679 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:27:03.280017 10204 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.112s	user 0.014s	sys 0.000s
I20260709 12:27:03.275909 10351 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:03.279040 10347 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:03.276201 10353 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:03.305171 10406 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:27:03.332371  4722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.010s	sys 0.034s
I20260709 12:27:03.278879  9951 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:03.342000  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.310577  9990 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:03.294087 10104 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:03.293704  9983 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.317445 10369 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:03.360982  4722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.010s	sys 0.037s
I20260709 12:27:03.314103 10388 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:03.319042 10322 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:03.325979  9772 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:03.332880 10129 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.011s	sys 0.032s
I20260709 12:27:03.414326  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:03.410602  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:03.361011 10351 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:03.414327 10324 consensus_meta.cc:287] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 1.261s	user 0.010s	sys 0.003s
I20260709 12:27:03.440577  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:03.441457  4751 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
W20260709 12:27:03.443918  4647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.045s	sys 0.056s
I20260709 12:27:03.383430 10349 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:03.472689  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:03.474608  4647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.045s	sys 0.056s
W20260709 12:27:03.441398 10248 consensus_meta.cc:287] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 1.119s	user 0.013s	sys 0.001s
I20260709 12:27:03.475448  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:03.479216  4752 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
W20260709 12:27:03.323005  4500 rpcz_store.cc:269] 0709 12:26:13.954140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.954352 (+   212us) service_pool.cc:224] Handling call
0709 12:27:03.303571 (+49349219us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:57.258313 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:57.267492 (+  9179us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:57.267547 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:26:57.267569 (+    22us) tablet.cc:662] Decoding operations
0709 12:26:57.268255 (+   686us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:57.268356 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:57.268389 (+    33us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:26:57.268885 (+   496us) tablet.cc:701] Row locks acquired
0709 12:26:57.268936 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:26:57.269124 (+   188us) write_op.cc:270] Start()
0709 12:26:57.269288 (+   164us) write_op.cc:276] Timestamp: P: 1783600017269058 usec, L: 0
0709 12:26:57.269328 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 12:26:57.312843 (+ 43515us) log.cc:844] Serialized 1674 byte log entry
0709 12:26:57.685226 (+372383us) op_driver.cc:464] REPLICATION: finished
0709 12:26:57.797623 (+112397us) write_op.cc:301] APPLY: starting
0709 12:26:57.803496 (+  5873us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:57.873088 (+ 69592us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:57.873274 (+   186us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:57.988822 (+115548us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:57.989159 (+   337us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:26:57.989233 (+    74us) write_op.cc:312] APPLY: finished
0709 12:26:57.999802 (+ 10569us) log.cc:844] Serialized 49 byte log entry
0709 12:26:57.999987 (+   185us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:58.000104 (+   117us) write_op.cc:454] Released schema lock
0709 12:26:58.000157 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:58.000776 (+   619us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:58.000814 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":299762,"reported_tablets":26,"spinlock_wait_cycles":2944,"child_traces":[["op",{"apply.queue_time_us":90102,"apply.run_cpu_time_us":16407,"apply.run_wall_time_us":216644,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20823,"num_ops":4,"prepare.queue_time_us":210347,"prepare.run_cpu_time_us":26136,"prepare.run_wall_time_us":300201,"raft.queue_time_us":116025,"raft.run_cpu_time_us":2078,"raft.run_wall_time_us":9619,"replication_time_us":415720,"thread_start_us":292441,"threads_started":4,"wal-append.queue_time_us":207353,"wal-append.run_cpu_time_us":8573,"wal-append.run_wall_time_us":1456311}]]}
W20260709 12:27:03.477021  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.440876  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 224056us
W20260709 12:27:03.474566 10284 consensus_meta.cc:287] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.821s	user 0.013s	sys 0.004s
W20260709 12:27:03.488953  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.489488  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.490283  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:27:03.491204  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 327299us
I20260709 12:27:03.487653  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:03.345690 10204 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.494819  5314 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:03.486887  4645 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:03.476168  9951 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:03.411551  9983 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:03.415813  9772 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.507548  5314 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:03.509210  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:03.482842 10415 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.512686  4751 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
I20260709 12:27:03.521633  4600 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:27:03.484979 10369 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:27:03.507853 10402 raft_consensus.cc:2764] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:27:03.299062  4495 catalog_manager.cc:5697] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 13 to 14, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 14 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:03.512352  9740 raft_consensus.cc:2804] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:03.481223 10349 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.506878  9951 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:03.507336  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:03.525280  5314 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:03.517704 10373 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:03.506453 10248 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:03.526515 10407 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:27:03.535372  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:03.559602  4751 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:03.488245 10351 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.511138 10276 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:27:03.505635 10284 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:03.540297  9946 consensus_meta.cc:287] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.004s
I20260709 12:27:03.546371  9951 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:03.547065 10152 raft_consensus.cc:3055] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:03.520059 10415 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:03.571898  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63021.3us
W20260709 12:27:03.569161  4600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 2122) took 1830 ms (client timeout 1935 ms). Trace:
I20260709 12:27:03.567896  9946 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:03.569958 10204 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:03.575465  9983 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:03.576602 10351 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:03.615290  4600 rpcz_store.cc:269] 0709 12:27:01.739316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.739463 (+   147us) service_pool.cc:224] Handling call
0709 12:27:03.568995 (+1829532us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b5400034154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:03.569100 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3977123584}
W20260709 12:27:03.576871 10410 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:27:03.557736 10373 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:03.577705 10405 log.cc:927] Time spent T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 12:27:03.578344 10403 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:27:03.579234  5314 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:03.582923  4495 catalog_manager.cc:5697] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 13 to 16, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), pending config changed. New cstate: current_term: 16 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } } pending_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:03.587049  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 900) took 1392 ms. Trace:
I20260709 12:27:03.606567  4677 raft_consensus.cc:1240] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:03.607293  4674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 2136) took 1469 ms. Trace:
I20260709 12:27:03.628970  4681 rpcz_store.cc:276] 0709 12:27:02.194298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.658345 (+464047us) service_pool.cc:224] Handling call
0709 12:27:02.658668 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.885994 (+227326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.886024 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:02.887754 (+  1730us) log.cc:844] Serialized 26 byte log entry
0709 12:27:03.262182 (+374428us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:03.262278 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.262413 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.586775 (+324362us) raft_consensus.cc:1685] finished
0709 12:27:03.586798 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.586981 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5595,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":96,"thread_start_us":600809,"threads_started":2,"wal-append.queue_time_us":646796}
W20260709 12:27:03.565600 10330 consensus_meta.cc:287] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.604s	user 0.006s	sys 0.006s
I20260709 12:27:03.616631 10322 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:03.635143  4495 catalog_manager.cc:5697] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:03.629186  4674 rpcz_store.cc:276] 0709 12:27:02.137611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.139072 (+  1461us) service_pool.cc:224] Handling call
0709 12:27:02.139353 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.139775 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.139815 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:02.141641 (+  1826us) log.cc:844] Serialized 26 byte log entry
0709 12:27:02.736069 (+594428us) spinlock_profiling.cc:243] Waited 8.82 ms on lock 0x7b18003d5a00. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:27:02.939038 (+202969us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:02.939096 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.939190 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.591231 (+652041us) raft_consensus.cc:1685] finished
0709 12:27:03.591254 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.607202 (+ 15948us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10078,"raft.queue_time_us":106,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":5652,"spinlock_wait_cycles":20291840,"thread_start_us":202749,"threads_started":1,"wal-append.queue_time_us":1219035}
W20260709 12:27:03.639181  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.639266 10363 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:27:03.620645  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 46777.7us
I20260709 12:27:03.590647 10152 raft_consensus.cc:740] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:03.590325 10391 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:03.620643  9776 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:03.642213  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:03.643050  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:03.656682  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:03.637318  4495 catalog_manager.cc:5697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:03.645637  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:03.617475 10412 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:27:03.650053  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:03.618978  9740 raft_consensus.cc:697] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:03.622999 10373 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:03.668452  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:03.651854  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2790) took 1769 ms. Trace:
I20260709 12:27:03.652987  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:03.659277  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:03.660480  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:03.660841  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:03.674798  4674 raft_consensus.cc:2410] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:03.648702 10331 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:03.670291  4598 rpcz_store.cc:276] 0709 12:27:01.882402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.284335 (+401933us) service_pool.cc:224] Handling call
0709 12:27:02.284670 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.284906 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:02.285274 (+   368us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.523310 (+238036us) log.cc:844] Serialized 14 byte log entry
0709 12:27:03.651328 (+1128018us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b18004398a0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007effeb3cba40 00007effeb3c64e4 00007effeb3c6455 00007effee4ae060 00007effee4c0716 00007effee4d1287 00007effee4d1208
0709 12:27:03.651473 (+   145us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:03.651522 (+    49us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:03.651568 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.651631 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.651782 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1126346,"prepare.queue_time_us":7381,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":5733,"raft.queue_time_us":1161700,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":11850,"spinlock_wait_cycles":12498176,"thread_start_us":221207,"threads_started":1,"wal-append.queue_time_us":282540}
I20260709 12:27:03.639418 10097 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:03.673808  4679 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:27:03.684224  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1039) took 1214 ms. Trace:
I20260709 12:27:03.684607  4753 rpcz_store.cc:276] 0709 12:27:02.469658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.474882 (+  5224us) service_pool.cc:224] Handling call
0709 12:27:02.513010 (+ 38128us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.551960 (+ 38950us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:02.576826 (+ 24866us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.978807 (+401981us) spinlock_profiling.cc:243] Waited 5.5 ms on lock 0x7b18003dd1a0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c9522 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:27:03.011191 (+ 32384us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:03.018636 (+  7445us) log.cc:844] Serialized 26 byte log entry
0709 12:27:03.018820 (+   184us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:03.018856 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.018947 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.669825 (+650878us) raft_consensus.cc:1685] finished
0709 12:27:03.684007 (+ 14182us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.684162 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73390,"prepare.queue_time_us":17895,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":90937,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":9306,"spinlock_wait_cycles":29678592,"thread_start_us":385587,"threads_started":2,"wal-append.queue_time_us":707855}
I20260709 12:27:03.686043  4674 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:03.686671  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:03.632150 10409 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:27:03.633258 10402 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:27:03.661130  9983 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.690400  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 915) took 3314 ms. Trace:
I20260709 12:27:03.705034  4602 rpcz_store.cc:276] 0709 12:27:00.368001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.870503 (+502502us) service_pool.cc:224] Handling call
0709 12:27:00.908878 (+ 38375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.326287 (+1417409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.682160 (+1355873us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400344254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:03.682404 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3234319232}
I20260709 12:27:03.692965  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2793) took 1705 ms. Trace:
I20260709 12:27:03.661087 10204 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:03.706296  4605 rpcz_store.cc:276] 0709 12:27:01.987855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.549300 (+561445us) service_pool.cc:224] Handling call
0709 12:27:02.564111 (+ 14811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.564394 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:02.586152 (+ 21758us) log.cc:844] Serialized 14 byte log entry
0709 12:27:03.692425 (+1106273us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:03.692536 (+   111us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:03.692607 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.692705 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.692874 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":871359,"prepare.queue_time_us":24135,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":873970,"thread_start_us":234448,"threads_started":1,"wal-append.queue_time_us":336220}
W20260709 12:27:03.707773  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 200525us
I20260709 12:27:03.712781  9776 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:03.713805  4679 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:03.714412  5348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 803) took 1008 ms. Trace:
I20260709 12:27:03.715013  5348 rpcz_store.cc:276] 0709 12:27:02.705506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.709754 (+  4248us) service_pool.cc:224] Handling call
0709 12:27:03.714239 (+1004485us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54008d7cd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:03.714353 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249281920}
I20260709 12:27:03.723369  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2800) took 1397 ms. Trace:
I20260709 12:27:03.704610 10229 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.723989  4604 rpcz_store.cc:276] 0709 12:27:02.326108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.998406 (+672298us) service_pool.cc:224] Handling call
0709 12:27:02.998703 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.998866 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:02.999325 (+   459us) log.cc:844] Serialized 14 byte log entry
0709 12:27:03.722876 (+723551us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:03.722952 (+    76us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:03.723028 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.723124 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.723288 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":361377,"prepare.queue_time_us":154,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"thread_start_us":359863,"threads_started":1,"wal-append.queue_time_us":514793}
I20260709 12:27:03.710556  9772 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:03.739437  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 1.03109e+06us
I20260709 12:27:03.742182  4568 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:03.731554 10402 raft_consensus.cc:3055] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:03.727770  9735 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.735950 10363 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:27:03.713339 10204 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.775071  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.776036  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.759483  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:03.745383 10372 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.764427 10363 consensus_queue.cc:1243] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:03.758762 10347 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.750702 10349 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:03.768659  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.770133 10323 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.771755  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.709681  9983 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.775219  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:03.809437  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 99868.6us
I20260709 12:27:03.758337 10402 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:03.738327 10229 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.741016  9653 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:03.783246  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.706844 10152 consensus_queue.cc:260] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:03.799831  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.800050 10009 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.802603  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:03.767944 10346 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:03.806821  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.747740 10284 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:03.831375  4601 raft_consensus.cc:2393] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:03.810091  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:03.776424  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:03.819766  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 60394us
I20260709 12:27:03.804978  9946 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:03.824018  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.835887  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:03.825253  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:03.837008  4598 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:03.827574  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.838071  4600 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:03.822468  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.833658  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:03.804776 10415 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:03.827579  9882 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.740383  9740 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.782233 10191 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:27:03.774299  9735 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.837778  4569 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:03.842370  4674 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 12:27:03.827653  9961 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:03.843326  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:03.843497  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 99735.1us
I20260709 12:27:03.846716  4674 pending_rounds.cc:85] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 12
I20260709 12:27:03.847155  4674 pending_rounds.cc:107] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 12:27:03.847348  4680 raft_consensus.cc:2410] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:27:03.847426  4674 raft_consensus.cc:2889] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:03.827782 10154 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.826768 10230 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:03.850497  4674 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:27:03.851008  4674 pending_rounds.cc:85] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 6
I20260709 12:27:03.851294  4674 pending_rounds.cc:107] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:27:03.851599  4674 raft_consensus.cc:2889] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:03.854343  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.855391  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.855953  4598 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:27:03.856593  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.857050  4598 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:03.839049 10152 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:03.861490  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.862636  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.805732 10372 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:03.859547  4571 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:03.857872  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.864607  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:03.865499  9983 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:03.865693 10229 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:03.858204  9961 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.759261  4495 catalog_manager.cc:5697] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:03.819110 10323 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:03.810010 10347 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.853549 10230 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:03.871527  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:03.872771  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:03.884464  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:03.834187 10009 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.884902  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:03.886436  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:03.845342 10364 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:03.915589  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:03.875880 10438 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:03.917764 10438 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:03.918881 10438 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:03.920954 10009 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:03.922006  4605 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:27:03.840777 10346 raft_consensus.cc:697] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:03.932245 10346 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.870596  4569 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:03.934144  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 63607.1us
I20260709 12:27:03.844957 10284 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.936455 10284 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.938884 10360 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:03.939424 10360 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.939833 10360 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:03.876272  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.942807 10439 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:03.943317 10439 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:03.943725 10439 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:03.860922 10154 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:03.947398 10301 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:03.905107  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.875312  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:03.833528 10409 consensus_queue.cc:1175] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5c3201048f8346009aefca07428828f3" responder_term: 13 status { last_received { term: 13 index: 8 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.8-12.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:03.842954  9882 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:03.873034  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.982169  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:03.982733  4598 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:03.888792 10229 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:03.986752 10229 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:03.889708 10171 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.886801 10445 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:27:03.887782  4571 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:03.890159  9961 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:03.890314 10437 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:03.890499 10412 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:03.890825  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 2133) took 1791 ms. Trace:
I20260709 12:27:03.890065 10099 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:03.892019  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:03.892714 10442 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:03.899479 10323 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:03.898957  4495 catalog_manager.cc:5697] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 11 to 12, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 12 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:03.875897 10324 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:03.907050 10204 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:03.909617 10100 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:03.915705 10251 raft_consensus.cc:3055] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:03.860694 10402 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:03.879050 10373 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:03.951563  4604 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:03.971506  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:03.973918  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:03.874325  9653 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:03.977870  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:03.979830  9776 consensus_queue.cc:1175] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49277a7653524787a715232ab908bbc6" responder_term: 13 status { last_received { term: 10 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: [12.8-12.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:03.858103 10149 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:27:03.888783 10230 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:03.984898  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:03.985186  9735 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:04.030404 10284 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:03.987213 10229 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:03.990537  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 102843us
I20260709 12:27:03.994355  4598 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:04.060729  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:03.981750 10347 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:04.009414  4603 rpcz_store.cc:276] 0709 12:27:02.099572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.714320 (+614748us) service_pool.cc:224] Handling call
0709 12:27:02.714631 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:03.691361 (+976730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.691413 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:03.692099 (+   686us) log.cc:844] Serialized 26 byte log entry
0709 12:27:03.763861 (+ 71762us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:03.763920 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.764016 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.890589 (+126573us) raft_consensus.cc:1685] finished
0709 12:27:03.890609 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.890752 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1009249,"raft.queue_time_us":977367,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":10543,"thread_start_us":58949,"threads_started":1,"wal-append.queue_time_us":128094}
W20260709 12:27:03.987646 10182 consensus_meta.cc:287] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.255s	user 0.016s	sys 0.000s
I20260709 12:27:03.975159 10301 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:03.874169 10443 consensus_queue.cc:1059] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.144s
W20260709 12:27:04.007356 10192 consensus_meta.cc:287] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 1.201s	user 0.005s	sys 0.008s
I20260709 12:27:04.011960 10099 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:04.022328  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 130409us
I20260709 12:27:04.008978 10364 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:04.012606 10412 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.032090  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:04.006716 10171 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:04.037082  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 87275.4us
I20260709 12:27:04.043707  4673 raft_consensus.cc:2368] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17.
I20260709 12:27:04.021934 10324 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:04.047600  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:04.077131  9990 consensus_meta.cc:287] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.695s	user 0.013s	sys 0.000s
W20260709 12:27:04.049146  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 77715.6us
I20260709 12:27:04.044059  9984 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:04.058907 10204 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:04.057641  9951 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.058213 10373 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.058490  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3394) took 1840 ms. Trace:
I20260709 12:27:04.032169 10442 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:04.127812 10442 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:04.128263 10442 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:04.064263  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.064824  9882 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.099481  4675 rpcz_store.cc:276] 0709 12:27:02.218070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.717945 (+499875us) service_pool.cc:224] Handling call
0709 12:27:02.718373 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.934515 (+216142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.934546 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.934594 (+    48us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:02.934633 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.058250 (+1123617us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54001d2ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:04.058299 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.058422 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496544640}
I20260709 12:27:04.130802  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:04.131676  4673 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:27:04.133633  4605 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:04.135708 10248 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:04.070477  4598 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:27:04.070315  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:04.145426  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 75226.4us
I20260709 12:27:04.146111  4598 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:04.077800  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:27:04.147282  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 69571us
I20260709 12:27:04.147414  4598 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:04.148206  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 902) took 1570 ms. Trace:
I20260709 12:27:04.148618  4677 rpcz_store.cc:276] 0709 12:27:02.578214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.854729 (+276515us) service_pool.cc:224] Handling call
0709 12:27:02.855098 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.632756 (+777658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.148014 (+515258us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b540017e654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:04.148153 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1719247616}
I20260709 12:27:04.079450  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:04.150025  4603 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:04.150933  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:04.077189  4571 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:04.082471  4601 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:04.044561 10154 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.091265  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:04.031543 10251 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:27:04.155495  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 78391us
I20260709 12:27:04.059309 10400 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:04.078938 10099 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:04.080202 10402 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:04.095806  4681 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 21.
I20260709 12:27:04.083410 10412 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:04.099264  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:04.150560 10261 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:04.227242  4602 raft_consensus.cc:2393] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 9: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:04.145052 10248 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.106011 10171 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.106977 10322 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:04.089979 10364 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:04.110291  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:04.111012  9951 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:04.094630  9984 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.114017  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 2134) took 1980 ms. Trace:
I20260709 12:27:04.235400 10248 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:04.114989  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:04.078925 10301 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.117141 10279 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.117816  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.117969 10439 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.119154  4753 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:04.119630  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.120822 10360 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.092705 10324 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:04.126202  4495 catalog_manager.cc:5697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:04.143095  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.143396  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.273944  4748 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
W20260709 12:27:04.076098  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 140262us
W20260709 12:27:04.144686  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 565521us
I20260709 12:27:04.153846  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:04.278683  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 124959us
I20260709 12:27:04.149169  4647 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:04.154851  4603 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:04.157734  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:04.166535  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 75334us
I20260709 12:27:04.190932 10143 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:27:04.147213 10449 log.cc:927] Time spent T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.105s	user 0.002s	sys 0.000s
I20260709 12:27:04.158855  4719 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:04.199826  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:04.193504 10453 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:04.201854  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.290829  4751 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:27:04.193827 10152 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.196566 10276 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:04.211288  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:04.212087  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:04.212739  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:04.293990  5346 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:04.213411  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:04.295735  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 136926us
I20260709 12:27:04.213662 10330 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.297554 10330 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.298416  4495 catalog_manager.cc:5697] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:04.213929 10383 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.269843 10372 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.216115  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:04.216606 10149 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.216998 10351 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.193153  9990 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:27:04.218725  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:04.223275  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:04.446489  5341 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:04.447345  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:04.447690  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:04.448908  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:04.224242  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:04.449435  4644 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:04.224845 10099 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.141947 10204 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:04.225621  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2793) took 1462 ms. Trace:
I20260709 12:27:04.225795  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:04.453962  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:04.452936  4756 rpcz_store.cc:276] 0709 12:27:02.763005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.871768 (+108763us) service_pool.cc:224] Handling call
0709 12:27:02.872159 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:02.886077 (+ 13918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:02.886109 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:02.893346 (+  7237us) log.cc:844] Serialized 26 byte log entry
0709 12:27:03.876202 (+982856us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b18002beca0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:27:04.018277 (+142075us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:04.018338 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.018431 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:04.225274 (+206843us) raft_consensus.cc:1685] finished
0709 12:27:04.225299 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.225541 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11783,"raft.queue_time_us":14399,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":41455,"spinlock_wait_cycles":4072704,"thread_start_us":141837,"threads_started":1,"wal-append.queue_time_us":1183806}
I20260709 12:27:04.455004  5348 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:04.228421  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:04.455170  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.078724 10192 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.236606  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3399) took 1769 ms. Trace:
I20260709 12:27:04.193598 10391 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:04.191380 10221 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:04.239341  4596 raft_consensus.cc:2376] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10. Re-sending same reply.
I20260709 12:27:04.245651  4599 rpcz_store.cc:276] 0709 12:27:02.133272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.734844 (+601572us) service_pool.cc:224] Handling call
0709 12:27:02.735209 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:03.695135 (+959926us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.695163 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:03.695588 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 12:27:03.760248 (+ 64660us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:03.760302 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.113542 (+353240us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400351bd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:04.113630 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:04.113704 (+    74us) raft_consensus.cc:1685] finished
0709 12:27:04.113720 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.113952 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":859641,"raft.queue_time_us":841440,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":2609,"spinlock_wait_cycles":295459712,"thread_start_us":50743,"threads_started":1,"wal-append.queue_time_us":132506}
I20260709 12:27:04.456419  4676 rpcz_store.cc:276] 0709 12:27:02.467462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.744953 (+277491us) service_pool.cc:224] Handling call
0709 12:27:02.745283 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.998808 (+253525us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:03.486123 (+487315us) log.cc:844] Serialized 14 byte log entry
0709 12:27:03.625687 (+139564us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:03.625764 (+    77us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:03.625823 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.236397 (+610574us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400084654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:04.236440 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.236552 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7760,"prepare.queue_time_us":575546,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":10915,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":1856,"spinlock_wait_cycles":582561024,"thread_start_us":617982,"threads_started":2,"wal-append.queue_time_us":147276}
I20260709 12:27:04.250938  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.262984  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:04.458583  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:04.459012  5347 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:04.459210  5344 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:27:04.300653  4495 catalog_manager.cc:5697] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 12 to 13, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 13 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:04.471798  4750 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 16: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:27:04.472010  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 253335us
I20260709 12:27:04.227439 10251 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.465184  4719 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:04.196193 10400 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.246706 10261 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.273247  4754 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:04.283861  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:04.284466  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:04.285509  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 136417us
I20260709 12:27:04.479877  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.286051  9776 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:04.480898  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.287192  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:04.481302  9776 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:04.215476  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:04.481588  4747 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:04.481597  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:04.303395  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.302237  4677 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 21.
I20260709 12:27:04.306090  4673 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:27:04.482667  4748 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:27:04.482944  4753 raft_consensus.cc:2393] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 21: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:04.483497  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.481746  9776 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:04.490846  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.407480 10372 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 12:27:04.434770  4755 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:04.288602  9740 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:04.306047 10390 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:04.424430  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 118702us
W20260709 12:27:04.436225  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 134270us
I20260709 12:27:04.444731  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:04.304121 10143 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:04.449712  5343 raft_consensus.cc:2376] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16. Re-sending same reply.
I20260709 12:27:04.223340  5342 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:27:04.449437  5341 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 20.
I20260709 12:27:04.450562  4568 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:04.312054 10330 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:04.450596 10191 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:27:04.240131 10322 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:04.406337 10149 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.488359 10391 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:04.458915  4752 raft_consensus.cc:2393] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 10: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:04.306090 10152 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.315709 10383 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.466584 10439 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:04.466720 10192 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:04.467654  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:04.405997 10453 raft_consensus.cc:697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:04.470995  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.482442  5348 raft_consensus.cc:2393] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 15: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:04.527602 10360 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.306169 10276 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.493103  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.268682 10279 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.512310  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:04.469660 10418 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:04.465142 10442 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.517390  9776 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.487546 10129 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:04.546336 10099 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.111531 10230 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.459988  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:04.514855  9740 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.469640 10204 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.490710 10373 consensus_queue.cc:1048] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:27:04.530748 10390 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.433954 10364 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.487151 10349 consensus_queue.cc:1059] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 12:27:04.602314  9776 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.432605 10351 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.491532 10261 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.599406 10192 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:04.602725 10129 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:04.574904 10391 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.620038 10390 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:04.630177 10301 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:04.649204 10383 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.649434 10230 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:04.649873 10460 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 12:27:04.656802 10458 log.cc:927] Time spent T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:27:04.640981 10435 raft_consensus.cc:3055] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:04.640483 10453 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.675063 10279 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:04.664955 10324 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.839084 10324 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.723973  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:04.725852  4602 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:04.491611  4604 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:27:04.308252  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:04.526026  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:04.731487  4495 catalog_manager.cc:5697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:04.513062  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:04.738404  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 273116us
I20260709 12:27:04.738718  4748 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:27:04.556994  4747 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:27:04.528856  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 1046) took 1115 ms. Trace:
I20260709 12:27:04.746182  4753 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:04.550292  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.881970  4754 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:04.883455  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.884299  4754 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 8.
I20260709 12:27:04.536897  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:04.747244  4756 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:27:04.887166  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 374048us
I20260709 12:27:04.475733  5312 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:04.749672  5344 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:04.603091  5349 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 9.
W20260709 12:27:04.892903  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 366797us
I20260709 12:27:04.752087  5314 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:27:04.922490  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 171043us
I20260709 12:27:04.922326  4571 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:04.613757  9740 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:04.924162 10324 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:04.674158 10435 raft_consensus.cc:740] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:04.889146  4647 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:04.931391 10435 consensus_queue.cc:260] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.932569  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:04.689653 10442 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:04.683593 10462 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:27:04.673717 10469 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:04.937561  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:04.615425  9984 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:04.621511 10143 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.791208 10391 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:04.791567 10279 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:04.940037  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:04.940220 10279 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.941021  4600 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:04.942737 10353 raft_consensus.cc:3055] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:04.943256 10279 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:04.793895 10470 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:04.637931 10192 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.944592 10470 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:04.945981 10402 consensus_meta.cc:287] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.719s	user 0.005s	sys 0.014s
I20260709 12:27:04.946260 10192 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:04.807602  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 927) took 1006 ms. Trace:
I20260709 12:27:04.819525 10152 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.820966  9772 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:04.827224 10276 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.671262 10301 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:04.834236 10351 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:04.951225 10301 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:04.685156 10400 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:04.720835  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:04.638067 10129 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:04.846170  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 81491.8us
W20260709 12:27:04.810479  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 359905us
I20260709 12:27:04.852632  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:04.850895 10149 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.879467  4750 rpcz_store.cc:276] 0709 12:27:03.413962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.414416 (+   454us) service_pool.cc:224] Handling call
0709 12:27:04.528592 (+1114176us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b5400117954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:04.528773 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2292914560}
W20260709 12:27:04.891480  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 415692us
I20260709 12:27:04.599311  5348 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:04.894161 10099 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:04.851758 10472 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.965231  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:04.893855 10383 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:04.853223 10191 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:04.898468  4752 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:04.899363 10204 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:04.900224 10448 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:04.908071  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:04.913915  4644 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:04.924170  5314 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:04.947650  9772 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.951717 10132 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:04.958050 10171 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:04.962085 10445 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:04.947678 10470 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:04.962900 10276 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:27:04.964186  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 75432.6us
W20260709 12:27:04.961244 10229 consensus_meta.cc:287] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.883s	user 0.000s	sys 0.022s
I20260709 12:27:04.965116 10472 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.947075  4598 rpcz_store.cc:276] 0709 12:27:03.800759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.148262 (+347503us) service_pool.cc:224] Handling call
0709 12:27:04.148565 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.148748 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:04.285433 (+136685us) log.cc:844] Serialized 14 byte log entry
0709 12:27:04.806529 (+521096us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:04.806680 (+   151us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:04.806827 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.806973 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.807293 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75496,"prepare.queue_time_us":386910,"prepare.run_cpu_time_us":7067,"prepare.run_wall_time_us":16529,"raft.queue_time_us":83426,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":336,"thread_start_us":577072,"threads_started":2,"wal-append.queue_time_us":326732}
I20260709 12:27:04.951781 10301 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:04.981446  4601 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:04.977859  4597 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:04.962512  4568 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:04.968096 10383 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.943281 10353 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:04.986682 10383 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:04.987906  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 74476.2us
W20260709 12:27:04.988658  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 704700us
I20260709 12:27:04.989419  4644 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:04.992702 10437 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:04.993227 10437 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.982966 10276 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.994102 10439 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:04.993742 10229 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:04.995904 10439 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:04.996400 10276 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.996528 10229 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:04.970475 10448 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:04.993788 10437 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:04.998333 10170 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:04.976285 10132 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.999317 10132 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:04.999979  4598 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:04.986608  4647 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:05.001847 10152 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:05.002348 10152 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.002704 10152 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:05.019356  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 95302.2us
I20260709 12:27:05.021875  4603 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:05.022347 10320 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:05.022938 10320 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.023342 10320 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:05.028560  4673 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:27:05.029207  4673 pending_rounds.cc:85] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 7
I20260709 12:27:05.023357  4600 raft_consensus.cc:2410] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:05.029912 10472 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:04.997781 10448 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:04.997701  9772 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:05.033910  4673 pending_rounds.cc:107] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 12.8
W20260709 12:27:05.042971  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 56543.8us
I20260709 12:27:05.040577  4673 raft_consensus.cc:2889] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:04.947060 10390 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:04.860471  4495 catalog_manager.cc:5697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:05.082641  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:05.085620  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:05.085879  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 123658us
I20260709 12:27:05.086818  4568 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:05.088781  4568 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:05.089802 10482 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:05.090879 10204 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
W20260709 12:27:05.093282 10248 consensus_meta.cc:287] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.827s	user 0.009s	sys 0.007s
I20260709 12:27:05.097764 10346 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:05.099113  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2821) took 1125 ms. Trace:
I20260709 12:27:05.100134  4605 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 13. Current term is 14. Ops: [11.9-13.10]
I20260709 12:27:05.080809  5314 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:05.106902  4676 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:27:05.107821  4755 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:05.119905  4676 pending_rounds.cc:85] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 9
I20260709 12:27:05.123565  4676 pending_rounds.cc:107] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:27:05.123834  4676 raft_consensus.cc:2889] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:05.105468 10412 consensus_meta.cc:287] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.870s	user 0.019s	sys 0.003s
I20260709 12:27:05.126106  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2816) took 1043 ms. Trace:
W20260709 12:27:05.126454  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 194240us
I20260709 12:27:05.109079  9772 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:05.085633 10453 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:05.129253  4604 rpcz_store.cc:276] 0709 12:27:03.970458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.169026 (+198568us) service_pool.cc:224] Handling call
0709 12:27:04.169427 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:04.856382 (+686955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.976632 (+120250us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400034654. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:05.013783 (+ 37151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736421248}
I20260709 12:27:05.065186 10472 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:27:05.118847 10346 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.141131  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:05.076297 10470 consensus_queue.cc:1059] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:27:05.129365  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:05.126483 10435 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:05.147065 10373 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.147538 10400 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:05.145027  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:05.153983 10261 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.155249  4495 catalog_manager.cc:5697] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:05.158113  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 82454.2us
I20260709 12:27:05.160279 10472 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:05.161761 10351 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:05.162133 10349 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:05.162957  4681 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:05.164114  4679 raft_consensus.cc:1240] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:05.165771 10409 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.166908  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:05.146090  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:05.170874  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47242.1us
W20260709 12:27:05.148689 10478 log.cc:927] Time spent T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:27:05.161209 10437 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.118920  4599 rpcz_store.cc:276] 0709 12:27:03.973107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.458006 (+484899us) service_pool.cc:224] Handling call
0709 12:27:04.470518 (+ 12512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.470712 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:04.835277 (+364565us) log.cc:844] Serialized 14 byte log entry
0709 12:27:05.098462 (+263185us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:05.098568 (+   106us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:05.098666 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.098784 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.098940 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80386,"prepare.queue_time_us":144352,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":85869,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":137,"thread_start_us":489531,"threads_started":2}
I20260709 12:27:05.175715 10373 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.174006 10400 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:05.140236 10412 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.179220 10470 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:05.176167 10472 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:05.180315 10349 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:05.172981  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 84399.8us
I20260709 12:27:05.184327  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:05.197639 10204 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.198211  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:05.198438 10204 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.184202  9772 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.190176  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:05.191697  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:05.192759  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:05.168236 10346 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.124989 10248 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:05.188236  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 52170.7us
I20260709 12:27:05.203665  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:05.202922  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:05.195647 10400 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:05.210222  5346 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:05.219358  4681 raft_consensus.cc:2410] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:27:05.214952  4673 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:05.201123 10349 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.196009 10229 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:05.201263 10470 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:05.230218  4571 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:05.201841 10453 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:05.235466 10453 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.235970 10453 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:05.237848  4675 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:05.238855 10320 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.240993 10437 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:05.241987  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.241743  9740 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.243046  4672 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 16: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:05.244459  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.230998  5348 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 8.
I20260709 12:27:05.171258 10482 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.242991  4680 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:05.232234  4674 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:05.248512  4604 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:05.248168  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:05.203850 10152 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.253224  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.254889  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:05.255344 10170 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:05.255905  4600 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:27:05.256093  5340 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 14: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:05.256117  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:05.261140  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.203634 10383 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.252689  4719 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:05.268067  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 79998.2us
W20260709 12:27:05.271489  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 50604.8us
I20260709 12:27:05.279704 10363 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:05.280234 10363 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:05.280741 10363 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.280869  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48198 (request call id 908) took 1082 ms. Trace:
I20260709 12:27:05.281404  4677 rpcz_store.cc:276] 0709 12:27:04.198245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.199489 (+  1244us) service_pool.cc:224] Handling call
0709 12:27:05.280510 (+1081021us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b54002df354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:05.280687 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27544960}
I20260709 12:27:05.263007 10204 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:05.284307  5347 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:05.284715  4676 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:05.269251 10390 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:05.220203 10409 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.287130 10472 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.296878 10487 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:05.288555 10439 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:05.305183  4720 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:05.313791  4599 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:27:05.251410 10261 consensus_queue.cc:1059] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:27:05.332465  4672 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:27:05.335460 10453 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:05.341220  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.342935 10482 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:05.343163  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:05.344142  4751 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
W20260709 12:27:05.328118  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 135500us
I20260709 12:27:05.346175  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:05.349987 10402 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:05.350961 10390 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:05.351615 10390 raft_consensus.cc:697] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:05.343469  9772 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:05.354009 10360 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:27:05.356011 10322 consensus_meta.cc:287] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.763s	user 0.003s	sys 0.015s
I20260709 12:27:05.356164 10360 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:27:05.357641 10170 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:05.308964 10346 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:05.350090  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 45002.6us
I20260709 12:27:05.278945  9740 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:05.277876 10229 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.315178 10487 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:05.365267 10248 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.366405  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:05.370280 10320 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.298267 10276 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 12:27:05.370857  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 3374) took 1062 ms. Trace:
I20260709 12:27:05.372150 10373 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.373885 10442 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.374526 10383 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.374707 10353 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.376016  9990 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.376645 10132 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.329465 10439 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:05.377821  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.352769 10390 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.377813  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:05.391228  4679 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:05.391808  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:05.391983 10229 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.299917 10301 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.396639  4748 rpcz_store.cc:276] 0709 12:27:04.308752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.490572 (+181820us) service_pool.cc:224] Handling call
0709 12:27:05.370664 (+880092us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b5400263c54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:05.370804 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567748992}
I20260709 12:27:05.394589 10351 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:05.394850 10487 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:05.414641 10412 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:05.438694  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.441653  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:05.442327 10485 log.cc:927] Time spent T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:27:05.442436  4755 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:05.443113 10132 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:05.443614 10489 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:27:05.443961  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:05.424721 10439 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.435313  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:05.447412  5344 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:05.447985  4751 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:05.448601 10204 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:05.424844 10482 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:27:05.438242 10320 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:05.450366  9990 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.450716 10320 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.452838 10320 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.453173 10143 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.418962 10453 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:05.424115  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 196591us
I20260709 12:27:05.404747  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:05.435483  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.456593  4750 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:27:05.438270  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.467489  5314 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:05.439812 10353 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:05.392861 10322 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.399282 10470 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.483358 10470 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:05.485710 10486 log.cc:927] Time spent T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:27:05.485602 10204 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:05.485977 10482 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:05.486467 10204 consensus_queue.cc:1243] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:05.487257 10204 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:05.487579 10204 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:05.487763 10402 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:05.488595 10204 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:05.489701  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 94573.9us
I20260709 12:27:05.490245  4756 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 16: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:05.491204 10409 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:05.491329  9951 consensus_queue.cc:1059] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:05.492375  9951 raft_consensus.cc:3055] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:05.492695  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.493800  4596 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 17: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:05.479602 10129 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.496160 10301 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:05.496160 10351 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.476913  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:05.480446  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:05.474764 10412 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:05.498878  4755 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 17: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:05.453890 10143 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.479921 10391 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:05.464128 10442 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.492815  9951 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:05.452854  9990 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.479071  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:05.486501 10482 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:05.479321  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:05.477947  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.515031  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:05.515166  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49915.1us
I20260709 12:27:05.499233 10412 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:05.512109 10143 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.513365  5342 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.516626  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:05.520699  4676 raft_consensus.cc:2410] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:27:05.453234 10349 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.529711  4604 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.478544 10152 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:05.540392  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 61195.5us
I20260709 12:27:05.541234 10322 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.541675 10320 consensus_queue.cc:1059] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:05.542304  9951 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.543673  9951 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.544414  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:05.545389 10469 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:05.534304  9990 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.546118  9990 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.547467 10469 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:05.548408  9990 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.525853 10482 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:05.555598  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 92754.7us
I20260709 12:27:05.556825  4750 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:27:05.557847  5312 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:05.532532 10437 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:05.562538  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.562857  5345 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:05.563488 10320 raft_consensus.cc:3055] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:05.563905 10320 raft_consensus.cc:740] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:05.564841 10320 consensus_queue.cc:260] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.566421 10320 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:05.566856  9990 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.567077  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.567760  4751 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.567407  9990 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.569895  9990 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.571072 10129 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:05.512756 10251 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:05.574236  5343 raft_consensus.cc:2410] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:27:05.575440  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.576632  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:05.566376 10248 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:05.540755 10346 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:05.617499 10448 consensus_meta.cc:287] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.003s
I20260709 12:27:05.614068  5348 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12796fc41f234615a607ba222506e893"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:27:05.630103  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:05.655977  5348 raft_consensus.cc:606] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:05.656373  5348 raft_consensus.cc:614] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:05.623862  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 66072.3us
I20260709 12:27:05.617070  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1055) took 1306 ms. Trace:
I20260709 12:27:05.623032 10373 raft_consensus.cc:3055] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:05.624619  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.625384  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:05.626101  5340 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 810) took 1416 ms. Trace:
I20260709 12:27:05.618129  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 925) took 1845 ms. Trace:
I20260709 12:27:05.578787 10442 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:05.625841  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:05.619721 10248 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.638103  5342 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:05.598987 10412 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.615196 10437 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.627420  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2823) took 1477 ms. Trace:
I20260709 12:27:05.667866  5344 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 22.
I20260709 12:27:05.614671 10349 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:05.589200 10204 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.635871  5349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53808 (request call id 673) took 1350 ms. Trace:
I20260709 12:27:05.662177  4605 rpcz_store.cc:276] 0709 12:27:03.772586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.134372 (+361786us) service_pool.cc:224] Handling call
0709 12:27:04.155387 (+ 21015us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:05.126168 (+970781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.617781 (+491613us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b5400033754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:05.618056 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2028291456}
I20260709 12:27:05.674173  4597 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:05.668892  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.668922  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 3369) took 1355 ms. Trace:
I20260709 12:27:05.661836  5340 rpcz_store.cc:276] 0709 12:27:04.209333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.215241 (+  5908us) service_pool.cc:224] Handling call
0709 12:27:05.625856 (+1410615us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b5400640354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:05.626029 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1753426304}
I20260709 12:27:05.667171  5314 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:05.677654 10360 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.678241 10360 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.679020  4571 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:05.680480 10360 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.674953  4753 rpcz_store.cc:276] 0709 12:27:04.306696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.492799 (+186103us) service_pool.cc:224] Handling call
0709 12:27:05.561297 (+1068498us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400263754. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:05.660804 (+ 99507us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341656320}
I20260709 12:27:05.681325  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.681509 10132 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.682113  4599 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:27:05.682160 10132 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.683120  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.683282  5343 raft_consensus.cc:1813] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Handling vote request from an unknown peer 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:05.683919  5343 raft_consensus.cc:2410] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:27:05.674687 10248 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:05.677709 10487 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.684829 10363 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.685710 10491 raft_consensus.cc:2764] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:27:05.686486  4645 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:05.686730 10487 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:05.687503  4645 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:05.687757 10152 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:05.688187 10152 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.688403 10402 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:05.688614 10152 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:05.696045  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.696324 10152 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.696816  4754 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:05.697199  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.697921  4748 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.700423  4597 raft_consensus.cc:2468] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:05.668058  4603 rpcz_store.cc:276] 0709 12:27:04.149743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.728719 (+578976us) service_pool.cc:224] Handling call
0709 12:27:04.729485 (+   766us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.022583 (+293098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.627201 (+604618us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54002760d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:05.627365 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":637215104}
I20260709 12:27:05.702819  4603 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:05.704114  4754 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:05.674445  5349 rpcz_store.cc:276] 0709 12:27:04.285544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.447122 (+161578us) service_pool.cc:224] Handling call
0709 12:27:05.635550 (+1188428us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54004b7b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:05.635710 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356600064}
I20260709 12:27:05.701424  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.705847  4645 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:05.684239  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.707060  4751 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:27:05.708559  5314 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:05.711618  4604 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:05.721349  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:05.657025  4747 rpcz_store.cc:276] 0709 12:27:04.310643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.895912 (+585269us) service_pool.cc:224] Handling call
0709 12:27:04.896738 (+   826us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:05.131952 (+235214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:05.131984 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:05.208712 (+ 76728us) log.cc:844] Serialized 52 byte log entry
0709 12:27:05.410938 (+202226us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:05.436377 (+ 25439us) log.cc:844] Serialized 14 byte log entry
0709 12:27:05.436677 (+   300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.436776 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.616700 (+179924us) raft_consensus.cc:1685] finished
0709 12:27:05.616736 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.616885 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21412,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":41284,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":222,"thread_start_us":390451,"threads_started":2,"wal-append.queue_time_us":267352}
I20260709 12:27:05.664549 10373 raft_consensus.cc:740] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:05.725348  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:27:05.726425  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 47553.3us
I20260709 12:27:05.730758  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.710031 10320 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:05.649852 10400 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.732196 10251 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:05.734998  4599 raft_consensus.cc:2410] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:27:05.735208  4754 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:05.736992  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:05.737126  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.714033 10470 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:05.743160  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:05.744488  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:05.674119 10439 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.744544  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:05.745172 10409 consensus_queue.cc:1059] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:27:05.745885  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 120604us
I20260709 12:27:05.746016  4675 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:05.746625 10409 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:05.746779  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:05.764391  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:05.765894 10276 raft_consensus.cc:3055] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:05.766372 10276 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:05.766970  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.767266 10276 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.768186  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:05.768612 10276 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:05.772181  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:05.772890  4605 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 17: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:05.773677 10251 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:05.773934  4755 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:27:05.774324  4755 pending_rounds.cc:85] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 9
I20260709 12:27:05.774605  4755 pending_rounds.cc:107] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:27:05.774857  4755 raft_consensus.cc:2889] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:05.774700 10251 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.775728 10149 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:05.729990 10402 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.776510 10402 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.778560  5349 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:27:05.778772 10402 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.779201  5349 raft_consensus.cc:606] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:05.779644  5349 raft_consensus.cc:614] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:05.749953 10320 raft_consensus.cc:697] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:05.781726  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:05.783185  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:05.784178 10400 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:05.784408  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:05.785980 10149 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:05.786434  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:05.786521  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:05.786972 10149 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.787691  4747 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 21.
I20260709 12:27:05.788626  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.789170  4747 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:05.789891  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:05.790678  4605 raft_consensus.cc:2393] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:27:05.791071  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.791507  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.792722  4753 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:05.793884  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:05.794735 10302 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:05.795416 10302 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:05.796236 10248 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.796643 10320 consensus_queue.cc:237] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.797852  5341 raft_consensus.cc:2393] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 18: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:05.799458  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:05.709204 10482 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.801621  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:27:05.809958  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 45650.4us
I20260709 12:27:05.726455 10353 raft_consensus.cc:2804] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:05.794853  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.769289  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:05.811956  4600 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:05.765389  4750 raft_consensus.cc:2393] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 15: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:05.813968  5348 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:05.817023  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:05.754531  4644 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:05.818634  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 64196.9us
W20260709 12:27:05.819655  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.822562  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:05.826272  4748 raft_consensus.cc:2410] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:27:05.826993  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:05.714655 10448 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.732710 10349 consensus_queue.cc:1059] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 12:27:05.838507  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:05.839527  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.815078 10383 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:05.844682  4754 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:05.850178  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:05.825515 10402 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:05.853646  4753 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:05.856985  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 110270us
I20260709 12:27:05.826233 10353 raft_consensus.cc:697] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:05.858126  4605 raft_consensus.cc:3060] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.833663 10470 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:05.859122  4676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f05b2c44da743e49df87266876b3604"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:27:05.859836  4676 raft_consensus.cc:606] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:05.855072  4748 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:27:05.863544  4754 raft_consensus.cc:1240] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:05.865097  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:05.866483  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:05.866822 10400 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:05.867659  5345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 787) took 1559 ms. Trace:
I20260709 12:27:05.867852  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:05.868072  5345 rpcz_store.cc:276] 0709 12:27:04.307799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.545394 (+237595us) service_pool.cc:224] Handling call
0709 12:27:04.547559 (+  2165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.563459 (+1015900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.867369 (+303910us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b54007dc154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:05.867567 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2249134976}
I20260709 12:27:05.869024 10482 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:05.869673 10482 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:05.870170 10230 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:05.871753 10230 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:05.870324  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:05.872464  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.869894  4678 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:05.875586  4604 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:05.876566  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 59627.4us
I20260709 12:27:05.857980 10351 raft_consensus.cc:2764] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:27:05.796643 10302 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.885478  4720 leader_election.cc:400] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17.
I20260709 12:27:05.887578  4754 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:27:05.887744  4720 leader_election.cc:403] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:27:05.872157 10230 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:05.876338 10349 consensus_queue.cc:1243] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:27:05.874702 10402 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 22 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:05.878422 10171 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:05.882949 10279 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:05.870339 10482 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:27:05.901904  4747 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:05.903038 10499 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:27:05.902935 10402 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.903290 10353 consensus_queue.cc:237] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.904619 10302 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:05.905130  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.973693  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:05.974731  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:05.974823  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:05.902974  4568 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:05.975481  4598 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:05.976125  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 778657us
W20260709 12:27:05.977090 10510 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:27:05.909263  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:05.917805 10171 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.977921  4755 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:05.927611 10437 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.931991 10439 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.978840  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.979465  4755 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:05.979588 10132 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.980729 10248 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.938603 10448 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:05.938964 10152 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.939687 10363 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:05.941813 10469 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.982496 10469 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.985728 10279 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.942128 10472 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.948359  4754 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:05.989215  4568 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:05.990592  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:05.951462 10400 raft_consensus.cc:2764] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:27:05.955065  4676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:27:05.992385  4676 raft_consensus.cc:606] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:05.992842  4676 raft_consensus.cc:614] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:05.956135  4753 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:05.957644  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:05.958343  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2818) took 1987 ms. Trace:
I20260709 12:27:05.994336  4597 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.994251  4602 rpcz_store.cc:276] 0709 12:27:03.970997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.456092 (+485095us) service_pool.cc:224] Handling call
0709 12:27:04.462593 (+  6501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.855687 (+393094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.958107 (+1102420us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b5400233cd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:05.958287 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603908864}
I20260709 12:27:05.996001  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.960949  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:05.961199  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:05.961535 10204 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:05.965227  4600 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
W20260709 12:27:05.966274  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 46843.7us
I20260709 12:27:05.998382  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:05.967955  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:05.953320  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:05.971730 10230 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:05.981470 10360 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:05.987191  4604 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:27:05.988184  4750 raft_consensus.cc:2410] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:05.988585  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:05.986390 10279 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.960076  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:27:05.994558  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.890816 10373 consensus_queue.cc:260] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:05.999061  4644 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:05.952589  4605 raft_consensus.cc:2468] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:27:05.999887  4601 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:27:06.010387  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:06.006215  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:06.013048  4753 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:05.980791 10171 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:06.016650  5312 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:05.996961 10470 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17.
I20260709 12:27:06.031272  4752 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:27:06.032773  4752 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:06.018258  4676 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:06.019140  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:06.036801  4755 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 22.
I20260709 12:27:06.037448  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:06.020247  4596 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:06.026870 10129 raft_consensus.cc:3055] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:06.050506 10129 raft_consensus.cc:740] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:06.051663 10230 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:06.051409 10129 consensus_queue.cc:260] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:06.044809  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:06.002907 10491 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:27:06.063545  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 102725us
I20260709 12:27:06.045354 10402 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:06.045384 10469 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:06.046129 10498 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:06.047495  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:06.047739 10192 raft_consensus.cc:2764] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:27:05.978894 10445 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:27:06.009209 10346 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:06.050119  4753 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:06.078253  4750 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:27:06.038705  5312 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:06.056053  4598 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:06.079335  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:06.081641  5341 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:06.057585  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:06.045225  4719 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:06.052752 10129 raft_consensus.cc:3060] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:06.062095  4597 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:06.066495  4754 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:06.072765  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:06.082696  5344 raft_consensus.cc:2393] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 16: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:06.083879  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.034930 10132 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:06.084105  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.084573  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:27:06.084471  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 45882.1us
I20260709 12:27:06.090052  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:06.079027  4645 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:06.087399  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:06.087539  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:06.095431  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:06.096076  4753 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:27:06.096985  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:06.089232  4679 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 10.
I20260709 12:27:06.093511  5345 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:27:06.091940  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:06.088222  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:06.086454  4681 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:06.092669  5342 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:06.095726  5348 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:27:06.099713  5348 pending_rounds.cc:85] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 8
I20260709 12:27:06.097594  4753 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:06.100684  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:06.104426  5314 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:06.102371  4676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8e62ff81eb34853811314530131292b"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:27:06.108618  5343 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 17: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:06.110790  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:06.113204  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:06.113090  4605 raft_consensus.cc:1240] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 12:27:06.155913  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 45266.5us
I20260709 12:27:06.072813 10171 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:06.163837  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:06.096606 10320 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:06.168823  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:06.099951  5348 pending_rounds.cc:107] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 11.9
I20260709 12:27:06.168848  4645 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:06.151237 10204 raft_consensus.cc:2764] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:27:06.186420  4752 raft_consensus.cc:1240] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:06.188659  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 143496us
I20260709 12:27:06.189239  5348 raft_consensus.cc:2917] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Aborting config change with OpId 11.9: Aborted: Op aborted by new leader
I20260709 12:27:06.189987  4752 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:06.108171 10445 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:06.147341 10491 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:06.085053 10437 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:27:06.201844  4753 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:06.192625  5348 raft_consensus.cc:2889] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:06.179361  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:06.203190  4753 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:06.113655 10149 raft_consensus.cc:3055] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:06.195266  4678 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:06.176509  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:06.187903 10373 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:06.205000  4750 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:27:06.168068 10251 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:06.204762  5314 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:06.175442 10320 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:06.192205  9990 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:06.195150 10445 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.239637  5312 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:06.235324 10204 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.242370 10204 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:06.246088  5342 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:06.235323 10349 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:06.246912 10229 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:06.247468  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.247968 10383 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.248309  4672 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 18: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:27:06.249100 10522 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:06.248657 10383 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:06.249394  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.249778  4571 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:06.219208 10512 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:27:06.250681  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:06.251248  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:06.251922  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.252746  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.253762  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.174188 10402 consensus_queue.cc:1059] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:27:06.206377 10149 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:06.259572  4755 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:06.259766  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:06.258198  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:06.257283  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.212141 10363 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:06.248215  5348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:41342 (request call id 1) took 14231 ms. Trace:
I20260709 12:27:06.244797 10204 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:06.220702 10320 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:06.270110  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:06.273283  5312 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:06.283038  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.275337  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.208961 10132 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
W20260709 12:27:06.291946  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.293145  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.256848 10445 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:06.298146  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.301833  4647 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:06.205701 10171 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:06.262058 10383 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:06.299584  4674 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:06.256737 10448 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:06.325729 10448 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:06.262652  9990 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:06.303735  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:06.296453 10251 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:06.322960 10482 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.323258 10230 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:06.271739  5348 rpcz_store.cc:276] 0709 12:26:52.016700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.055059 (+ 38359us) service_pool.cc:224] Handling call
0709 12:26:53.713488 (+1658429us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b580052fad0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4120aa 00007effee494447 00007effee4b170a 00007effee4b276a 00007effee4b191b 00007effee4bd03c 00007effee4bc6b3 00007effee4b49fa 00007efff347c4c9 00007effec9f8c54
0709 12:26:53.730655 (+ 17167us) log.cc:844] Serialized 494 byte log entry
0709 12:26:54.143176 (+412521us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b180045c760. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007effeb3cba40 00007effeb3c64e4 00007effeb3c6455 00007effee40fb0b 00007effee494b63 00007effee4bc752 00007effee4b49fa
0709 12:26:54.356032 (+212856us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b5400854920. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee42f062 00007effee40feaa 00007effee416d53 00007effee416cd9 00007effee416c69 00007effee416c31 00007effee415dfd 0000000000388634 0000000000388479 00007effeb3c9475
0709 12:26:54.627124 (+271092us) spinlock_profiling.cc:243] Waited 4.97 ms on lock 0x7b180045c760. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3972a6 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee42bdbe 00007effee436930 00007effee4368d8
0709 12:26:54.960117 (+332993us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400854920. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee42bdbe 00007effee436930 00007effee4368d8 00007effee436858 00007effee436813 00007effee435a9c 00007efff43212f6 00007efff431baa8 00007effee4727cb 00007effee4752c0
0709 12:27:06.248083 (+11287966us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":310680,"raft.queue_time_us":334334,"raft.run_cpu_time_us":3444,"raft.run_wall_time_us":202293,"spinlock_wait_cycles":2624096384,"thread_start_us":120402,"threads_started":1,"wal-append.queue_time_us":160253,"wal-append.run_cpu_time_us":19049,"wal-append.run_wall_time_us":2070865}
I20260709 12:27:06.325021  5349 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:06.319854 10453 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:06.321099  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:06.277324 10469 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:06.310909 10363 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:06.341676 10132 raft_consensus.cc:3055] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:27:06.346120  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 128234us
I20260709 12:27:06.328254 10448 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:06.307996 10351 raft_consensus.cc:2804] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:06.351907 10230 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:06.352696 10453 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.366395  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:27:06.366801  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 69223.1us
I20260709 12:27:06.367177 10192 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:06.367028 10230 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:06.367849 10192 raft_consensus.cc:697] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:06.368916 10192 consensus_queue.cc:237] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:06.370129 10351 raft_consensus.cc:697] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:06.371145 10351 consensus_queue.cc:237] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:06.375128 10516 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
W20260709 12:27:06.376775 10524 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:27:06.356184 10482 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:06.379801 10482 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:06.381280  4751 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:27:06.382108 10437 raft_consensus.cc:3055] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:06.382525 10437 raft_consensus.cc:740] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:06.383435 10437 consensus_queue.cc:260] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:06.384703 10437 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:06.386004  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 84276.2us
I20260709 12:27:06.387017  4750 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:06.386847  4647 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:06.356001  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:06.388375  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:06.388804  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:06.274837 10514 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:27:06.352532  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:06.355120  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.390702  4755 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:27:06.355036 10469 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:06.364573 10132 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:06.352514  5348 pending_rounds.cc:107] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:27:06.394835  5348 raft_consensus.cc:2889] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:06.391004  4673 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:27:06.366484 10453 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:06.397688 10442 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:06.400329  4747 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:06.354249  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:06.358724  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:06.402170  4754 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:06.391251  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:06.388306  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.411192  4647 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:06.406407  4680 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:06.358024 10251 raft_consensus.cc:697] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:27:06.415742  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 162852us
I20260709 12:27:06.418627  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:06.395715 10448 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:06.410355 10525 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:27:06.416932 10472 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:06.420380 10520 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:06.439590  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:06.437247  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 149126us
I20260709 12:27:06.423667 10499 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:06.447124  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:06.429064 10349 raft_consensus.cc:697] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:06.454432  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:06.453224  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:06.455866 10279 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:06.460273 10517 log.cc:927] Time spent T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:27:06.461715 10132 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:06.462426 10453 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:06.463414 10132 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:06.464793 10453 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:06.467020 10349 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:06.470888  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:06.472287  5342 raft_consensus.cc:2410] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:27:06.458649  9740 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:06.445405 10469 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:06.479017  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.479741  4675 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:27:06.483738 10251 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:06.486452  5347 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:06.390210 10149 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:06.487814  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:27:06.489818  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 50331us
I20260709 12:27:06.497057  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.497731  4675 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
W20260709 12:27:06.499271  5312 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.452553 10472 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:06.498862 10469 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:06.500164 10469 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.500075 10442 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:06.500556 10472 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:06.500735 10469 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:06.501037 10472 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:06.501083  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:06.503860 10149 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:06.505880  4485 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e8a74137ba104fe0b465cab6399c4940 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) due to error UNKNOWN_ERROR; will retry: Aborted: Op aborted by new leader
I20260709 12:27:06.489012 10279 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:27:06.444367  9990 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:06.459290 10520 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:27:06.459460 10353 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:06.511677 10353 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:27:06.513923 10521 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.124s	user 0.002s	sys 0.000s
I20260709 12:27:06.512678 10353 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:06.521673  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:06.527060  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 73968.6us
I20260709 12:27:06.528386  4571 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:06.532724  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:06.535442  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:27:06.506914  4485 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 53045.2us
I20260709 12:27:06.543275  4603 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:27:06.537951  4605 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:06.538666  9740 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:06.545439  9740 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:06.545977  9740 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:06.547389  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:06.543831  4603 pending_rounds.cc:85] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 10
I20260709 12:27:06.551007  4603 pending_rounds.cc:107] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:27:06.551365  4603 raft_consensus.cc:2889] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:06.553958 10520 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:06.554546 10520 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.555014 10520 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:06.558634 10360 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:06.452320 10363 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:06.565667  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:06.566509  4599 raft_consensus.cc:2393] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:06.567656  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:06.571473  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 152963us
I20260709 12:27:06.577868  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:06.610095  9740 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:06.645677  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 76833.1us
I20260709 12:27:06.646706  4647 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:06.544107 10491 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.694001  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:06.696717  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 50131.8us
I20260709 12:27:06.703342  4597 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:27:06.550395 10531 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:06.708171  4599 raft_consensus.cc:1240] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:06.708999  9740 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:06.693140  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:06.690690 10279 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.648896 10192 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:27:06.743400  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.753192  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:06.746191  4751 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:27:06.749606  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:06.759181  4601 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 17: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:06.760308  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:06.742504 10531 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:27:06.764074  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 71050us
I20260709 12:27:06.766309  4678 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:06.735816 10482 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:06.739553 10171 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:06.728950 10498 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:27:06.774096  5314 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:06.777915 10537 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:06.778077 10531 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:06.778494  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:06.779013 10383 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:06.777678  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 224273us
I20260709 12:27:06.780627  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:06.693113 10402 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:27:06.781821 10402 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:06.782078  4602 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:27:06.782256 10402 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:06.783231  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:06.784044  4604 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:06.783427 10402 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:06.785696  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:06.698345 10491 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:06.731099 10528 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:27:06.785485  4602 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:27:06.786490  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:06.786958  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 89411.4us
I20260709 12:27:06.779458 10383 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.763621 10330 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:06.833758  4596 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:27:06.779320  4601 raft_consensus.cc:2368] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19.
I20260709 12:27:06.789201  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:06.789294  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:06.745007 10532 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
W20260709 12:27:06.852612  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 124929us
I20260709 12:27:06.832260  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:06.767148 10192 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:06.851337  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 63527.5us
I20260709 12:27:06.885888  4674 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:27:06.760248 10279 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:06.886087  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 121177us
I20260709 12:27:06.834894 10383 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:06.896529  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:06.900602  4569 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:06.907019  4597 raft_consensus.cc:2393] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 15: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:06.883908 10143 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:06.777159 10482 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:06.912665  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.916704  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:06.908779 10353 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:06.900419 10248 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:27:06.946031  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:06.918732 10330 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:06.902909 10363 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:27:06.950235  5348 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:06.951205 10409 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:06.956983 10402 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:06.915124 10143 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:06.912238 10251 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
W20260709 12:27:06.971458  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 192861us
I20260709 12:27:06.928308 10442 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:27:06.974234  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 89007.7us
I20260709 12:27:06.976090  4751 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:27:06.976436  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 144287us
I20260709 12:27:06.977262  4647 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:06.992702  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:06.980235 10129 consensus_meta.cc:287] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.879s	user 0.007s	sys 0.006s
I20260709 12:27:06.997139  4597 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:06.996344  4647 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:06.948513 10353 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:06.979604 10442 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:06.996505  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 95958.7us
I20260709 12:27:07.147922  4604 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:07.012249  5343 raft_consensus.cc:1275] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:07.042723  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:07.154577  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:06.982067 10514 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:07.056912  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:07.077555  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.000438  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:07.009922  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:07.007915  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:07.165673  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.059484  4598 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:07.008956  4602 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:07.050909  4751 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:07.167948  4755 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:07.165458  5347 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:07.035772  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2833) took 1132 ms. Trace:
I20260709 12:27:07.164770  4754 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:27:07.168807  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:07.091813  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:07.175405  4750 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:27:07.176762  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:07.177732  4747 rpcz_store.cc:276] 0709 12:27:05.903366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.953036 (+ 49670us) service_pool.cc:224] Handling call
0709 12:27:07.035542 (+1082506us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b5400117454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:07.035687 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":920984832}
W20260709 12:27:07.003407 10546 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260709 12:27:07.008809 10538 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:27:07.179849  4675 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 19.
I20260709 12:27:07.175438  4752 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
W20260709 12:27:07.036574 10472 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.535s	user 0.008s	sys 0.000s
W20260709 12:27:07.005380 10445 consensus_meta.cc:287] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.675s	user 0.007s	sys 0.002s
W20260709 12:27:07.105345 10149 consensus_meta.cc:287] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.601s	user 0.011s	sys 0.000s
W20260709 12:27:07.050207 10545 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.084s	user 0.004s	sys 0.001s
W20260709 12:27:07.002681 10469 consensus_meta.cc:287] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.002s
I20260709 12:27:07.208200 10469 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:07.002893 10541 log.cc:927] Time spent T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:27:07.036574 10320 consensus_meta.cc:287] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.725s	user 0.002s	sys 0.007s
W20260709 12:27:07.179901  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 183646us
W20260709 12:27:07.057606 10540 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:27:07.052354 10276 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:27:07.197938  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 190457us
I20260709 12:27:07.056859 10279 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:07.055691 10499 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:07.167177 10400 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.003s	sys 0.014s
I20260709 12:27:07.054279 10544 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.177s
I20260709 12:27:07.206416  4748 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:27:07.222128 10520 consensus_meta.cc:287] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.661s	user 0.013s	sys 0.000s
I20260709 12:27:07.230931  4568 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:07.055761 10192 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:07.218613  4676 raft_consensus.cc:606] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:07.240715  4676 raft_consensus.cc:614] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:07.219228  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 65863.1us
W20260709 12:27:07.197328 10230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.019s	sys 0.008s
I20260709 12:27:07.001199 10552 raft_consensus.cc:2764] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 12:27:07.220566  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 51386.1us
I20260709 12:27:07.012996 10132 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.199s
I20260709 12:27:07.200158  5312 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:07.248816  4750 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:07.008318 10346 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.208s
I20260709 12:27:07.107278 10143 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:07.105748 10561 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.157s
W20260709 12:27:07.227504 10437 consensus_meta.cc:287] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.834s	user 0.015s	sys 0.000s
I20260709 12:27:07.056417 10491 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:07.249223  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49190.4us
I20260709 12:27:07.213352 10445 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.251348  4678 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:27:07.237804 10400 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.004s	sys 0.014s
I20260709 12:27:07.167781 10562 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 12:27:07.019132 10558 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.211s
I20260709 12:27:07.202930 10439 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.040s
W20260709 12:27:07.276681  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 96644.9us
I20260709 12:27:07.226547 10276 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
W20260709 12:27:07.248298 10230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.020s	sys 0.008s
I20260709 12:27:07.213662 10472 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.237754 10499 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:07.262269 10551 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:07.250213  4752 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:07.210254 10351 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
W20260709 12:27:07.262845  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 66251.1us
I20260709 12:27:07.226383 10320 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.268180  4569 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:07.256274 10498 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:07.277673  4677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 2173) took 1028 ms. Trace:
I20260709 12:27:07.286513 10469 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:07.287231 10491 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:07.286605 10562 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:27:07.288755 10439 raft_consensus.cc:3055] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:07.289278 10192 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:07.273108 10552 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:27:07.292804  4672 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 15. Ops: []
I20260709 12:27:07.305375  4596 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:27:07.248871 10520 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.314170  4605 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:07.315435 10353 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:07.312716 10560 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:07.315548 10530 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:27:07.319499  4751 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: []
W20260709 12:27:07.309233 10373 consensus_meta.cc:287] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 1.092s	user 0.010s	sys 0.005s
I20260709 12:27:07.291038  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:27:07.378111  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 87146.3us
I20260709 12:27:07.379061  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:07.317355  4677 rpcz_store.cc:276] 0709 12:27:06.249488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.253753 (+  4265us) service_pool.cc:224] Handling call
0709 12:27:06.254106 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.254659 (+   553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.254709 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.255311 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.645717 (+390406us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:06.906361 (+260644us) log.cc:844] Serialized 14 byte log entry
0709 12:27:06.948877 (+ 42516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.952844 (+  3967us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.277452 (+324608us) raft_consensus.cc:1685] finished
0709 12:27:07.277471 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.277617 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58092,"prepare.queue_time_us":328698,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":77723,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":16856,"thread_start_us":596762,"threads_started":2,"wal-append.queue_time_us":516082}
I20260709 12:27:07.309852 10437 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:07.354499  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 123649us
I20260709 12:27:07.317273 10498 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:07.359351  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 91283.2us
I20260709 12:27:07.302811  9951 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
W20260709 12:27:07.376150  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 82128.9us
I20260709 12:27:07.330466 10439 raft_consensus.cc:740] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:07.383566  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:07.387696 10514 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:07.368919 10192 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.393275 10469 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.388419 10353 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:07.396229 10498 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:07.357887 10573 consensus_queue.cc:1059] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:27:07.395298 10132 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:27:07.399720 10442 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:07.404261 10448 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:07.395561 10560 consensus_queue.cc:1059] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
W20260709 12:27:07.424705 10569 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:27:07.425166 10536 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:07.425189 10553 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.412458 10469 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.425547 10192 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:07.427906  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:27:07.473974  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 46165us
I20260709 12:27:07.430559 10445 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:07.430912 10576 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:07.409312 10514 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.393081 10437 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:07.443001 10548 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.446s	user 0.002s	sys 0.001s
I20260709 12:27:07.443166 10575 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 12:27:07.443001 10566 log.cc:927] Time spent T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 12:27:07.449527 10383 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.532s	user 0.006s	sys 0.006s
I20260709 12:27:07.402841 10537 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:27:07.455745  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:07.456650 10570 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.167s	user 0.002s	sys 0.000s
I20260709 12:27:07.457084 10472 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:07.457867  4719 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:07.481025  9951 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:07.459197 10557 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.457s	user 0.001s	sys 0.001s
I20260709 12:27:07.458770 10499 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.430599 10520 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:07.416659 10402 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:27:07.471951 10581 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.005s
I20260709 12:27:07.495736  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:07.499177  5349 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 814) took 1124 ms. Trace:
I20260709 12:27:07.519618  5349 rpcz_store.cc:276] 0709 12:27:06.374404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.390578 (+ 16174us) service_pool.cc:224] Handling call
0709 12:27:06.410954 (+ 20376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.532538 (+121584us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:06.532569 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:06.532618 (+    49us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:06.532654 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.498923 (+966269us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540084ce54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:07.498987 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.499120 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279085440}
I20260709 12:27:07.502135  5344 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 15: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:07.465024 10439 consensus_queue.cc:260] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.484340 10514 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:07.505169  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 47387.9us
I20260709 12:27:07.458293 10320 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:07.518018  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 118318us
I20260709 12:27:07.479861 10553 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.485591 10143 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:27:07.511828 10192 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:07.536571  4644 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:07.510361 10579 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:07.485782 10550 consensus_queue.cc:1059] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:27:07.511967 10469 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:07.542303  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:07.546825  5342 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:07.506814 10383 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.548542  5348 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:27:07.549839  4569 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:07.558884  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:07.550231 10383 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:07.550513 10536 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:07.551061 10437 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:07.551416  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:07.543751 10579 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:07.534096 10558 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.557015 10499 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:07.548622 10520 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:07.533442 10560 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.585160 10560 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.588323 10230 raft_consensus.cc:3055] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:07.588650 10573 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:07.588999 10230 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:07.590284 10152 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:07.590826 10152 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.592854 10152 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:07.593478 10383 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.594041 10383 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.564196  4569 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:07.568574 10437 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:07.568977 10585 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:07.538156 10192 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:07.561439 10439 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:07.560768  9951 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:07.575850  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:07.570286 10575 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:07.602886  5341 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:07.581262  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:07.583371 10551 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.583567  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:47062 (request call id 26) took 1513 ms. Trace:
I20260709 12:27:07.584165 10520 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.600414  4753 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:07.605513  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:07.607872  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:07.608836  4753 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:07.609669  4569 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:07.606498  4676 rpcz_store.cc:276] 0709 12:27:06.070280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.081242 (+ 10962us) service_pool.cc:224] Handling call
0709 12:27:07.583361 (+1502119us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400085554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b0853 00007efff347e60a 00007effeca047d4 00007effeca04762 00007effeca04691 00007effeca0460c 00007effeca038c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:07.583485 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2554352512}
I20260709 12:27:07.596110 10383 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:07.618274  4644 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:07.619062 10143 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:07.602769 10437 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:07.620903  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:07.603689 10192 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:07.573222 10558 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.629419  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:07.631044  4749 raft_consensus.cc:3055] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:07.631407  4749 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:07.631583 10558 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:07.632280  4749 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.633334  4749 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:07.635309 10579 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.636528 10579 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:07.612035 10551 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.620472  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:07.602641 10585 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:07.589314 10573 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.622748  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:07.623008  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:07.621991  4601 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:27:07.642164  4569 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:07.642504  4747 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:07.624051 10230 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.624266 10437 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.641101  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:07.644224  4752 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:07.645715  4747 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:07.646230  4747 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:07.646780  4749 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:07.619454 10143 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.615876 10575 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:07.654546  4673 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:07.657372 10520 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:07.657414  4676 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:07.658771 10553 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:07.653285  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:07.659931  4681 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:07.660254 10560 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:07.661357 10230 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:07.645371 10558 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:07.662170 10558 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:07.662549 10558 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:07.662662  4680 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:07.643306 10585 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:07.610944 10588 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:07.650044  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:07.652272  4677 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:07.653558 10437 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:07.668380  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:07.668479  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:07.669013  5346 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:27:07.670255  5340 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:27:07.653841  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:07.670697  4568 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:07.672499 10576 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:07.672884 10576 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:07.673233 10576 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:07.673362  5340 raft_consensus.cc:1240] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:27:07.620028 10498 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.675561 10498 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:07.673420 10520 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:27:07.677008 10573 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:07.677814  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:07.678426  5346 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:27:07.679577  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:07.679725 10537 consensus_queue.cc:1059] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:07.680141  4679 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:07.680485 10537 consensus_queue.cc:1243] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:07.648269  4678 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:07.670750 10551 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:07.646996  4747 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.661640 10143 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:07.684005  4644 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:07.691622 10551 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:07.692026 10551 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.692121  4600 raft_consensus.cc:3055] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:07.692400 10551 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:07.692584  4600 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:07.693536  4600 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.694864  4600 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:07.698318 10563 log.cc:927] Time spent T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.422s	user 0.002s	sys 0.000s
I20260709 12:27:07.689736  4753 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:07.700139  4747 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:07.701830  4644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.706983  4681 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
W20260709 12:27:07.709811  4485 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e8a74137ba104fe0b465cab6399c4940 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica dbe960c35c84451faf4e5b5c9233fa39 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:07.711478  4679 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
W20260709 12:27:07.715472  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.718434  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:07.723596  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:07.724757  4753 raft_consensus.cc:2410] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:27:07.727432  4600 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:27:07.727974  4600 pending_rounds.cc:85] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 10
I20260709 12:27:07.728235  4600 pending_rounds.cc:107] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:27:07.728503  4600 raft_consensus.cc:2889] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:07.729404  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:07.730460  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:07.731788  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:07.725955 10302 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.734704 10302 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.738989 10537 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:07.744195  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:07.753161 10302 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:07.770831 10514 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.783546  4754 raft_consensus.cc:2410] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 12:27:07.783833 10537 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:07.783994  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:07.745658 10579 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:07.725766 10439 consensus_queue.cc:1048] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:27:07.804133  4569 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:07.795814  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:07.800508  4747 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:27:07.802023  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:07.805333  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:07.804339  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:07.808192  4569 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:07.809325  4754 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 17: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:07.805702  4747 pending_rounds.cc:85] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 8
I20260709 12:27:07.810721  4747 pending_rounds.cc:107] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:27:07.810981  4747 raft_consensus.cc:2889] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:07.813462  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:07.813496  4597 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:07.809667  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:07.817308  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:07.816109  4569 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:07.819299  4675 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:27:07.802304 10437 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.808182 10302 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:07.806262 10581 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.840276  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:27:07.816454 10531 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.028s	sys 0.038s
I20260709 12:27:07.808336 10439 consensus_queue.cc:1243] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:27:07.842206  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:07.848227  4598 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:07.812041 10230 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:07.842159  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:07.850271  4747 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:07.842571 10437 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:07.860006 10595 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:27:07.837426 10498 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:07.861101 10596 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:27:07.861356 10498 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.862015 10143 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.863458 10498 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:07.815322 10400 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.870987  4598 raft_consensus.cc:2468] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:07.825989 10360 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:07.803066 10192 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.837285 10588 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:07.877718  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:07.879237  4597 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 12. Current term is 13. Ops: []
W20260709 12:27:07.852416 10531 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.030s	sys 0.039s
I20260709 12:27:07.888260  4747 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:27:07.883607 10597 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:07.883620 10383 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:07.895074  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:07.858832 10581 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.885641 10276 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.886318 10469 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:07.887985  9951 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:07.875782  4645 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:07.889464  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:07.891183  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:07.864236 10143 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:07.892592  5314 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:07.819111 10560 consensus_queue.cc:1059] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:27:07.875168 10349 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:07.903297  4747 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:07.895596 10472 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.915337  4679 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:07.877044 10553 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:27:07.892767 10353 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:27:07.918367 10514 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:07.922796 10349 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.923616 10558 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.924575  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:07.925141  4750 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:07.897226 10499 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:27:07.902438 10469 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:07.928551 10469 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:07.931442  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:07.900606 10360 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:07.944284  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:07.944443  4750 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 20.
I20260709 12:27:07.945645  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:07.946826  4747 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:07.899713 10588 raft_consensus.cc:697] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:07.949259  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:07.949694  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:07.949898  4747 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 12.
I20260709 12:27:07.950997  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:07.952553  5314 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:07.914850 10572 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.603s	user 0.001s	sys 0.001s
I20260709 12:27:07.878093 10400 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:07.954596  5314 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:07.954725  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:07.958632  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:07.959383  4678 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:07.914237 10437 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:07.955403 10373 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:08.345944 10373 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.346465 10373 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:08.346164 10149 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:08.347204 10149 consensus_queue.cc:1243] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:07.988675  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:08.350183  4677 raft_consensus.cc:2368] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 17. Current term is 18.
I20260709 12:27:07.961045 10537 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:27:07.955818 10558 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:08.000313  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.003489 10601 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:27:07.943142 10472 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.367480 10579 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:08.368108 10579 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:08.369057 10472 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:07.899397 10383 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.373106  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1077) took 1399 ms. Trace:
I20260709 12:27:07.935994 10276 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:08.373513  4756 rpcz_store.cc:276] 0709 12:27:06.973213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.973347 (+   134us) service_pool.cc:224] Handling call
0709 12:27:06.973778 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.981861 (+  8083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.981894 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.994427 (+ 12533us) log.cc:844] Serialized 26 byte log entry
0709 12:27:07.682335 (+687908us) spinlock_profiling.cc:243] Waited 6.46 ms on lock 0x7b1800321b80. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:27:07.802933 (+120598us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:07.802982 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.803051 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:08.369654 (+566603us) raft_consensus.cc:1685] finished
0709 12:27:08.369681 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.373020 (+  3339us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33929,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":14270,"spinlock_wait_cycles":14868352,"thread_start_us":109805,"threads_started":1,"wal-append.queue_time_us":842167}
I20260709 12:27:07.925462 10349 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:07.989596 10514 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:07.962039 10439 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:07.955574 10550 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:07.914985 10498 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:07.954396 10143 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:08.391515  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 2174) took 2088 ms. Trace:
W20260709 12:27:08.198256  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.226212 10360 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:08.247926 10564 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.932s	user 0.007s	sys 0.000s
I20260709 12:27:08.249898  4605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2867) took 1549 ms. Trace:
I20260709 12:27:08.395982  4605 rpcz_store.cc:276] 0709 12:27:06.700104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.728856 (+ 28752us) service_pool.cc:224] Handling call
0709 12:27:06.760277 (+ 31421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:07.368259 (+607982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.249671 (+881412us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b5400233cd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:08.249842 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1273372032}
W20260709 12:27:08.250708 10576 consensus_meta.cc:287] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.007s
W20260709 12:27:08.251071 10551 consensus_meta.cc:287] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.558s	user 0.015s	sys 0.006s
I20260709 12:27:08.223047 10588 consensus_queue.cc:237] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:08.253454 10598 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.351s	user 0.007s	sys 0.000s
I20260709 12:27:08.407305  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:08.408128  4677 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:08.198153 10279 raft_consensus.cc:3055] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:08.255505  4748 raft_consensus.cc:1275] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:27:08.255914  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 956) took 1196 ms. Trace:
W20260709 12:27:08.263007 10585 consensus_meta.cc:287] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.598s	user 0.014s	sys 0.003s
I20260709 12:27:08.258257 10482 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:08.263448 10581 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:08.369223 10579 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.340127  5341 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:27:07.990005 10353 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:27:07.987094 10320 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:07.987558 10530 raft_consensus.cc:2804] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:08.445895  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:08.446542  4674 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 13.
W20260709 12:27:08.343334  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 386022us
W20260709 12:27:08.363278  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.370762  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:08.383777  5312 leader_election.cc:400] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 17. Current term is 18.
I20260709 12:27:08.447515  5312 leader_election.cc:403] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:27:08.391916  4672 rpcz_store.cc:276] 0709 12:27:06.292425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.292565 (+   140us) service_pool.cc:224] Handling call
0709 12:27:06.314945 (+ 22380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:07.357923 (+1042978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.380720 (+1022797us) spinlock_profiling.cc:243] Waited 976 ms on lock 0x7b5400086954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:08.380906 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2244630912}
I20260709 12:27:08.409256  4599 rpcz_store.cc:276] 0709 12:27:07.059834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.094176 (+ 34342us) service_pool.cc:224] Handling call
0709 12:27:07.094584 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:07.110248 (+ 15664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:07.110297 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:07.110920 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 12:27:07.239241 (+128321us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:07.239306 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.239417 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.882415 (+642998us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:27:08.255645 (+373230us) raft_consensus.cc:1685] finished
0709 12:27:08.255672 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.255857 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32501,"raft.queue_time_us":53156,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":11692,"spinlock_wait_cycles":53227264,"thread_start_us":40575,"threads_started":1,"wal-append.queue_time_us":98269}
I20260709 12:27:07.973723  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:08.414450  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3407) took 1251 ms. Trace:
W20260709 12:27:08.417023  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 460401us
I20260709 12:27:08.409198  4748 pending_rounds.cc:85] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 10
I20260709 12:27:08.424683  9740 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:07.946368  9951 raft_consensus.cc:2804] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:27:08.443368  4600 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.016s	sys 0.007s
I20260709 12:27:08.470366  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:08.470764  4600 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.016s	sys 0.007s
W20260709 12:27:08.471864  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 88843.2us
I20260709 12:27:08.450457  4748 pending_rounds.cc:107] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:27:08.450738  9740 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.474929  4748 raft_consensus.cc:2889] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:08.414419 10279 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:27:08.429646  4645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.007s	sys 0.012s
W20260709 12:27:08.478963  4645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.007s	sys 0.012s
W20260709 12:27:08.479621  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 603859us
I20260709 12:27:08.459869  4678 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:08.448813  4751 rpcz_store.cc:276] 0709 12:27:07.162822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.182254 (+ 19432us) service_pool.cc:224] Handling call
0709 12:27:07.182583 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:07.371051 (+188468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.414182 (+1043131us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b540030af54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:08.414391 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311525248}
W20260709 12:27:08.451323  4753 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.006s	sys 0.009s
W20260709 12:27:08.484315  4753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.006s	sys 0.010s
I20260709 12:27:08.429646 10143 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:08.486989  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:08.445384 10320 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.499490  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:08.500862  4678 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:08.499058  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:08.501479  4672 raft_consensus.cc:3060] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:08.405912 10576 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.504606  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:08.505339  4747 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:08.487381 10530 raft_consensus.cc:697] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 20 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:08.512516  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:08.513350  4674 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 20.
I20260709 12:27:08.513996 10279 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.514720  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:08.515230 10279 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:08.516532  9740 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:08.517648 10608 consensus_queue.cc:1059] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:27:08.518301 10607 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:27:08.519712  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:08.486299 10581 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 12:27:08.449520 10482 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.523324 10482 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.417315 10551 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.527725 10551 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:08.528071 10576 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:08.426440 10585 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.485194  9951 raft_consensus.cc:697] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:08.529960 10412 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:08.530457 10585 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:08.534291  4676 raft_consensus.cc:2410] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:27:08.538460  4677 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:27:08.538465  4681 raft_consensus.cc:2368] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14.
I20260709 12:27:08.529635  9951 consensus_queue.cc:237] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.536103  5312 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:08.546496  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.547364  4755 pending_rounds.cc:85] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 7
I20260709 12:27:08.549168  4755 pending_rounds.cc:107] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:27:08.549552  4755 raft_consensus.cc:2889] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:08.549693  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:08.482896 10532 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:08.560251  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 109826us
I20260709 12:27:08.553960  5312 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:08.554857  4678 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:08.558398  4647 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:08.565363  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:08.567904  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:08.568359  4677 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:08.572170 10373 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.533144 10320 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:08.579852  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 46872.2us
I20260709 12:27:08.580825  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:27:08.559386 10171 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.531s	user 0.007s	sys 0.040s
I20260709 12:27:08.583333  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:08.583997  4678 raft_consensus.cc:1240] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17. Ops: [14.10-16.11]
I20260709 12:27:08.552829 10579 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:08.547840 10472 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.554188 10482 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:08.564946 10251 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:08.560380 10498 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:08.589788 10171 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.559s	user 0.008s	sys 0.040s
I20260709 12:27:08.582844 10442 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.579444 10412 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:08.606171 10442 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.606801 10412 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.584129 10562 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.609427 10562 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.610533 10442 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:08.603542  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:08.587393 10553 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:08.519276 10530 consensus_queue.cc:237] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.614303  4601 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:08.615945  4645 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:27:08.618991  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.586534 10571 raft_consensus.cc:3055] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:08.623188  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:08.623421  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:08.625011  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:08.625149  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:08.625620  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.629078 10373 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:08.630307  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:08.631325  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1084) took 1039 ms. Trace:
I20260709 12:27:08.599979 10251 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:08.652091  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.616336 10554 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.596s	user 0.002s	sys 0.023s
W20260709 12:27:08.654013  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 58673.8us
I20260709 12:27:08.653409 10437 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:08.621069 10585 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:08.646344  4752 rpcz_store.cc:276] 0709 12:27:07.591962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.683111 (+ 91149us) service_pool.cc:224] Handling call
0709 12:27:07.720801 (+ 37690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:07.727179 (+  6378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:07.727209 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:07.734513 (+  7304us) log.cc:844] Serialized 26 byte log entry
0709 12:27:08.261320 (+526807us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b1800451960. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:27:08.477367 (+216047us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:08.477414 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.477501 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:08.631038 (+153537us) raft_consensus.cc:1685] finished
0709 12:27:08.631059 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.631266 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5902,"raft.queue_time_us":12008,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":117,"spinlock_wait_cycles":57190656,"thread_start_us":215847,"threads_started":1,"wal-append.queue_time_us":859384}
I20260709 12:27:08.653810 10469 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:08.658679  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:08.658427 10469 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.659435  4752 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 17: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:08.660339  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:08.660852  4752 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:08.661042 10469 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:08.614421 10552 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:08.661935 10552 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.653723 10482 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.662658 10553 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:08.663278 10482 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.664017 10552 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:08.665547  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:08.666059  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:08.667311  4601 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:08.615453 10562 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:08.670311  4752 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 20: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:08.666913  4603 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:27:08.671226  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:08.602612 10612 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:08.664077 10553 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.601436 10472 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:08.665462 10554 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.645s	user 0.002s	sys 0.023s
I20260709 12:27:08.687256  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:27:08.656028  4720 connection.cc:256] RPC call timeout handler was delayed by -1.0020140279998486s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
I20260709 12:27:08.690747  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:08.649257 10171 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.638509 10571 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:08.698140  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:08.654170 10616 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.699214  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2870) took 1087 ms. Trace:
I20260709 12:27:08.703204  4677 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:08.669373 10251 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:08.705767  4749 rpcz_store.cc:276] 0709 12:27:07.611624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.686729 (+ 75105us) service_pool.cc:224] Handling call
0709 12:27:07.687272 (+   543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:07.709001 (+ 21729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:07.709050 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:07.727219 (+ 18169us) log.cc:844] Serialized 26 byte log entry
0709 12:27:07.849826 (+122607us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:07.878265 (+ 28439us) log.cc:844] Serialized 14 byte log entry
0709 12:27:07.878694 (+   429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.879002 (+   308us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.928922 (+ 49920us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b180024bf80. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c9522 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:27:08.656103 (+727181us) spinlock_profiling.cc:243] Waited 7.54 ms on lock 0x7b180024bf80. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:27:08.698987 (+ 42884us) raft_consensus.cc:1685] finished
0709 12:27:08.699009 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.699150 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17065,"prepare.queue_time_us":30411,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":1088,"raft.queue_time_us":21381,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":15231,"spinlock_wait_cycles":19662336,"thread_start_us":45815,"threads_started":1,"wal-append.queue_time_us":104169}
I20260709 12:27:08.714370  4672 raft_consensus.cc:2468] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 20.
I20260709 12:27:08.678319 10612 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.684489 10442 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.724963  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:08.648386 10576 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:27:08.727429 10571 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.728777 10571 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:08.729602 10580 raft_consensus.cc:3055] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:08.730033 10580 raft_consensus.cc:740] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:08.730259 10576 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:08.730737 10576 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:08.731612 10576 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.733176 10576 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:08.734380  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3451) took 1017 ms. Trace:
I20260709 12:27:08.734771  4679 rpcz_store.cc:276] 0709 12:27:07.717178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.718218 (+  1040us) service_pool.cc:224] Handling call
0709 12:27:07.718611 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:07.916144 (+197533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.734133 (+817989us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b540017e154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:08.734319 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":451948800}
I20260709 12:27:08.735934  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:08.713513 10583 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:08.737439 10583 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.739598 10583 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:08.713652 10373 consensus_queue.cc:1048] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:27:08.702226 10552 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:27:08.751322  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 60712.2us
I20260709 12:27:08.756280  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:08.636374 10143 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.719336 10469 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:08.709389 10482 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:08.755090  4720 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:08.716738 10551 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:08.764488  4747 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:08.772354  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:08.726259 10612 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:08.718176 10171 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.845228  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:08.846333  4677 raft_consensus.cc:2368] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 19. Current term is 20.
I20260709 12:27:08.807796  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:08.809803  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:08.811273  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:08.823253  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.847901  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.847383  5342 raft_consensus.cc:2368] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14.
I20260709 12:27:08.877090  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:08.877884  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 78653.7us
I20260709 12:27:08.778162 10482 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:08.718745 10616 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.749979 10442 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.895249  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:08.896315  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:08.904682  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:08.905588  4677 raft_consensus.cc:2368] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 19. Current term is 20.
I20260709 12:27:08.778903 10579 consensus_queue.cc:1059] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 12:27:08.775653 10552 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.867779 10349 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.779100 10551 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.910924  4569 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:08.924503  5347 pending_rounds.cc:85] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 11
I20260709 12:27:08.932758  5347 pending_rounds.cc:107] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:27:08.933053  5347 raft_consensus.cc:2889] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:08.925745  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:08.940207  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:08.945521  4677 raft_consensus.cc:2393] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:08.893225 10229 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:27:08.948326  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:08.954388  5343 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:08.928746 10402 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:08.935531 10532 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for earlier term 17. Current term is 18.
I20260709 12:27:08.938549 10400 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:08.967298  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:08.970198  4677 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:08.973654  4645 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:08.953004 10302 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:08.962271 10623 raft_consensus.cc:3055] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:08.936334 10349 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:08.987277  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:08.989686  4644 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:08.992118  4568 leader_election.cc:400] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14.
I20260709 12:27:08.992475  4568 leader_election.cc:403] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 12:27:08.994081  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 328249us
I20260709 12:27:08.977182  9951 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:08.972289 10482 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:08.962824 10579 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:27:09.011214  4677 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:09.012506  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.013847  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.014436  4677 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:08.983477 10472 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:08.965349 10402 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:08.983744 10623 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:09.002810 10149 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:09.016301 10618 log.cc:927] Time spent T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:27:09.002231 10532 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:09.033167 10530 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.004599 10453 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.054734  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:09.078300  4603 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:09.055686 10472 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:09.065852  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:09.086405  5346 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:09.086930  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3452) took 1369 ms. Trace:
I20260709 12:27:09.065881  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:09.067868  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:09.090876  4719 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:09.061261  4753 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:09.071455  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:09.075855 10149 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.087337  4681 rpcz_store.cc:276] 0709 12:27:07.717621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.717754 (+   133us) service_pool.cc:224] Handling call
0709 12:27:09.086716 (+1368962us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b54002dfd54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:09.086867 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1858933504}
W20260709 12:27:09.125759  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 58874.7us
I20260709 12:27:09.129875  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:09.130321  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:09.130749  4603 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 17: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:09.017886 10482 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:09.135914  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 194264us
I20260709 12:27:09.136623  4569 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:09.137751  4569 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:09.143049  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 60873us
W20260709 12:27:09.117935 10622 log.cc:927] Time spent T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 12:27:09.133751  4605 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:09.118507 10400 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:09.131505 10621 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:27:09.126757 10171 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.156978  4645 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:09.141709 10532 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:09.151887 10579 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:09.151598 10528 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:09.088243 10530 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.152089 10353 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:09.126430 10149 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:09.176167  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 421127us
I20260709 12:27:09.178406  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:09.180559  4681 raft_consensus.cc:2368] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14.
I20260709 12:27:09.153752 10400 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:27:09.181824  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 46238.2us
I20260709 12:27:09.131142 10472 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:09.176872  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:09.184357 10472 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:09.181829 10400 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:09.185235  4603 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:09.189072  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.187886  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:09.151299 10537 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:27:09.184384  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:09.198426  4674 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:09.204955  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:09.176380 10353 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.178905 10551 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.176467 10616 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:09.206019  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.196796 10412 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:09.191864 10552 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.191533 10579 raft_consensus.cc:697] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:09.214869  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.200169 10129 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:09.216749  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1085) took 1424 ms. Trace:
I20260709 12:27:09.218945  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:09.177074 10532 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.219151  4754 rpcz_store.cc:276] 0709 12:27:07.792654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.956099 (+163445us) service_pool.cc:224] Handling call
0709 12:27:07.957097 (+   998us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:08.197883 (+240786us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:08.197920 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:08.282161 (+ 84241us) log.cc:844] Serialized 26 byte log entry
0709 12:27:08.616827 (+334666us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:08.616881 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.616986 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:09.216534 (+599548us) raft_consensus.cc:1685] finished
0709 12:27:09.216560 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.216694 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":214854,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":55912,"thread_start_us":196118,"threads_started":1,"wal-append.queue_time_us":457081}
I20260709 12:27:09.225240  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.226079  5348 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:09.209287 10353 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:09.230260  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:09.205668 10472 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:09.216303 10624 log.cc:927] Time spent T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.235s	user 0.001s	sys 0.001s
I20260709 12:27:09.210402 10171 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:09.206166 10639 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:09.233738 10610 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:27:09.233258 10636 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:27:09.237226 10149 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:09.238111 10632 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:27:09.238229 10349 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:09.239945  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1091) took 1402 ms. Trace:
I20260709 12:27:09.227414 10279 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:09.269683  4756 rpcz_store.cc:276] 0709 12:27:07.837793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.375269 (+537476us) service_pool.cc:224] Handling call
0709 12:27:08.375626 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:08.375953 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:08.375979 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:08.376455 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:27:08.799427 (+422972us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:08.799468 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.799547 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:09.239724 (+440177us) raft_consensus.cc:1685] finished
0709 12:27:09.239744 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.239868 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1220,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":13982,"thread_start_us":421362,"threads_started":1,"wal-append.queue_time_us":551735}
I20260709 12:27:09.271596  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.242168  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.247277  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.247756  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:09.249373 10530 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.216518 10537 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:09.275631  4756 raft_consensus.cc:2410] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 12:27:09.240491 10576 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.002s
W20260709 12:27:09.236562 10630 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:27:09.273891  4680 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:09.193302 10623 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:09.286146  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.289718  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:09.293078  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.333348  4756 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:09.337726  4753 raft_consensus.cc:2410] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:27:09.229835 10129 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:09.347914  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:09.348461  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:09.344530 10571 consensus_meta.cc:287] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.612s	user 0.014s	sys 0.004s
I20260709 12:27:09.255820 10171 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.339707 10363 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:09.357295 10612 consensus_meta.cc:287] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.005s
I20260709 12:27:09.346220 10567 consensus_queue.cc:1059] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:27:09.370469  4753 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:09.374092 10532 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:09.376739  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 125588us
I20260709 12:27:09.378029  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:09.378495  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.362732 10482 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:09.379402  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:09.383780 10639 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:09.384223  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:27:09.387420 10633 log.cc:927] Time spent T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:27:09.369771 10612 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:09.391619  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3453) took 1673 ms. Trace:
I20260709 12:27:09.392073  4676 rpcz_store.cc:276] 0709 12:27:07.718035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.718153 (+   118us) service_pool.cc:224] Handling call
0709 12:27:09.391398 (+1673245us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b54001bbf54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:09.391555 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1874133760}
I20260709 12:27:09.385926 10610 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:09.393124 10612 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.398803 10171 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:09.384259  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.402671  4753 raft_consensus.cc:2393] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:09.335146 10537 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.348276 10579 consensus_queue.cc:237] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:09.415611  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:09.397395 10251 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.674s	user 0.008s	sys 0.000s
W20260709 12:27:09.403992 10320 consensus_meta.cc:287] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.815s	user 0.011s	sys 0.001s
I20260709 12:27:09.440605 10320 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.441825  5344 raft_consensus.cc:2368] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14.
I20260709 12:27:09.442111 10537 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:09.444800 10623 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:09.403391 10532 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 12:27:09.453266 10320 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.419829 10383 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.404471 10171 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:09.458763 10171 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:09.460583  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:09.461025 10171 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.419723 10400 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.406823 10610 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:09.428928 10637 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 12:27:09.463382 10400 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:09.463451  4754 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:09.461902  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:09.469215  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 52387.7us
I20260709 12:27:09.470374  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:09.471698  4605 raft_consensus.cc:2410] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:27:09.437297  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:09.414186 10576 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:09.439452 10645 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:27:09.439713 10656 log.cc:927] Time spent T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:27:09.385545 10472 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:27:09.360984 10363 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.453697 10442 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.486490  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.417831  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:09.436276 10353 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:09.481407  4676 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:09.483695  4601 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:09.471784  4720 leader_election.cc:400] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19.
I20260709 12:27:09.487901  4753 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:09.491159  4720 leader_election.cc:403] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:09.473433 10558 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.495900  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.478248 10400 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:09.475451 10560 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.495944  4601 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:09.499784  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:09.515893  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:09.517011  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:09.484354 10171 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:27:09.480188 10576 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:09.523867  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.472724 10251 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:09.540300  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.509414 10610 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19.
I20260709 12:27:09.518083 10472 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:27:09.525364 10588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.873s	user 0.019s	sys 0.032s
I20260709 12:27:09.546125  5345 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:09.547348  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.505581 10558 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.534847 10400 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:09.511703 10560 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.558914  4753 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
W20260709 12:27:09.544590 10588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.020s	sys 0.032s
I20260709 12:27:09.543087 10453 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14.
I20260709 12:27:09.574092 10623 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.582823 10353 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.583312 10639 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:09.587791 10576 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.588006  4674 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 12:27:09.545465 10544 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:27:09.590683  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:09.596525  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 79550.8us
I20260709 12:27:09.606802  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:09.592114 10229 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:09.602252  4602 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:09.608458  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:09.600302  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.612556  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:09.614442  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:09.616487 10472 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:09.617264 10472 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:09.619493  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:09.620339  4756 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:09.618534 10610 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:09.610466 10400 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:09.638597 10560 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:09.639304  4676 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:27:09.641048  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.642647  4674 raft_consensus.cc:2410] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:27:09.643128 10251 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.643997 10229 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:09.641094 10363 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:09.658174  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.598971 10623 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:09.647544 10229 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:09.664296  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48913.1us
I20260709 12:27:09.663944 10623 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.666913  4720 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:09.668200  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:09.669129  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.669747  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.669781  4674 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
W20260709 12:27:09.664898 10664 log.cc:927] Time spent T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:27:09.670774 10610 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:09.672497 10610 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:27:09.672645 10670 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:09.671916  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.673856  4749 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:09.674538  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.672916 10610 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:09.668383 10192 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:09.671423 10400 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:09.680958 10402 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.679868 10192 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:09.683770 10558 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:09.685906  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 969) took 1150 ms. Trace:
I20260709 12:27:09.686635 10129 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.692591 10571 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.693828 10571 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:09.697427 10571 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.699945  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2882) took 1882 ms. Trace:
I20260709 12:27:09.701256  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 2885) took 1142 ms. Trace:
I20260709 12:27:09.701813  4747 rpcz_store.cc:276] 0709 12:27:08.558490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.582997 (+ 24507us) service_pool.cc:224] Handling call
0709 12:27:09.701036 (+1118039us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400117454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:09.701179 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298834304}
I20260709 12:27:09.703305  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:09.704000  4747 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:27:09.705544 10666 log.cc:927] Time spent T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:27:09.709414  4569 leader_election.cc:400] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14.
I20260709 12:27:09.709525  4597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 2883) took 1888 ms. Trace:
I20260709 12:27:09.716873  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.710079  4597 rpcz_store.cc:276] 0709 12:27:07.820968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.821153 (+   185us) service_pool.cc:224] Handling call
0709 12:27:07.821544 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:07.879682 (+ 58138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.709269 (+1829587us) spinlock_profiling.cc:243] Waited 55.9 ms on lock 0x7b54003543d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:09.709459 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128648576}
I20260709 12:27:09.725004  4747 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:27:09.726274  5314 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:09.726464  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.728005  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.681349 10400 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 17 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:09.686338  4599 rpcz_store.cc:276] 0709 12:27:08.535696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.535815 (+   119us) service_pool.cc:224] Handling call
0709 12:27:08.563494 (+ 27679us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:08.566827 (+  3333us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:08.593479 (+ 26652us) log.cc:844] Serialized 14 byte log entry
0709 12:27:09.436176 (+842697us) spinlock_profiling.cc:243] Waited 6.17 ms on lock 0x7b180019e220. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:27:09.536735 (+100559us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:09.542214 (+  5479us) log.cc:844] Serialized 26 byte log entry
0709 12:27:09.542356 (+   142us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:09.542390 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.542461 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:09.685630 (+143169us) raft_consensus.cc:1685] finished
0709 12:27:09.685716 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.685845 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4945,"prepare.queue_time_us":26827,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":6065,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":14357,"spinlock_wait_cycles":14190848,"thread_start_us":100224,"threads_started":1,"wal-append.queue_time_us":1010098}
I20260709 12:27:09.728964  4569 leader_election.cc:400] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 13. Current term is 14.
I20260709 12:27:09.732883  4603 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
I20260709 12:27:09.733971  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:09.734987  4747 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 20: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:09.736886  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:09.738341  4747 raft_consensus.cc:2410] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:27:09.700371  4600 rpcz_store.cc:276] 0709 12:27:07.818091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.818426 (+   335us) service_pool.cc:224] Handling call
0709 12:27:07.818756 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:07.870166 (+ 51410us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:08.471223 (+601057us) log.cc:844] Serialized 14 byte log entry
0709 12:27:08.615096 (+143873us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:08.615146 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:08.615185 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.699709 (+1084524us) spinlock_profiling.cc:243] Waited 51.1 ms on lock 0x7b540032c3d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:09.699758 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.699883 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31699,"prepare.queue_time_us":123934,"prepare.run_cpu_time_us":6846,"prepare.run_wall_time_us":203232,"raft.queue_time_us":12911,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":16103,"spinlock_wait_cycles":117587712,"thread_start_us":721527,"threads_started":2,"wal-append.queue_time_us":312463}
I20260709 12:27:09.717813 10409 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.729995 10400 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:09.740650 10409 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:09.744400  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.745337  4672 raft_consensus.cc:2410] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:27:09.738951 10615 raft_consensus.cc:2804] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:09.746302 10615 raft_consensus.cc:697] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:09.730986  4571 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:09.755705  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:09.757153  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:09.757949  4747 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:09.766058 10576 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:09.747166 10615 consensus_queue.cc:237] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.775041 10610 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.777227  4749 raft_consensus.cc:1240] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:09.779841  4756 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:27:09.783195  4747 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:09.783092  4569 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:09.789176 10676 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.789964 10676 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:09.765424 10229 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:09.793274 10552 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:09.793794 10552 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:09.794195 10552 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:09.769375 10674 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:09.798259  4569 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:09.758612 10409 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:09.755307 10560 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:27:09.789275 10610 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:09.803812  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 124875us
I20260709 12:27:09.806938  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.806982  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.807842  4674 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:09.808070  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:09.808300  4569 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:09.809394  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:09.799748 10674 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.818074  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:09.827157  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.827503  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:09.827992  4672 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 17: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:09.829025  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:09.831192 10644 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:09.832417 10676 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:09.834112  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.835036  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.835804  4674 raft_consensus.cc:2393] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:09.837909  4602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 2182) took 1235 ms. Trace:
I20260709 12:27:09.838455  4602 rpcz_store.cc:276] 0709 12:27:08.602417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.614766 (+ 12349us) service_pool.cc:224] Handling call
0709 12:27:08.646246 (+ 31480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:09.615994 (+969748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.837677 (+221683us) spinlock_profiling.cc:243] Waited 955 ms on lock 0x7b5400034154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:09.837839 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2196687232}
W20260709 12:27:09.841117 10642 log.cc:927] Time spent T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.494s	user 0.002s	sys 0.001s
I20260709 12:27:09.841243 10671 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.842967  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:09.812100 10639 consensus_queue.cc:1059] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:27:09.804813 10229 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:09.819401 10674 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:09.845708  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.808923 10537 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:09.810936 10171 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 12:27:09.848894  4644 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.196:39067 timed out after 1.586s (SENT) [suppressed 4 similar messages]
I20260709 12:27:09.835559  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:09.813620 10623 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:09.870805  4675 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 17: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:09.850559 10676 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.854857 10383 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.856678  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:09.858975  5348 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:09.808525 10360 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:27:09.861794  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:09.881862  5342 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:09.868391  4600 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
W20260709 12:27:09.875228  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 69327us
I20260709 12:27:09.875689  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:09.876922  4750 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1086) took 2083 ms. Trace:
I20260709 12:27:09.875331  4677 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 15. Ops: []
I20260709 12:27:09.944007  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:09.906297  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:09.948495  4672 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:09.882465  4599 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:09.950156  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:09.954434  5348 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:27:09.950909  4750 rpcz_store.cc:276] 0709 12:27:07.793028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.958432 (+165404us) service_pool.cc:224] Handling call
0709 12:27:07.958817 (+   385us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:08.285362 (+326545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:08.285410 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:08.372201 (+ 86791us) log.cc:844] Serialized 52 byte log entry
0709 12:27:08.554669 (+182468us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:08.554724 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.863186 (+1308462us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400117e54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:09.863253 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:09.863321 (+    68us) raft_consensus.cc:1685] finished
0709 12:27:09.863339 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.876857 (+ 13518us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3603,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":253,"spinlock_wait_cycles":555669248,"thread_start_us":137543,"threads_started":1,"wal-append.queue_time_us":290499,"wal-append.run_cpu_time_us":1676,"wal-append.run_wall_time_us":1012298}
I20260709 12:27:09.957370  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:09.962492  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 150218us
I20260709 12:27:09.964600  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:09.942989 10537 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.978340  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2884) took 1420 ms. Trace:
I20260709 12:27:09.983179  4569 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:09.942777 10383 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.883569 10644 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 12:27:09.995586 10531 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:09.995723 10528 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:09.996626  5345 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:09.884956 10676 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.998876 10676 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:09.992717 10302 raft_consensus.cc:3055] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:10.003947  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:10.006582  4753 raft_consensus.cc:3055] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:10.007033  4753 raft_consensus.cc:740] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:10.007838  4753 consensus_queue.cc:260] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.009009  4753 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:09.992717 10552 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.010113  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 972) took 1473 ms. Trace:
I20260709 12:27:10.010551  4604 rpcz_store.cc:276] 0709 12:27:08.536558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.536691 (+   133us) service_pool.cc:224] Handling call
0709 12:27:09.762197 (+1225506us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b5800002798. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff348ffc0 00007efff347a929 00007efff347a3d2 00007effec9f1bd4
0709 12:27:09.762452 (+   255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:09.762826 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:09.762867 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:09.766766 (+  3899us) log.cc:844] Serialized 26 byte log entry
0709 12:27:09.869988 (+103222us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:09.870033 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.870098 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:10.009764 (+139666us) raft_consensus.cc:1685] finished
0709 12:27:10.009793 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:10.010048 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":317,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":2262,"spinlock_wait_cycles":7655680,"thread_start_us":102673,"threads_started":1,"wal-append.queue_time_us":200453}
I20260709 12:27:10.012018 10552 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:10.015632  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:10.004015 10302 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:09.996778 10383 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:09.978943  4752 rpcz_store.cc:276] 0709 12:27:08.558162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.726144 (+167982us) service_pool.cc:224] Handling call
0709 12:27:08.758588 (+ 32444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:08.759044 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:08.759072 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:08.778752 (+ 19680us) log.cc:844] Serialized 26 byte log entry
0709 12:27:09.197464 (+418712us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:09.197608 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.197811 (+   203us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:09.840461 (+642650us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:27:09.967154 (+126693us) raft_consensus.cc:1685] finished
0709 12:27:09.967183 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.978245 (+ 11062us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14848,"raft.queue_time_us":95,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":19780,"spinlock_wait_cycles":3779328,"thread_start_us":296476,"threads_started":1,"wal-append.queue_time_us":412454}
I20260709 12:27:10.019148  5314 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:10.022059  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:10.034057  5345 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:10.038146  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:10.039844  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:10.041303  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:09.870496 10671 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:09.989044  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:10.054703  4720 leader_election.cc:400] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 19. Current term is 20.
I20260709 12:27:10.055439  4719 leader_election.cc:400] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14.
I20260709 12:27:10.055812  4719 leader_election.cc:403] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:10.055892  4720 leader_election.cc:400] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 19. Current term is 20.
I20260709 12:27:10.056416  4720 leader_election.cc:403] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:27:09.876693 10229 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.057782  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:10.060330  4568 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:10.060937  5347 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:10.057255  5342 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:09.883945 10623 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.080683  5345 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:09.986263 10610 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:10.059034 10279 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:10.126236  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.127295  4568 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:10.059741 10472 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:10.059383 10576 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.059624 10579 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:10.020408 10583 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 12:27:10.104888 10544 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:10.085861 10132 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:09.986819 10537 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.138757  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.140232  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.101816 10676 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14.
I20260709 12:27:10.148559  4605 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
W20260709 12:27:10.135215 10650 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:10.151507  4604 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 15: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:10.152868  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:10.175800  4600 raft_consensus.cc:2410] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:10.132751 10610 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.138883 10571 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:27:10.181480  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 196524us
I20260709 12:27:10.133049 10639 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 19. Current term is 20.
I20260709 12:27:10.186892  5341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:41342 (request call id 5) took 1506 ms. Trace:
I20260709 12:27:10.102584 10553 consensus_queue.cc:1059] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:27:10.170811 10531 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:10.211328  4605 raft_consensus.cc:3060] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:10.139245 10576 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.215636  4569 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:10.147673 10579 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.230677 10579 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:10.218135 10571 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.233965 10579 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:10.216598  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 161316us
W20260709 12:27:10.212637 10683 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.343s	user 0.001s	sys 0.000s
W20260709 12:27:10.227084 10689 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:10.142341 10302 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.238255 10302 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:10.238447 10676 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:10.239023 10676 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.239809 10623 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:10.223547  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:10.241091  4569 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:10.241529  4604 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:27:10.243613  4756 raft_consensus.cc:3055] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:10.243983  4756 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:10.217200  5341 rpcz_store.cc:276] 0709 12:27:08.680904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.681066 (+   162us) service_pool.cc:224] Handling call
0709 12:27:10.186436 (+1505370us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540055e6d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4bc735 00007effee4b49fa 00007efff347c4c9 00007effec9f8c54 00007effec9f8be2 00007effec9f8b11 00007effec9f8a8c 00007effec9f7d42 00007effec7f148c 00007effec7f08c6
0709 12:27:10.186783 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2940622080}
I20260709 12:27:10.245466  4719 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:10.147017 10580 consensus_queue.cc:260] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:10.267519  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 211785us
W20260709 12:27:10.270058  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.270125  4719 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:10.244661  4756 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.233343 10571 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:10.271162  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:10.273005 10676 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:10.279948 10537 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:10.280148 10676 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:10.273208 10671 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:10.273165 10639 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:10.281142 10639 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.281672 10639 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:10.282630  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:10.284368  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:10.242491 10551 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:10.289656 10551 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.290343 10685 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.291219 10685 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.292579 10551 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:10.273888 10657 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.889s	user 0.005s	sys 0.000s
W20260709 12:27:10.276656 10687 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:27:10.277881 10610 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:10.295035  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:10.293902 10676 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:10.308797  4756 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:10.312374  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:10.314564  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:10.317073  4605 raft_consensus.cc:2468] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:10.257432 10320 raft_consensus.cc:3055] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:10.257696 10623 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:10.294003 10671 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:10.323468  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.320844 10610 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:10.321916 10671 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.321890 10676 consensus_queue.cc:1059] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.015s
W20260709 12:27:10.352317  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 503406us
I20260709 12:27:10.337181  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.338796  4605 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
W20260709 12:27:10.346043  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 218811us
I20260709 12:27:10.354589  4568 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:10.359829 10482 raft_consensus.cc:3055] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:10.364419  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3427) took 1716 ms. Trace:
I20260709 12:27:10.365286  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.360337 10482 raft_consensus.cc:740] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:10.369823  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:10.349440 10671 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:10.322464 10580 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:10.320490 10320 raft_consensus.cc:740] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:10.347496  4604 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:27:10.365288  4755 rpcz_store.cc:276] 0709 12:27:08.647906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.080422 (+432516us) service_pool.cc:224] Handling call
0709 12:27:09.080787 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:09.081080 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:09.128606 (+ 47526us) log.cc:844] Serialized 14 byte log entry
0709 12:27:10.019992 (+891386us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b18004a55a0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:27:10.363687 (+343695us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:10.363768 (+    81us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:10.363845 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:10.364177 (+   332us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:10.364356 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":47303,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":1932,"spinlock_wait_cycles":10937728,"thread_start_us":342940,"threads_started":1}
I20260709 12:27:10.377426  4568 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:10.370626  5341 raft_consensus.cc:2376] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18. Re-sending same reply.
I20260709 12:27:10.362738 10229 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:10.403733  4568 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:10.387442 10692 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:10.379261  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 109302us
I20260709 12:27:10.406858  4719 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:10.407148  4604 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:10.374396 10610 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:10.408162 10610 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.408370  4719 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:10.408780 10610 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.346244  4756 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:10.366403 10678 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:10.372202 10676 consensus_queue.cc:1243] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:10.417362 10676 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:10.417527  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:10.417972 10676 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 17 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:10.418851  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:10.419039 10482 consensus_queue.cc:260] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.419492  4679 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:10.403746 10695 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:10.373994 10551 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.422094  4672 raft_consensus.cc:1275] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:10.420559  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:10.422262  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.422402  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.425216  4600 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
W20260709 12:27:10.423436 10400 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:27:10.435078  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:10.436020  4679 raft_consensus.cc:3060] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:10.419983 10482 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:10.421244 10695 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:27:10.443303  4485 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e8a74137ba104fe0b465cab6399c4940 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:27:10.422200 10676 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.426126 10697 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:10.454011 10697 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.454434 10697 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:10.429649 10623 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:27:10.458678 10685 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:10.426811 10537 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:10.459470 10537 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.459896 10537 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.460196  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:10.439441 10320 consensus_queue.cc:260] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.470369  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:10.471537  4675 raft_consensus.cc:2410] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 12:27:10.475435 10674 consensus_meta.cc:287] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.594s	user 0.011s	sys 0.000s
I20260709 12:27:10.476006 10674 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.477205  4601 raft_consensus.cc:2368] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17.
I20260709 12:27:10.422783 10551 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.479555  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.479709 10551 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:10.477721 10576 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:10.485798 10623 consensus_queue.cc:1243] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:27:10.499801  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 184200us
I20260709 12:27:10.503502  4597 raft_consensus.cc:1240] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 15. Ops: [13.11-13.11]
I20260709 12:27:10.506821  4672 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 20.
I20260709 12:27:10.507640  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:10.510639  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.507680 10279 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:10.524827  9740 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:10.528095  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:10.531257  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:10.531142  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:10.531410  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 122686us
I20260709 12:27:10.539881 10585 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:10.540763  4644 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:10.638924  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 98238.6us
I20260709 12:27:10.639623  4644 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:10.544827  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.642010  9740 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.641973  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.532822 10639 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.546597 10132 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.566210 10553 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:10.549758 10674 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:10.577509  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:10.583344  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:10.583678 10588 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:10.590939  5312 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:10.594368  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.624584  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.657841  4601 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:27:10.658094  4598 raft_consensus.cc:2376] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19. Re-sending same reply.
I20260709 12:27:10.596835  4679 raft_consensus.cc:2468] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14.
W20260709 12:27:10.660096  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 69314.3us
I20260709 12:27:10.661123  5312 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:10.632704  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:10.595503 10383 consensus_queue.cc:1048] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:27:10.640486  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:10.640662  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:10.541677  4673 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:27:10.631630  4678 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:10.663793  4569 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:10.663962  4678 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:10.544533 10537 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.632604  4645 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:10.646546  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:10.647429  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.650996  4676 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:10.651713  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:10.668107  4597 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:10.655874  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:10.668184  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:10.655493  4680 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:10.662123  5340 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:10.670040  9740 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:10.670820  9740 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:10.671128  4645 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:10.673630  4569 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:10.673857  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:10.668912  5348 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:10.677709  4601 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:10.677955  9740 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:10.681555  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:10.684489  4678 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:10.652972 10132 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:10.692713  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:10.693939  4751 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.686064  4569 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:10.694588  4719 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:10.699783  4604 raft_consensus.cc:2393] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 17: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:10.650203 10703 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:10.661279 10585 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:10.701926  9740 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:10.622351 10610 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:10.706625  4603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 976) took 1723 ms (client timeout 2247 ms). Trace:
W20260709 12:27:10.707113  4603 rpcz_store.cc:269] 0709 12:27:08.974625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.187117 (+212492us) service_pool.cc:224] Handling call
0709 12:27:10.697762 (+1510645us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b54002e54d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:10.697949 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":927047040}
I20260709 12:27:10.670274 10251 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:10.716642 10706 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:10.676349 10498 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:10.696542 10704 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:10.726670 10704 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.727092 10704 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:10.680197 10676 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:10.727914 10676 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.690316 10567 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:10.732844 10567 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.733520 10567 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:10.690001 10544 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:27:10.689899 10674 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:10.740566  4752 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:10.660336 10576 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:27:10.660545 10553 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.702831 10571 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:10.696210 10671 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.702123 10703 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.702375 10585 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.764216 10585 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.757083 10229 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:27:10.667320 10588 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.737857 10498 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:10.763546  4571 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:10.760566 10703 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:10.796450  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:10.797387 10697 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.798840  4605 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 18: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:10.798811  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:10.801802 10553 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:10.522599  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:10.759886 10571 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.816776  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:10.817559  4756 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
W20260709 12:27:10.818967  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.819093  4753 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:10.776854 10498 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.821200  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:10.822927  4753 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:10.765865 10537 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:10.831907  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:10.846460  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:10.848223  4753 raft_consensus.cc:3060] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:10.807789 10579 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:10.849308  4751 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:10.862095  4748 raft_consensus.cc:2393] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 18: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:10.818295 10571 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:10.817991 10674 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.867865  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:10.868398  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:10.866122  4571 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:10.824697 10551 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:10.874015  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:10.875523  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:10.833619 10676 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:10.878741  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:10.879638  4720 leader_election.cc:400] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17.
I20260709 12:27:10.880205  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:10.879931  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 198452us
I20260709 12:27:10.862612 10639 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:10.894357  4720 leader_election.cc:403] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:27:10.896453  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 195021us
I20260709 12:27:10.880928 10537 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:10.882334  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:10.897979  5349 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.883145  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:10.885082  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:10.886045 10588 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:10.876555 10674 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.864662 10671 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:10.902132  4748 pending_rounds.cc:85] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 7
I20260709 12:27:10.911479  4748 pending_rounds.cc:107] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:27:10.911767  4748 raft_consensus.cc:2889] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:10.902423  4756 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:27:10.903584  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:10.914062  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:10.907799  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 80724us
W20260709 12:27:10.909415  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 230532us
I20260709 12:27:10.915532  4753 raft_consensus.cc:1275] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:10.915664  4647 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:10.925621  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:10.939337 10671 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:10.939858 10671 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:10.940235 10671 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:10.946594 10639 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:10.947149  4600 raft_consensus.cc:3055] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:10.947582  4600 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:10.947351 10639 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.911055 10482 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:10.949332 10482 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.949744 10482 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:10.949939 10472 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:10.950462 10472 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.948464  4600 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.955312  4600 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:10.959237 10712 log.cc:927] Time spent T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:27:10.961777 10472 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:10.956351  5349 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:10.968425 10671 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.976019 10671 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:10.894275 10551 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:10.983979 10551 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:10.987830 10580 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:10.988389 10580 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:10.912531 10696 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17.
I20260709 12:27:10.977165 10674 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:11.004660 10320 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:10.999547  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:10.980705 10567 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.009737  4747 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:11.000303  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:11.011996  4755 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
W20260709 12:27:11.014483  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 92229us
I20260709 12:27:11.011497  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:10.921751 10537 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:10.949745 10676 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:10.997491 10682 log.cc:927] Time spent T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.844s	user 0.000s	sys 0.002s
I20260709 12:27:10.999845 10639 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:10.955466 10560 consensus_queue.cc:1059] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 12:27:11.043196  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 2191) took 1236 ms. Trace:
I20260709 12:27:11.043757  4599 rpcz_store.cc:276] 0709 12:27:09.806813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.807764 (+   951us) service_pool.cc:224] Handling call
0709 12:27:11.043016 (+1235252us) spinlock_profiling.cc:243] Waited 54.2 ms on lock 0x7b54002337d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:11.043136 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124553600}
I20260709 12:27:11.020460 10453 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:10.998497 10585 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.027231 10726 raft_consensus.cc:2804] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:11.031602 10696 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.067268  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:11.067978  4678 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:11.070186  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 154130us
I20260709 12:27:11.071287  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:11.073553  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:11.072032  4599 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:27:11.075891  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:11.077803  4681 raft_consensus.cc:2393] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 17: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:11.049638 10579 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:11.051862 10453 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:11.025472 10676 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.042299 10704 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.066272 10714 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:11.105535  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:11.113138 10717 log.cc:927] Time spent T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:27:11.114722  4678 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:11.077263 10583 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:11.084601 10453 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:11.123319  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 207784us
I20260709 12:27:11.086035 10726 raft_consensus.cc:697] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:11.100394 10544 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.085816 10579 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.079228 10537 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.123078 10583 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:11.077584 10696 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.127301 10580 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:11.120424 10714 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:11.185720 10597 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:27:11.179941 10723 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.189720  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 111018us
I20260709 12:27:11.138334 10571 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.132575 10544 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.189702 10567 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:11.207016  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:11.207690  4678 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:11.209471  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 81995.5us
I20260709 12:27:11.210203  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:27:11.205708  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 182131us
I20260709 12:27:11.217494  4678 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:11.225472  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:11.202682 10723 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.239440  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:11.248234  5345 raft_consensus.cc:2393] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 17: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:11.246181  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:11.238133  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:11.251039  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:11.250308 10730 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:11.255627  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:11.234265 10727 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:11.219811 10729 consensus_queue.cc:1059] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:27:11.253243  4647 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:11.225225 10585 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:11.264770 10279 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:11.265540 10704 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:11.265326 10279 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.267587 10279 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:11.228672 10724 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:27:11.270341 10544 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:11.271166 10728 log.cc:927] Time spent T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:27:11.200188 10726 consensus_queue.cc:237] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.206228 10583 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.273536 10583 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:11.276351 10706 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:11.282347 10579 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:11.256534 10725 log.cc:927] Time spent T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:27:11.255442 10716 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:11.254601 10715 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:11.226294 10721 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:27:11.339051  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:11.348145  4604 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:27:11.330260  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:11.263320  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:11.350147  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 86898.5us
I20260709 12:27:11.352177  4644 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:11.353091  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:11.353837  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:11.353893 10571 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:11.355923 10706 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:11.356042 10585 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:11.356412 10706 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:11.356585 10585 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:11.357074 10585 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:11.358670 10704 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:11.359283 10704 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:11.361135 10132 consensus_meta.cc:287] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.656s	user 0.004s	sys 0.007s
I20260709 12:27:11.354113 10639 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:11.362133 10639 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.364464 10639 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:11.353778  4605 raft_consensus.cc:2376] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 14. Re-sending same reply.
I20260709 12:27:11.369935  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:11.370735 10704 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:11.371032 10719 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:11.345803 10714 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:11.352118  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:11.371680 10719 raft_consensus.cc:697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:11.372622 10719 consensus_queue.cc:237] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.374826 10453 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.375694  4605 pending_rounds.cc:85] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 8
I20260709 12:27:11.376049  4605 pending_rounds.cc:107] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:27:11.376299  4605 raft_consensus.cc:2889] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:11.376991 10453 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:11.339303 10279 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:11.379482 10723 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:11.379519 10279 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.380430  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:11.381856 10279 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:11.356891 10706 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:11.382304 10585 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.384595 10585 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:11.386499  4569 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:11.387168  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 132574us
I20260709 12:27:11.388975  4645 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:11.391884 10610 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:11.349463  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:11.393043  5348 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:11.354544  4599 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:11.396368  4748 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:11.393528  4719 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:11.397861  4749 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:27:11.405627  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 53533.7us
I20260709 12:27:11.408392  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:11.394778  4750 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:11.409039  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:11.409950  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:11.410269  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:11.410676  4605 raft_consensus.cc:2368] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:27:11.410583  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:11.411337  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:11.411597  4596 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:27:11.411248  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:11.411257  4568 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:11.412500  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:11.409965  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:11.394330  4604 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:11.418928  4597 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:11.419656  4601 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 18: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:11.261626 10676 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:11.433300 10723 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:11.434049 10723 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:11.434707 10639 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:11.435031 10723 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.436945 10697 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:11.426393 10551 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:11.433316 10320 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:27:11.438308 10741 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:11.290382 10737 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:11.232606 10703 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.447085  4568 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:11.448925  4568 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:11.450196  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:11.452132  4748 raft_consensus.cc:2410] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 12:27:11.453040  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 188557us
I20260709 12:27:11.455070  4568 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:11.456672  4568 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:11.450022  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:11.403429 10695 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:11.477212  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:11.478169  4755 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:11.373426 10482 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.446105 10737 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:11.495478  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:11.496177  4753 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:11.484290  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:11.454196 10588 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:11.452244 10705 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:11.453145 10697 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:11.487272  4647 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:11.484452  4720 leader_election.cc:400] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:27:11.509117  4720 leader_election.cc:403] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:27:11.460793 10551 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.453900 10644 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:11.497944  4747 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:11.499297  4599 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:11.419800 10132 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.505641  4753 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:11.507440  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:11.507506  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:11.488306  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:11.523273  4750 raft_consensus.cc:2393] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 18: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:11.465991 10676 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:11.469506 10558 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:11.516453  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:11.508476 10251 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:11.522284  4602 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:27:11.493849 10544 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:11.526721  4752 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:11.508651 10705 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.501819 10737 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:11.539692  4755 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:11.522403 10551 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.483608 10695 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.539371 10705 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:11.551010  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.529176 10251 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:11.525738 10676 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:11.521931 10741 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:27:11.541543 10279 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:27:11.541168 10558 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:11.605108  5345 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:27:11.605851 10302 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:11.606493 10302 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.605868  5345 raft_consensus.cc:606] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:11.609483  5345 raft_consensus.cc:614] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:11.610904 10302 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:11.585152 10676 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:11.581252  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:11.632314  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:11.665753  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 259832us
I20260709 12:27:11.681257  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:11.689666  4752 raft_consensus.cc:2393] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 11: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:11.624222 10738 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:27:11.662318 10716 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:11.694932 10716 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.691735  4571 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:11.700907  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:11.680048 10302 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:11.702778 10302 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.689425 10727 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:11.631091 10706 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.706655  4571 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:11.714604  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:11.693843 10741 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:11.630026 10251 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:11.706422 10498 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.009s	sys 0.039s
I20260709 12:27:11.753759 10685 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:11.755571 10695 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:11.756623 10571 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:11.757300 10571 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.762904 10551 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:11.755687 10580 raft_consensus.cc:3055] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:27:11.749135 10498 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.012s	sys 0.048s
I20260709 12:27:11.764164 10580 raft_consensus.cc:740] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:11.765101 10580 consensus_queue.cc:260] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.766654 10580 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:11.764015  9740 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:27:11.698256 10703 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:11.782249  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 373493us
I20260709 12:27:11.747659  4571 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:11.730763 10528 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:11.729041 10132 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:11.776580  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:11.776595  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:11.754289 10747 consensus_queue.cc:1048] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:11.789289  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:11.790278  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 350642us
I20260709 12:27:11.775159  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:11.805871  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:11.736119 10482 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:11.753265 10730 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:11.747171 10579 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:11.783084 10571 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:11.827322  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 336169us
I20260709 12:27:11.806205 10537 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:11.829535  9740 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:11.834359  4676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:27:11.835129  4676 raft_consensus.cc:606] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
W20260709 12:27:11.835153  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 46064us
I20260709 12:27:11.835788  4676 raft_consensus.cc:614] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:11.807452 10302 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:11.836210  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 314892us
I20260709 12:27:11.837333  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:11.834939  9740 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.816957 10251 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:11.705353 10558 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.821869 10579 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:11.852576  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:11.853471  4752 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:11.859299  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:11.865046 10695 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.865765 10695 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.866519  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:11.867295  4677 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:11.862324  4752 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:11.868220 10695 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:11.870172  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:11.872790  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:11.875990  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:11.878167  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:11.878818  4674 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:11.881449  4673 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:11.893338  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:11.897285  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:11.834632 10537 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.902787  4676 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:11.903991  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:11.905164  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:11.905472  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:11.914951  4678 raft_consensus.cc:2368] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:27:11.907263  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:11.910442  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:11.819489 10719 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:27:11.920898  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:11.843930 10706 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:11.916752  9740 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:11.822686 10553 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
W20260709 12:27:11.917418  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 433184us
I20260709 12:27:11.842689 10571 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:11.845955 10721 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.923141  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 67463us
I20260709 12:27:11.922997  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:11.951488  4747 raft_consensus.cc:2410] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:27:11.919576 10537 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:11.967626  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:11.971112  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:11.911567  4674 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 21.
I20260709 12:27:11.929415  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:11.930308  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:11.936611  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:11.937198  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:11.976693  4679 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:11.937340  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:11.938414  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:11.856104 10579 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:11.946146  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:11.948118  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:11.949218  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:11.926321  4751 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:27:11.911391 10644 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.817173 10727 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.951069  4571 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:11.947273 10171 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.951850 10360 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.986402  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:11.959112 10482 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.959288 10695 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:12.009904  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:12.019085  4680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:27:12.019873  4680 raft_consensus.cc:606] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 20 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:11.955577 10721 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.987078 10703 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:12.027283  9740 consensus_queue.cc:1059] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:12.021126 10695 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:11.994124 10747 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:11.999701 10639 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:12.052675  9740 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:12.053222  9740 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:11.997916 10597 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:11.997319 10571 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:11.987713 10644 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:12.030792 10726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.009s	sys 0.019s
I20260709 12:27:12.055743 10597 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:12.061268 10744 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:27:12.061972  9740 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.078670  9740 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:12.061228 10571 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:11.994969 10171 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:12.014726 10482 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:12.072007  4677 raft_consensus.cc:2410] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 12:27:12.074052  4646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.005s	sys 0.017s
W20260709 12:27:12.098730  4646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.005s	sys 0.017s
I20260709 12:27:12.000659 10360 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.052996 10747 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:12.069826  4673 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:27:12.053186 10639 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:12.101154  4680 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:27:12.075886 10726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.010s	sys 0.020s
I20260709 12:27:12.108291  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:12.078547 10597 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:12.118634  4680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53908897ce974579947af90acaf0aed7"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:27:12.123472  4680 raft_consensus.cc:606] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:12.123853  4680 raft_consensus.cc:614] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:12.120343  4674 raft_consensus.cc:1275] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:27:12.119529  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:12.128129  4676 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 18: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:12.167223  4752 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:12.101265 10685 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:12.106086 10747 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:12.176865  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 68898.1us
I20260709 12:27:12.109371 10639 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:12.176735  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 166887us
I20260709 12:27:12.075114 10705 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:12.184300  4605 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:12.117821 10765 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.211818  4674 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:12.124173 10757 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:12.205302  4680 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:27:12.202967  5312 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:12.204260  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:27:12.181951 10728 log.cc:927] Time spent T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:27:12.221310  4605 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:12.193238 10745 log.cc:927] Time spent T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.271s	user 0.001s	sys 0.001s
I20260709 12:27:12.230554  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:12.185457 10763 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.224654  4676 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:12.225612  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:12.225828  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:12.226938  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:12.227667  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:12.228173  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:12.224048  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:12.274855  4605 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:12.277248  4752 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:12.279336  9740 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:12.285509  4674 raft_consensus.cc:2410] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
W20260709 12:27:12.278492  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49960.1us
I20260709 12:27:12.297928  5312 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:12.212857 10626 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.080s
I20260709 12:27:12.234700 10756 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:27:12.358235  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 60431.9us
W20260709 12:27:12.315449 10764 log.cc:927] Time spent T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.302s	user 0.001s	sys 0.001s
I20260709 12:27:12.236956 10132 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:27:12.368470  5345 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:27:12.373027  5345 raft_consensus.cc:606] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:12.312774 10767 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:12.224347 10765 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.355803 10756 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:12.389190  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 161076us
I20260709 12:27:12.308725 10722 consensus_queue.cc:1048] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 12:27:12.372418 10626 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:12.400733  4605 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
W20260709 12:27:12.390262 10676 consensus_meta.cc:287] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.764s	user 0.008s	sys 0.004s
I20260709 12:27:12.384290 10786 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:12.279353 10763 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:12.395907 10695 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:12.308102 10685 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:12.430544 10715 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 1.069s	user 0.007s	sys 0.006s
W20260709 12:27:12.423399 10761 log.cc:927] Time spent T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.318s	user 0.001s	sys 0.001s
I20260709 12:27:12.394834 10756 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:12.424011 10757 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 17 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:12.405665 10706 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.039s
I20260709 12:27:12.455207  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:12.437281 10482 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:12.439940 10644 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:12.437656 10786 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:12.442010 10360 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:12.454104 10715 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:12.389798 10767 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.435024 10676 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:12.468305  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 43593.2us
W20260709 12:27:12.473482  4569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.002s	sys 0.013s
I20260709 12:27:12.490528  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:27:12.510957  4569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.002s	sys 0.022s
I20260709 12:27:12.457490 10695 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:12.508500  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 149273us
I20260709 12:27:12.463156 10705 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:12.465082 10766 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
W20260709 12:27:12.514833 10779 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:27:12.515149 10776 log.cc:927] Time spent T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:27:12.515399 10762 log.cc:927] Time spent T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.236s	user 0.001s	sys 0.001s
W20260709 12:27:12.516053 10783 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 12:27:12.516742 10774 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.213s	user 0.001s	sys 0.001s
I20260709 12:27:12.517068  4754 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:12.516824 10792 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:12.517726 10780 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:12.518426 10757 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.521807 10696 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:12.524070 10571 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:12.535880  4719 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:12.667475  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 131728us
I20260709 12:27:12.536932  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:12.537839  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:12.540540  5341 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:12.543610 10721 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:12.554885 10765 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:12.557175 10784 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.557310 10794 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:12.557529 10320 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:12.557912 10799 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.470744 10786 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:12.560763 10782 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.151s	user 0.001s	sys 0.001s
I20260709 12:27:12.561244 10676 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:12.563865 10580 raft_consensus.cc:993] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:12.588888  5343 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:12.677297 10780 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:12.589964  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:12.596989  4755 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 12:27:12.593020  4678 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 12:27:12.605845  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:12.620879  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:12.584988 10626 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:27:12.648439  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:27:12.584463 10716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.011s	sys 0.049s
I20260709 12:27:12.652534  5347 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:12.660889  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:12.672254  4753 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:27:12.708397  5346 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:27:12.708986  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:12.648720 10781 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:12.684280  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:12.709983  5348 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:12.709852 10781 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.735466  4753 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 18: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:12.698843 10171 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:12.742972 10579 consensus_meta.cc:287] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.749s	user 0.012s	sys 0.000s
I20260709 12:27:12.743470 10579 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:12.747634 10741 consensus_meta.cc:287] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.977s	user 0.012s	sys 0.000s
I20260709 12:27:12.748097 10741 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:12.749730 10251 consensus_meta.cc:287] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.899s	user 0.007s	sys 0.006s
I20260709 12:27:12.712318 10721 raft_consensus.cc:1081] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 20 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
W20260709 12:27:12.747488 10747 consensus_meta.cc:287] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.545s	user 0.012s	sys 0.000s
W20260709 12:27:12.753418  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 56619.3us
I20260709 12:27:12.756614  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:12.686357  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:12.683718 10597 consensus_meta.cc:287] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.519s	user 0.005s	sys 0.010s
I20260709 12:27:12.700007  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:12.585965  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 95536.6us
I20260709 12:27:12.705418  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:12.653121 10705 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:12.687585 10320 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:27:12.710896 10716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.011s	sys 0.049s
I20260709 12:27:12.690040 10784 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.719329  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:12.723237 10763 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:12.764457  4720 leader_election.cc:400] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15.
I20260709 12:27:12.776031  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:12.754015 10797 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:12.778096  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 75425.7us
W20260709 12:27:12.708004 10737 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 1.118s	user 0.006s	sys 0.009s
I20260709 12:27:12.753736 10747 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:12.763161  4756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43876
I20260709 12:27:12.765468 10767 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:12.810920  4756 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:12.816414  4756 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:12.765072 10697 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:12.813547  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:27:12.776791 10787 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:27:12.767398 10571 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:12.824800  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 48860.1us
I20260709 12:27:12.754814 10796 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 12:27:12.695591 10799 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:12.831061  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:12.835825  4681 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:12.841758  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:12.842584  4747 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:12.838063  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:12.824044 10779 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:27:12.769547 10705 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.846289  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:12.789259 10579 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:12.770129 10786 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.846527  4571 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:12.770023 10597 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.834579 10806 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:12.811381 10737 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.838414 10767 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.865613  4750 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 11.
W20260709 12:27:12.862571 10537 consensus_meta.cc:287] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.894s	user 0.013s	sys 0.000s
I20260709 12:27:12.879076  4752 raft_consensus.cc:2410] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:27:12.839133 10676 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:12.838743 10644 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.827746 10229 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:27:12.860340 10805 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:12.846906  4756 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.881947 10757 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:12.877823 10583 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:12.891110 10583 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:12.891541 10583 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:12.883723 10806 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:27:12.884357 10759 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.804s	user 0.003s	sys 0.000s
I20260709 12:27:12.884661 10784 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:12.859261 10383 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:27:12.886550 10695 raft_consensus.cc:3055] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:12.812379 10797 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.828042 10697 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:12.864270 10792 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:12.844810  4485 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 251578us
I20260709 12:27:12.890087  4600 raft_consensus.cc:2393] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 17: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:27:12.890551  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 136373us
I20260709 12:27:12.868585 10685 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.897306  4675 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:12.918267  4597 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 17. Current term is 18. Ops: [16.8-17.9]
I20260709 12:27:12.919312  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:12.839542 10571 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.884696  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:12.929312  4598 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:27:12.929942  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:12.896718 10714 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:27:12.931967  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:12.932664 10714 raft_consensus.cc:3055] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:12.932679  4598 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 10.
I20260709 12:27:12.933319 10714 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:12.934223 10714 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.935428 10714 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:12.935745 10748 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.988s	user 0.003s	sys 0.000s
I20260709 12:27:12.936987 10615 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:12.937429  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:12.934252 10799 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:12.938423 10796 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:12.896445 10644 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.940846 10644 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:12.922016 10685 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.944664 10685 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:12.946040  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:12.948872  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.923072  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:12.898430 10695 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:12.960035  4602 raft_consensus.cc:1275] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:27:12.920399 10808 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:27:12.962442  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:12.901400 10805 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 21 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:12.956780  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:12.963217  4747 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:12.952106  5312 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:12.964766 10703 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.888712 10767 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:12.966562  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 43563.6us
I20260709 12:27:12.966427  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:12.963706  4597 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:12.978147  4602 raft_consensus.cc:2410] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:27:12.969099  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:12.979177  5348 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:27:12.969709  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:12.970794  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:12.973073  4645 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:12.889873 10537 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:12.961364 10639 consensus_meta.cc:287] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.754s	user 0.012s	sys 0.001s
I20260709 12:27:12.985600  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:12.988466  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:12.993017  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:13.000005  4603 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:12.982820  4719 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:12.941264 10583 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:12.954911 10704 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:12.949779 10780 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:12.965276 10703 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:12.985160 10696 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:12.984143 10585 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:12.986073 10799 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:12.949975 10814 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:27:12.993911 10722 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:12.997557 10757 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:13.024223  4605 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:13.001736 10738 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:12.991107 10792 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 12:27:13.062891 10792 raft_consensus.cc:3055] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:27:13.063084 10807 log.cc:927] Time spent T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 12:27:13.063361 10792 raft_consensus.cc:740] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:13.079839 10784 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:13.008833 10806 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:13.080602 10785 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:13.080857 10806 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.081346 10806 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:13.081282 10785 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.082902 10537 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:13.084884 10767 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:13.066648 10812 log.cc:927] Time spent T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:13.085822 10805 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:13.074301 10809 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 12:27:13.089476 10785 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:13.028471 10799 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.091306 10703 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:13.024499 10747 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:13.092680 10799 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:13.093920 10583 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:13.095530 10797 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:13.026635 10696 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:13.097285 10696 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.099000 10763 consensus_queue.cc:1059] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:13.029484 10722 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.023690 10585 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.100801 10722 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:13.102811 10585 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:13.104357 10697 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:12.994701 10639 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.010347 10695 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.158104  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:13.159247  5346 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 17: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:13.017593 10780 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.078126 10597 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:13.079581 10571 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:13.186766  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 214438us
I20260709 12:27:13.126941  4673 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:13.154513  4675 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:13.123494  4603 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:13.188544  4645 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:13.058061 10757 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:13.197424  4748 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:13.181193 10814 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.195474 10757 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:13.245510  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48416us
I20260709 12:27:13.246975  4645 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:13.222287 10814 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.195799 10585 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:13.254379 10747 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:13.255517  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:13.195158 10737 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:13.280284 10780 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:13.288811 10714 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:13.289695  4645 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:13.290805 10765 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:13.292006 10639 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:13.304205 10814 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:13.269263 10756 consensus_meta.cc:287] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.809s	user 0.009s	sys 0.006s
W20260709 12:27:13.381848  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 46230.5us
I20260709 12:27:13.382997  4645 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:13.398392  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:13.399055  5342 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:13.400942  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.401726  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:13.401818  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:12.707518  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:13.406574  4720 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:13.408742  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 213731us
I20260709 12:27:13.404472  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:13.404554  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:13.411082  5340 raft_consensus.cc:1240] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:27:13.413852  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:13.414635  5340 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:13.419432  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:13.437702  4719 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:13.438575  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:13.442050  5342 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:13.443042  4601 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:13.448060  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:13.451668  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:13.297469 10806 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.396462 10737 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.448972  4680 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:13.462620  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:13.462843  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:13.463712  4676 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:13.403759 10714 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:13.462106  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:13.468358  4681 raft_consensus.cc:2393] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 17: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:13.409698 10765 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:13.381760 10747 consensus_queue.cc:1048] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 12:27:13.418592 10805 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:13.442176 10780 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:13.338100 10756 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.450254 10785 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:13.490852  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:27:13.503929  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 251864us
I20260709 12:27:13.434672 10639 consensus_queue.cc:1059] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:27:13.472849 10704 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:13.471052 10799 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:13.470726 10765 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:13.516332  5346 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:27:13.470855 10737 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.504026 10785 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:13.529193  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1124) took 1426 ms. Trace:
I20260709 12:27:13.529870  4751 rpcz_store.cc:276] 0709 12:27:12.102625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.102786 (+   161us) service_pool.cc:224] Handling call
0709 12:27:12.103105 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:12.302179 (+199074us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:12.351499 (+ 49320us) log.cc:844] Serialized 14 byte log entry
0709 12:27:12.688630 (+337131us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:12.689240 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:27:12.689445 (+   205us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:12.689524 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.519734 (+830210us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b54000a7b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:13.519818 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:13.519892 (+    74us) raft_consensus.cc:1685] finished
0709 12:27:13.528882 (+  8990us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:13.529103 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79203,"prepare.queue_time_us":49842,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2314,"raft.queue_time_us":305862,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":11734,"spinlock_wait_cycles":36692480,"thread_start_us":519971,"threads_started":2,"wal-append.queue_time_us":482589}
I20260709 12:27:13.472901 10714 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.512777 10610 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:13.515926 10704 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.520483 10805 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.521611 10765 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:13.549780 10722 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.534848 10785 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:13.539326 10610 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:13.553922 10610 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:13.555141 10722 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:13.541436 10704 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:13.564049 10737 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:13.566531 10714 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:13.578369 10823 log.cc:927] Time spent T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:27:13.580837  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:13.581143  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:13.596259  5344 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:13.591940 10822 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:27:13.597828  5341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:13.523953 10799 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.599903  4719 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:13.604096 10639 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:13.584896  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:13.606072  5349 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 16: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:13.584249  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:13.583712  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:13.608237 10320 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.609616 10229 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:13.610268 10229 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:13.612743 10803 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.744s	user 0.005s	sys 0.017s
I20260709 12:27:13.613014 10229 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:13.618265  4568 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:13.619863 10705 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:13.621955 10576 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:13.623507  4568 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:13.629966 10786 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:13.546687 10805 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.591150 10756 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:13.589149  4756 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:13.581957  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 947) took 1478 ms. Trace:
I20260709 12:27:13.638939 10567 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:13.639726 10567 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:13.640331 10567 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:13.647068 10826 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:27:13.625272 10579 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:13.649534 10579 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:13.650748 10579 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.652009 10579 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:13.639549  4672 rpcz_store.cc:276] 0709 12:27:12.103640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.103786 (+   146us) service_pool.cc:224] Handling call
0709 12:27:12.104103 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:12.112608 (+  8505us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:12.566143 (+453535us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b18003f6b80. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c9557 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:27:12.907015 (+340872us) spinlock_profiling.cc:243] Waited 7.09 ms on lock 0x7b18003f6b80. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c6455 00007efff1eae3c7 00007effee4c09a7 00007effee4d00bf 00007effee4d0038
0709 12:27:13.016014 (+108999us) log.cc:844] Serialized 14 byte log entry
0709 12:27:13.396476 (+380462us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:13.407816 (+ 11340us) log.cc:844] Serialized 26 byte log entry
0709 12:27:13.435055 (+ 27239us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:13.435100 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.435182 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:13.581695 (+146513us) raft_consensus.cc:1685] finished
0709 12:27:13.581728 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:13.581902 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":146736,"prepare.queue_time_us":238836,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":6745,"raft.queue_time_us":143489,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":70965,"spinlock_wait_cycles":29192704,"thread_start_us":304058,"threads_started":2,"wal-append.queue_time_us":342138}
I20260709 12:27:13.671144 10251 raft_consensus.cc:3055] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:13.671676 10251 raft_consensus.cc:740] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:13.672567 10251 consensus_queue.cc:260] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.675585 10805 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:13.676400 10320 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:13.680207 10785 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.684496 10610 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.685864 10792 consensus_queue.cc:260] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.601475 10799 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:13.687254 10792 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:13.689666  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 66272.5us
I20260709 12:27:13.690652  4568 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:13.691622 10537 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:13.692081 10537 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:13.692533 10537 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:13.693174  4675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3514) took 1822 ms (client timeout 2178 ms). Trace:
W20260709 12:27:13.693660  4675 rpcz_store.cc:269] 0709 12:27:11.872113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.872508 (+   395us) service_pool.cc:224] Handling call
0709 12:27:13.692982 (+1820474us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001d2ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:13.693105 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2354346368}
I20260709 12:27:13.698899  4568 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:13.699566 10803 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.086s	user 0.000s	sys 0.002s
I20260709 12:27:13.703161 10806 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:13.703752  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:13.704540  5345 raft_consensus.cc:2393] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 12: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:13.705965 10537 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.708482 10537 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:13.708827  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.709059 10537 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:13.713013  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 2924) took 1437 ms. Trace:
I20260709 12:27:13.713534  4748 rpcz_store.cc:276] 0709 12:27:12.276012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.276183 (+   171us) service_pool.cc:224] Handling call
0709 12:27:12.276641 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:13.197993 (+921352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.712697 (+514704us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b5400264154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:13.712922 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2110653312}
I20260709 12:27:13.714143 10704 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:13.716354  5347 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53808 (request call id 700) took 2368 ms (client timeout 1912 ms). Trace:
I20260709 12:27:13.716620 10704 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:13.716847  5347 rpcz_store.cc:269] 0709 12:27:11.348423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.351869 (+  3446us) service_pool.cc:224] Handling call
0709 12:27:13.716141 (+2364272us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54004c5ed4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:13.716246 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2592397184}
I20260709 12:27:13.718572 10781 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:13.719336  4600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 2214) took 1494 ms (client timeout 1508 ms). Trace:
W20260709 12:27:13.719708  4600 rpcz_store.cc:269] 0709 12:27:12.225472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.225612 (+   140us) service_pool.cc:224] Handling call
0709 12:27:13.719160 (+1493548us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b5400300a54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:13.719283 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1402390784}
I20260709 12:27:13.723348  4604 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:13.725732 10251 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:13.727126 10785 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:13.728048 10821 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.472s	user 0.002s	sys 0.002s
I20260709 12:27:13.731244  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:13.731881  4753 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:13.793292  5344 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
W20260709 12:27:13.818198  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 69701.9us
I20260709 12:27:13.764864  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:13.775048  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:13.773866  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:13.853865  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:13.854498  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:13.849555  4598 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:13.774977  4597 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:13.854707  4599 raft_consensus.cc:2393] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:13.859678 10695 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:13.745966  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:13.863197  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:13.863631  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:13.734874  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:13.865302  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:13.774597  4677 raft_consensus.cc:1240] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:13.864588  4752 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:13.866082  4600 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:13.821344  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:13.873749  5346 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:27:13.740463 10757 consensus_meta.cc:287] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.000s
W20260709 12:27:13.759091  4485 catalog_manager.cc:4755] TS dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): ChangeConfig:REMOVE_PEER RPC failed for tablet e8a74137ba104fe0b465cab6399c4940: Remote error: Service unavailable: leader transfer in progress
I20260709 12:27:13.806459  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:13.734261  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:13.796104  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:13.879119  4601 raft_consensus.cc:2368] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
I20260709 12:27:13.801829  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:13.806005  4748 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:13.868606  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:27:13.867946  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 377193us
I20260709 12:27:13.764317 10585 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.892940  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:13.887593  4756 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 21 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 12:27:13.830502  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:13.869360  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
W20260709 12:27:13.847563  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63301.9us
I20260709 12:27:13.894407  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:13.772701  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:13.871922  4599 raft_consensus.cc:1240] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:13.896119  4751 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:13.778682  4495 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 166) took 18209 ms (client timeout 15000 ms). Trace:
I20260709 12:27:13.808466  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:13.875648  4753 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:13.894402  4747 raft_consensus.cc:2393] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 9: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:13.863899  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:13.897228  4599 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:13.895462  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:13.899946  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 127316us
I20260709 12:27:13.773561  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:13.896211  4754 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:13.867300 10560 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:13.927765  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:13.743149 10610 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:13.903658  4755 raft_consensus.cc:2393] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 12: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:27:13.924220  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 60445.5us
W20260709 12:27:13.872761 10696 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.071s	sys 0.123s
I20260709 12:27:13.930471  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:13.756383  4499 catalog_manager.cc:5697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 6 to 12, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 12 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:13.733465 10567 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.776077 10749 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.121s
I20260709 12:27:13.873454 10814 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.874547 10837 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:13.837733 10765 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.880393 10757 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.959126 10757 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:13.943547  4499 catalog_manager.cc:5697] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 6 to 7, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 7 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:13.942514 10837 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.960232  4499 catalog_manager.cc:5697] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:13.947605  4603 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:27:13.948479 10580 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:13.949183 10727 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:13.962002 10580 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.962538 10727 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.933928 10780 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:13.963855 10780 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:13.964033 10580 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:13.946930  4747 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:13.964721 10727 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:13.867786 10758 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:27:13.965934 10758 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:13.923316 10585 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.966295 10758 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:13.964773 10780 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.967276 10758 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.967965 10585 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:13.968587 10758 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:13.934505 10697 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:27:13.970353  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:13.969903  4499 catalog_manager.cc:5697] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 7 to 10, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 10 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:13.970991  4755 raft_consensus.cc:2368] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12.
I20260709 12:27:13.971132 10814 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.972638  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:13.933806 10741 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:27:13.975346 10727 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:13.975714 10727 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:13.976639 10727 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:13.977787 10727 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:13.980942 10741 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.954030  4644 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:13.981483 10741 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:13.982789 10836 log.cc:927] Time spent T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.238s	user 0.003s	sys 0.001s
I20260709 12:27:13.983093  4644 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:13.983382 10838 log.cc:927] Time spent T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 12:27:13.983882 10741 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:13.984025 10585 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:13.984459 10741 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:13.983877 10705 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:13.980430  5344 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:13.986152 10585 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:13.986574 10585 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:13.972673  4499 catalog_manager.cc:5697] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 6 to 11, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 11 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:13.987562 10585 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:13.987838  4499 catalog_manager.cc:5697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 8 to 11, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 11 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:13.988925 10585 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:13.981307 10814 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:13.940054 10551 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:13.994465 10551 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.952347 10567 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:13.959177 10765 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:13.994839 10551 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:13.961311 10837 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:13.995895  4644 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:13.957010  4752 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:13.997839 10741 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:14.001195  5314 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.002097 10747 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:13.935591 10738 consensus_queue.cc:1059] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:27:14.003429 10831 consensus_queue.cc:1059] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:27:13.935593 10696 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.071s	sys 0.123s
I20260709 12:27:13.932475 10716 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:13.997442  4499 catalog_manager.cc:5697] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:14.019351  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.020897  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:14.040582  5346 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 17: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:14.049481  4676 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:27:13.893947  4485 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 134931us
I20260709 12:27:14.028357  4679 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:27:14.029915  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.030216  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:14.030889  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.031838  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.032908  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:14.034322  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.034830 10498 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:13.923534 10737 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:14.033841  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:14.036962  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.004204  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:13.933099 10816 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:14.053238  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:14.056450  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.059973  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.090353  5346 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:14.039629  4571 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:14.058298  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:14.058589  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:14.058311  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.059721  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:14.060884  4600 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:14.060573  4755 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 17 LEADER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:14.060547  5347 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 17. (term mismatch)
I20260709 12:27:14.027983  4751 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:14.062285  4672 raft_consensus.cc:2410] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
W20260709 12:27:14.090242  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 86130.8us
I20260709 12:27:14.057307  4681 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 9.
I20260709 12:27:14.119349  4601 raft_consensus.cc:3060] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:14.119782  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 80228.2us
I20260709 12:27:14.122394  4568 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:14.119911  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.121131  5347 pending_rounds.cc:85] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 16
I20260709 12:27:14.122948  4752 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:14.123257  5347 pending_rounds.cc:107] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 20.17
I20260709 12:27:13.934635 10560 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.120817  5312 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:14.026648 10741 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:14.123515  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:14.123611  5347 raft_consensus.cc:2889] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:14.064154  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:14.136250  4603 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:14.124910  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.138083  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:14.138314  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.138612  4680 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:14.077914  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:14.090600  4679 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:14.139400  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.140117  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:14.126214  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.124994  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:14.129092  4602 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 14.
I20260709 12:27:14.141839  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:14.143080  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:14.143802  4756 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 20: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:14.144203  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:14.144654  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.145274  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:14.135241  4571 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:14.145799  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.131199  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:14.146857  4674 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:14.148020  4755 raft_consensus.cc:1240] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:14.111218  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:14.132472  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.149550  4751 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 9.
I20260709 12:27:14.139029  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:14.145943  4676 raft_consensus.cc:3055] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:14.151646  4676 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:14.151855  4753 raft_consensus.cc:2368] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19.
I20260709 12:27:14.026815 10676 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:14.155184  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.155572  5344 raft_consensus.cc:2368] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12.
I20260709 12:27:14.133721 10360 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:14.147109 10792 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:27:14.159744 10132 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:27:14.160104  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 138792us
I20260709 12:27:14.133337 10763 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:14.162031 10763 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.164070 10763 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:14.166496  4603 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 12:27:13.904109  4495 rpcz_store.cc:269] 0709 12:26:55.569162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.569471 (+   309us) service_pool.cc:224] Handling call
0709 12:27:13.778538 (+18209067us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:05.741041 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:05.866247 (+125206us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:05.866297 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:27:05.866337 (+    40us) tablet.cc:662] Decoding operations
0709 12:27:05.867584 (+  1247us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:05.867688 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:05.867718 (+    30us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:27:05.868882 (+  1164us) tablet.cc:701] Row locks acquired
0709 12:27:05.868924 (+    42us) write_op.cc:260] PREPARE: finished
0709 12:27:05.869132 (+   208us) write_op.cc:270] Start()
0709 12:27:05.869280 (+   148us) write_op.cc:276] Timestamp: P: 1783600025869067 usec, L: 0
0709 12:27:05.869319 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:27:05.935643 (+ 66324us) log.cc:844] Serialized 5465 byte log entry
0709 12:27:07.019970 (+1084327us) op_driver.cc:464] REPLICATION: finished
0709 12:27:08.415380 (+1395410us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effeb3ac0ac 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007efff1eecef0
0709 12:27:08.925954 (+510574us) write_op.cc:301] APPLY: starting
0709 12:27:08.938878 (+ 12924us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:09.447935 (+509057us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:09.478146 (+ 30211us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:10.016001 (+537855us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:10.016374 (+   373us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 12:27:10.016468 (+    94us) write_op.cc:312] APPLY: finished
0709 12:27:10.016950 (+   482us) log.cc:844] Serialized 121 byte log entry
0709 12:27:10.250271 (+233321us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:10.269081 (+ 18810us) write_op.cc:454] Released schema lock
0709 12:27:10.273962 (+  4881us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:10.274426 (+   464us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:10.274458 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2872899,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":1858782,"apply.run_cpu_time_us":42694,"apply.run_wall_time_us":1370137,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58389,"num_ops":13,"prepare.queue_time_us":191029,"prepare.run_cpu_time_us":23576,"prepare.run_wall_time_us":374419,"raft.queue_time_us":504299,"raft.run_cpu_time_us":26076,"raft.run_wall_time_us":1688304,"replication_time_us":1150458,"spinlock_wait_cycles":62037248,"thread_start_us":2437639,"threads_started":5,"wal-append.queue_time_us":752156,"wal-append.run_cpu_time_us":33878,"wal-append.run_wall_time_us":2656111}]]}
I20260709 12:27:14.167832  4645 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:14.134594 10703 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:14.170317  4603 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:14.171185 10703 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:14.171658 10703 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:14.171994  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:14.173466  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:14.174240  4499 catalog_manager.cc:5697] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 8 to 11, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 11 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:14.175967  4499 catalog_manager.cc:5697] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 6 to 10, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 10 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:14.176381 10551 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.176759 10571 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:14.177243 10571 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:14.174211  4601 raft_consensus.cc:2468] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 12.
I20260709 12:27:14.179314 10560 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:14.179945 10571 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:14.180346  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.182591  4674 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:14.184033  4674 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:27:14.161506 10553 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:14.171316 10579 consensus_meta.cc:287] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.519s	user 0.008s	sys 0.004s
I20260709 12:27:14.169430  4600 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 21.
I20260709 12:27:14.191438  4673 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:14.209462  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:14.063535 10716 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:14.155872  4672 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:27:14.198331  4604 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:14.201082  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.061470 10727 raft_consensus.cc:3055] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:14.204949  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:14.063375 10832 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:14.099164 10816 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:14.152746  4676 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.259966  4676 raft_consensus.cc:3060] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:14.144838 10807 log.cc:927] Time spent T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:27:14.128805 10696 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:14.206861  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 69437.5us
I20260709 12:27:14.263234  4720 leader_election.cc:400] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
I20260709 12:27:14.265311  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:14.266007  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:14.098256 10580 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:14.245724  4681 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:14.268083  4720 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:14.256292  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.256388  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.144001 10644 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:14.227566  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.206269 10626 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.186381 10553 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.091361 10723 consensus_queue.cc:1059] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.183s
I20260709 12:27:14.276247  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:14.277684  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.211303 10360 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:14.283707 10615 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:14.284613 10610 raft_consensus.cc:2804] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:14.284343 10360 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.287324  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.288166  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.273607  4720 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:14.207404 10498 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:14.262872 10816 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:14.290436  4754 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:14.293609  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:14.310786  4602 raft_consensus.cc:2393] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 13: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:14.296166  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.312306  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.317925  4678 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:14.275067 10644 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:14.313412  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.297947  4674 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 16: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:14.246969 10737 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:14.178292 10453 consensus_queue.cc:1059] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:27:14.257524 10749 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:14.343145 10749 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:14.343716 10749 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:14.345726  4749 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:14.346124  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.247561 10727 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:14.307325  4676 raft_consensus.cc:2468] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 21.
I20260709 12:27:14.279585 10780 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:14.324247  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.351147  4681 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:27:14.351476  4673 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:14.331148  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.324687  4604 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:14.336782 10453 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:14.292276 10610 raft_consensus.cc:697] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 12 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:14.317632 10696 raft_consensus.cc:2804] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:14.266863 10738 consensus_queue.cc:1059] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:27:14.335276 10704 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:14.351598  4681 pending_rounds.cc:85] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 10
I20260709 12:27:14.388917  4681 pending_rounds.cc:107] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:27:14.389313  4681 raft_consensus.cc:2889] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:14.277231 10792 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:27:14.316305 10835 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:14.322428 10781 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:14.277231 10580 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.332527 10644 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:14.286919 10626 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.341364  4719 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:14.354121 10453 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:27:14.411875  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 165461us
I20260709 12:27:14.314752 10498 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.447337  4647 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:27:14.448226  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 106988us
W20260709 12:27:14.428094  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 251698us
I20260709 12:27:14.262384 10832 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.387490 10132 raft_consensus.cc:697] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:14.452389  4681 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:14.453486  4571 leader_election.cc:400] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12.
I20260709 12:27:14.453927  4571 leader_election.cc:403] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:27:14.457383  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.475139  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.458640  4752 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:14.315259 10741 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.354475 10737 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.461066  4647 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.446657 10615 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:14.462325 10843 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 12:27:14.462878 10839 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
W20260709 12:27:14.463642 10842 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:27:14.349336 10727 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.434208 10696 raft_consensus.cc:697] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 21 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:14.469859 10551 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:14.425450 10757 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:14.423512 10797 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:14.476961  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:14.479353  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.461493 10553 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:14.491513  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:27:14.461598 10845 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:27:14.554340  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.555150  4673 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
W20260709 12:27:14.556020  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 224951us
I20260709 12:27:14.500339  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:14.419581 10716 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:14.558604  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 67187.4us
I20260709 12:27:14.505992  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.470888 10676 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:14.518395  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:14.524564  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.495338 10824 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.471094 10695 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 12:27:14.502923 10567 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12.
I20260709 12:27:14.526032 10765 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:14.510291 10741 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:14.532917  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:14.543234  4568 leader_election.cc:400] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 11 election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12.
I20260709 12:27:14.585680  4647 leader_election.cc:400] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19.
I20260709 12:27:14.585991  4647 leader_election.cc:403] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 12:27:14.501099  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 51765.5us
I20260709 12:27:14.561813  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.588562  4571 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:14.559693  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 172142us
I20260709 12:27:14.590679  4571 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:14.590831  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:14.543898 10851 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:14.517517 10580 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:14.591550  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:14.510082 10615 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.599853  4680 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:14.583309 10585 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:14.610767 10585 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:14.612388 10498 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:14.537905 10795 raft_consensus.cc:3055] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:14.597328 10767 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:14.618173 10795 raft_consensus.cc:740] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:14.618402 10767 raft_consensus.cc:697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:14.619141 10795 consensus_queue.cc:260] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.619349 10767 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.502588 10610 consensus_queue.cc:237] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.620980 10795 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:14.559458 10757 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:14.473629 10453 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.589529 10676 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:14.561440  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:14.592464 10553 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:14.601847 10837 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:14.604758 10579 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:14.609896 10857 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.623955  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.666777  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:14.580360  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 145428us
I20260709 12:27:14.589247  4678 raft_consensus.cc:2376] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19. Re-sending same reply.
I20260709 12:27:14.558388 10797 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.559759 10571 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.679739  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.636881 10453 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:14.637272 10863 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.639286 10696 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.584966 10824 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.700111 10824 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:14.640697 10737 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:14.702525 10579 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:14.703606 10579 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:14.642108  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.701359  5344 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 21.
I20260709 12:27:14.587738 10781 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:27:14.642993 10758 raft_consensus.cc:3055] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:14.644673 10576 raft_consensus.cc:3055] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:14.712535 10576 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:14.511345 10780 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:14.713798 10576 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.649758  4747 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:14.674050  4605 raft_consensus.cc:1217] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 14.7->[15.8-16.9]   Dedup: 15.8->[16.9-16.9]
I20260709 12:27:14.715153 10576 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:14.715587  4605 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: [15.8-16.9]
I20260709 12:27:14.562239 10132 consensus_queue.cc:237] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.717790 10780 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.635370 10749 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.681985  4755 raft_consensus.cc:2393] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 16: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:14.631855 10229 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.684839  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:27:14.687808  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 126437us
I20260709 12:27:14.725028  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:27:14.689015  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.691435  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.694410 10863 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.728366  4678 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:14.730199  4673 raft_consensus.cc:1240] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:14.730367  4672 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:14.731884  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:14.660513 10837 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:14.743227  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 152451us
I20260709 12:27:14.746970  4672 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:14.757397  4605 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 17. Current term is 18. Ops: []
W20260709 12:27:14.749794  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.766142  4674 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:27:14.745241  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.770978  4755 raft_consensus.cc:2410] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
W20260709 12:27:14.508792 10841 consensus_queue.cc:1175] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49277a7653524787a715232ab908bbc6" responder_term: 17 status { last_received { term: 17 index: 9 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:14.679172 10580 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19.
W20260709 12:27:14.745062 10703 consensus_meta.cc:287] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.573s	user 0.005s	sys 0.005s
I20260709 12:27:14.645450 10816 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:14.773389  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 46763.1us
I20260709 12:27:14.791909 10876 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.791730  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.688644 10626 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:14.795315  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.778107  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.709491 10758 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:14.741384 10862 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:14.739517 10553 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.799276  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:14.801257  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:14.836508  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 58477us
I20260709 12:27:14.741590 10704 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:14.692219 10644 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:14.794134 10839 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:27:14.769953 10837 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:14.810115  4596 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:14.769861 10835 consensus_queue.cc:1059] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:27:14.817116 10567 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:14.817430  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:14.821249 10816 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:14.822011 10874 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:14.829943  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:14.859822  4748 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:14.799816  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.844162  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.689270 10857 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.800438 10832 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:14.744297 10229 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.845753 10551 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:14.845710 10704 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:14.868034  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.869633  4599 raft_consensus.cc:2368] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17.
I20260709 12:27:14.864171  4569 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:14.866552  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:14.845548 10749 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:14.803576 10703 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.884492  4569 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:14.886590  4569 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:14.900947  4750 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:14.861884 10528 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:14.915694  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.916285  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.918924  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:14.920411  4748 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:14.920905  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:14.921725  4749 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:14.874341 10785 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:14.930225  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:14.935070  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 110686us
I20260709 12:27:14.872231 10704 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:14.872133 10737 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:14.951310 10850 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:14.926800  4571 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:14.929222  4647 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.939718  4605 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:14.949612 10857 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:14.836190 10676 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.835763 10876 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.955420  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:14.965431  4749 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:14.956352  4747 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:14.895174 10697 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:14.954514  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:14.852578 10553 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:14.960325  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:14.881083 10865 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:14.904282 10814 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:14.966675  4672 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:14.845162 10757 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:14.873775 10875 consensus_queue.cc:1059] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.099s
I20260709 12:27:14.978881 10229 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:14.967845 10785 raft_consensus.cc:697] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:14.962733 10528 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:14.962066 10585 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:14.899151 10729 consensus_queue.cc:1059] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:27:14.894845 10758 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.016916 10862 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:15.015633 10498 raft_consensus.cc:993] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:15.005909 10765 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:15.031966 10571 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:15.030105 10528 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:15.035142 10835 consensus_queue.cc:1243] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:15.034407 10797 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:15.051548 10615 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:15.038554 10703 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:15.055526 10862 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.055678 10785 consensus_queue.cc:237] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.040426 10863 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:15.056396 10837 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.056334 10876 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:15.068614 10851 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:15.058280 10758 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:15.070892 10837 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:14.950742  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 64293.3us
I20260709 12:27:14.954519  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:15.112723  4600 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:15.118371  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:15.120317  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:27:15.120106  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 157536us
I20260709 12:27:15.126231  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:15.128345  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:15.122428  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:15.129465  4597 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:15.129547  4680 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:15.130719  4675 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:27:15.131719  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:15.148444  4675 pending_rounds.cc:85] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 8
I20260709 12:27:15.148891  4675 pending_rounds.cc:107] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:27:15.140246  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:15.140427  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:15.149262  4675 raft_consensus.cc:2889] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:15.140652  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:15.150736  5344 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:15.142079  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:15.143252  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:15.153103  5348 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:15.154800  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:15.155874  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:15.156741  4755 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:15.159000  4645 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:15.165079  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:15.166342  4748 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 16: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:15.166146  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:15.181879 10571 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:15.182981 10528 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.185402 10553 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:15.045047 10644 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:15.188009 10757 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:15.040917 10498 raft_consensus.cc:1081] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:15.191614 10835 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:15.194300 10716 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:15.195178 10716 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:15.198889 10567 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:15.201107  4747 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:42632
I20260709 12:27:15.175736 10703 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:15.202440  4747 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:15.203199  4747 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:15.194324 10851 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:15.210471 10851 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.212496 10757 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.221148 10757 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:15.221833 10576 consensus_meta.cc:287] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.506s	user 0.016s	sys 0.002s
I20260709 12:27:15.183130 10571 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.221679 10763 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.223251 10763 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.224771 10571 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:15.192636 10835 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.221081 10851 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:15.238898 10835 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:15.235394 10763 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:15.244379  5344 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:27:15.251315  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 106151us
I20260709 12:27:15.264554  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:15.268680  4675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 2233) took 1176 ms. Trace:
I20260709 12:27:15.269222  4675 rpcz_store.cc:276] 0709 12:27:14.092381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.093843 (+  1462us) service_pool.cc:224] Handling call
0709 12:27:14.094237 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.150116 (+1055879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.268373 (+118257us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001bbf54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:15.268582 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2381331328}
I20260709 12:27:15.273501 10576 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:15.274890  5348 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:15.257171  4569 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:15.281594  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:15.282490  4753 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
W20260709 12:27:15.285148  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.197460 10716 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.297191  4719 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:15.303409  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:15.308522  4644 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:15.309669  4568 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:15.310211 10580 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:15.310881 10580 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.311383 10580 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:15.311415 10704 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.310218 10610 raft_consensus.cc:3055] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:15.314079 10610 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:27:15.315240  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 151762us
I20260709 12:27:15.317107 10835 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:15.320272 10835 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.298003  5314 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:15.297760  4747 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.320968 10835 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:15.325559 10756 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:15.326263 10756 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:15.326773 10453 consensus_meta.cc:287] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.604s	user 0.006s	sys 0.009s
I20260709 12:27:15.329471 10704 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:15.331785 10723 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:15.334156 10360 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:15.334931 10360 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:15.326795 10756 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:15.346046  4749 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:15.349772  4600 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:27:15.342901  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:15.352698  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:15.349278 10873 log.cc:927] Time spent T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.443s	user 0.002s	sys 0.000s
I20260709 12:27:15.336566 10360 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.358253  4747 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:15.364189 10727 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:15.372789 10840 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 1.038s	user 0.006s	sys 0.000s
I20260709 12:27:15.374581 10857 raft_consensus.cc:3055] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:15.375263 10857 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:15.387660  4605 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:15.391127  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:15.397011 10741 raft_consensus.cc:3055] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:15.397720 10741 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:15.399107 10741 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.408802 10741 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:15.409554 10841 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.410365 10841 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:15.402609  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 233625us
W20260709 12:27:15.418298 10882 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.492s	user 0.002s	sys 0.006s
I20260709 12:27:15.421025  4597 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:15.425078  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3540) took 1270 ms. Trace:
I20260709 12:27:15.425527  4679 rpcz_store.cc:276] 0709 12:27:14.154932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.180106 (+ 25174us) service_pool.cc:224] Handling call
0709 12:27:15.424733 (+1244627us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b540017e154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:15.424899 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2164239232}
I20260709 12:27:15.427321 10757 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.411602 10851 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:15.433938  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 176979us
I20260709 12:27:15.444217  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:15.445007  4754 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:15.435693 10780 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:15.446725 10780 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:15.447480  4599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 2233) took 1413 ms (client timeout 1511 ms). Trace:
W20260709 12:27:15.447980  4599 rpcz_store.cc:269] 0709 12:27:14.033852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.063922 (+ 30070us) service_pool.cc:224] Handling call
0709 12:27:15.447071 (+1383149us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b5400343854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:15.447273 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1685364608}
I20260709 12:27:15.449661 10780 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:15.450356  4751 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:15.435344  4604 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:15.429992  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:15.419998 10716 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:27:15.461066  4679 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 12:27:15.433910 10883 log.cc:927] Time spent T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.514s	user 0.004s	sys 0.000s
I20260709 12:27:15.467437  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:15.468400  4679 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:15.460472  4597 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:27:15.433676 10860 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:27:15.471299 10795 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:27:15.477032 10860 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.477380 10795 raft_consensus.cc:3055] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:15.476969 10756 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.457487  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:15.483736  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:15.485100  5344 raft_consensus.cc:2368] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
I20260709 12:27:15.477849 10795 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:27:15.487402  4571 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.195:41049 timed out after 1.505s (SENT) [suppressed 9 similar messages]
I20260709 12:27:15.488437 10756 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:15.488512 10795 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.477874 10860 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.478667  4677 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:15.479563  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:15.491706  4679 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:15.490535 10795 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:15.491940 10860 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:15.377185 10857 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.493784 10857 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:15.492004  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:15.494922  4754 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:15.498873 10841 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:15.502511  4755 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 10.
I20260709 12:27:15.480737  5312 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:15.439782  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:15.316072 10610 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.504688  4753 raft_consensus.cc:1240] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:15.505888  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:15.466741 10851 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:15.506232  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:15.520346  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:15.521314  4679 raft_consensus.cc:2393] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 17: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:15.522497  5347 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:27:15.525410  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 217298us
I20260709 12:27:15.527357  4604 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:15.429557 10757 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:15.537019 10360 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:15.528728 10756 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:15.544332  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 63833.8us
I20260709 12:27:15.508821 10576 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:27:15.551855 10757 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:15.556181  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:15.563897  4754 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:15.571972  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:15.573763  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:15.579670  4597 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:15.580984  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:15.581701  4751 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:15.583176 10874 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:15.583966 10874 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.588722  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:15.592888  4679 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:15.594017  5312 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:15.598132 10610 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:15.600864  5312 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:15.604367  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:15.603017  5314 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:15.608246 10874 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:15.610193  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 122939us
I20260709 12:27:15.613509 10763 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:15.614076  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:15.614382 10763 raft_consensus.cc:2804] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:15.614880  4672 raft_consensus.cc:2368] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
I20260709 12:27:15.614925 10763 raft_consensus.cc:697] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:15.615991  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:15.616055 10763 consensus_queue.cc:237] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.616879  5346 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:15.620689  4647 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:15.629760 10831 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:15.630267 10831 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.630631 10831 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:15.633769  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:15.634536  4678 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:15.635025  4681 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:15.639441  4569 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:15.648535  4596 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:15.650484  4720 leader_election.cc:400] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17.
I20260709 12:27:15.651118  4719 leader_election.cc:400] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
I20260709 12:27:15.651410  4719 leader_election.cc:403] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:15.656515  4751 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:15.658288  4676 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:15.666324  4719 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:15.609926 10756 consensus_queue.cc:1059] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:27:15.669205 10756 raft_consensus.cc:3055] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:15.659516 10716 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.669592 10781 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:15.652309 10841 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:15.671079 10841 raft_consensus.cc:697] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:15.672222 10841 consensus_queue.cc:237] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:15.673812  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 76330.5us
I20260709 12:27:15.674355  4672 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:15.676332  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:15.669657 10756 raft_consensus.cc:740] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:27:15.679111  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 77276.9us
I20260709 12:27:15.679730 10615 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:15.680135 10615 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.680485 10615 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:15.677451  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 90725.4us
I20260709 12:27:15.678695 10756 consensus_queue.cc:260] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.689076  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:15.690094  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:15.648310 10576 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:15.681008  4647 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:15.691980 10360 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:15.692703  5312 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:15.669186 10610 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
I20260709 12:27:15.693847 10360 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:15.702288 10360 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:15.703749 10757 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:15.670121 10716 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.710726 10756 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:15.715678 10747 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.716286 10747 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.718604 10747 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:15.718995  4751 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:15.719537  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3562) took 1210 ms. Trace:
I20260709 12:27:15.719924  4676 rpcz_store.cc:276] 0709 12:27:14.509088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.524324 (+ 15236us) service_pool.cc:224] Handling call
0709 12:27:15.719317 (+1194993us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400201054. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:15.719459 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2467363584}
I20260709 12:27:15.709827 10758 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:15.720866 10758 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:15.720911  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:15.740649  5344 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:15.745265  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:15.745736 10571 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:15.773691 10251 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:15.774484 10251 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.728135 10757 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:27:15.774992 10251 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.777211 10251 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:15.746515 10870 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:15.778015 10870 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:15.778343 10870 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:15.778916 10716 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:15.779266 10870 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.750130 10837 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:27:15.752894  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 63917us
I20260709 12:27:15.753095  5346 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:15.754099  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:15.784829  4596 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:15.756435  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:15.760298  4678 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:15.761973  4747 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:15.765992  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:15.767748  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 85594.4us
I20260709 12:27:15.788314  4749 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:27:15.788143  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:15.670163 10780 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:15.793545  4679 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:15.780652  5348 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:27:15.791313  4681 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:15.787853 10758 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.789649  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:15.784355 10723 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:27:15.808007 10615 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.808688 10723 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.810258  4601 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:15.751924 10580 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.812850  4597 raft_consensus.cc:1275] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:27:15.813133 10580 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:15.813913  4675 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 22: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:15.826635  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:15.827327  4676 raft_consensus.cc:3060] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:15.827077 10571 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:15.827586  4677 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 12:27:15.814465  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:15.800323  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:15.811652  4603 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:15.823951  4752 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:15.816210  4598 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:27:15.820397  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:15.830941  4598 pending_rounds.cc:85] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 10
I20260709 12:27:15.850276  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:15.850937  4598 pending_rounds.cc:107] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:27:15.851220  4677 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:15.851989  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:15.852156  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:15.852567  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:15.820802 10528 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.855976  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:15.857743 10644 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:15.858712  4601 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:15.789819 10757 raft_consensus.cc:2764] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:27:15.821040 10615 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:15.830765 10780 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.866014 10780 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:15.868834 10747 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:15.849956 10453 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:15.855618 10528 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.857705  4569 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:15.851300  4598 raft_consensus.cc:2889] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:15.780546 10870 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:15.879606  4604 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:15.887557 10703 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:15.887657  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:15.892853 10857 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:15.893662  4602 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:15.809319 10723 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:15.730759 10835 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:15.907971  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.897734 10874 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:15.911237  4720 leader_election.cc:400] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
W20260709 12:27:15.899271  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 50681.7us
I20260709 12:27:15.926726  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:15.881356 10453 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:15.927996  4597 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:27:15.924470  5312 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:15.910745 10837 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:15.942960  4645 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:15.904301 10576 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:27:15.977913  4596 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:15.927439 10610 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:27:15.909220 10857 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.996980  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:15.990888 10874 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:27:16.014482 10831 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:15.988024 10756 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:16.023837 10857 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:15.967298 10781 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
W20260709 12:27:16.048162  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 145135us
W20260709 12:27:16.045066  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48172.1us
I20260709 12:27:16.057873 10875 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:16.058339 10875 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:16.055171  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:16.063374  4748 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 19: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:16.068524  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:16.069243  5346 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:27:16.008122 10851 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:27:16.072101  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 214539us
I20260709 12:27:16.074893  4719 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:16.009052 10860 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:27:16.008864 10757 consensus_queue.cc:1048] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:27:16.055892 10857 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:16.093091  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:16.056269 10360 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:16.141153  4597 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 15: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:27:16.143359  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 50426.4us
I20260709 12:27:16.094182 10781 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:16.139521 10831 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:16.172855  4603 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:16.094658 10857 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:16.085227 10860 consensus_queue.cc:1243] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:27:16.205109 10857 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:16.172855 10723 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:16.092967 10851 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.215727 10851 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:16.082587 10874 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:27:16.170598 10705 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:16.239161 10904 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
W20260709 12:27:16.250185  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 74830.8us
I20260709 12:27:16.252481 10780 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:16.254820 10360 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:16.095297 10453 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.173679 10781 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:16.263214  4676 raft_consensus.cc:2468] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:16.257306 10453 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:16.265147 10781 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:16.287768  4677 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:16.292511  4569 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:16.294973 10780 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:16.298079 10756 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:16.317631 10870 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:16.318859  4569 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:16.323565  4603 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:16.335657 10781 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:16.340427 10905 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.140s	user 0.003s	sys 0.000s
I20260709 12:27:16.348564 10780 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:16.353065 10874 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:16.356108 10610 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:27:16.361120  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 393581us
W20260709 12:27:16.374399  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 299565us
I20260709 12:27:16.359578 10870 raft_consensus.cc:2764] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:27:16.408206  5346 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:27:16.386178  4645 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:16.363647 10723 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.417958  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:16.418610  5347 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:16.429756  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:16.438237  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:16.441800  4750 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:16.422792 10757 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:16.416566 10610 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.446137  5312 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:16.421993 10723 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.427318 10780 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:16.454088  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.475955  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:16.483407  4598 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:16.488580  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:16.464226 10626 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:16.485405  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:16.493026 10874 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:16.495839  5342 raft_consensus.cc:2368] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18.
I20260709 12:27:16.497255 10874 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.512679 10874 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:16.469075 10757 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.491914 10626 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.524683  4568 leader_election.cc:400] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18.
I20260709 12:27:16.525031  4568 leader_election.cc:403] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:27:16.523137  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:16.524056  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:16.524991 10626 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:16.532516 10571 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:16.533061 10571 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.533476 10571 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:16.533878  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:16.536468 10835 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:16.539017 10610 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:16.540035 10911 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:27:16.542115 10626 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:16.544297 10626 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:16.545792  4752 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:27:16.554457 10921 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:27:16.531351  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:16.576047  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 51509.6us
W20260709 12:27:16.577744  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 46512.4us
I20260709 12:27:16.523360 10874 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:27:16.586045 10916 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18.
I20260709 12:27:16.592577  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:16.592942 10132 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:16.593642 10723 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:16.604483  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:16.605283  4677 raft_consensus.cc:3060] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:16.568921 10553 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:16.586738 10780 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:16.628906  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:16.555495 10571 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:16.577826 10781 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:16.581667 10615 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:16.587368 10610 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:16.625699 10741 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:16.625185 10820 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.589767 10576 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.667245  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:16.667518  4603 raft_consensus.cc:2368] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
I20260709 12:27:16.665889  4676 raft_consensus.cc:1275] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:16.659142 10580 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.660620 10727 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:16.682147  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:16.668622 10615 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:16.684628  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:16.656817 10553 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:16.688973  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:16.689164  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:16.689558  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:16.689693  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:16.689968  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:16.690563  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:16.690780  5340 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:16.695693  5346 raft_consensus.cc:2368] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
I20260709 12:27:16.695572  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:16.703851  5342 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 19: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:16.705438  4680 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:16.683353 10917 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:16.704979  4673 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:27:16.683421 10747 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:16.728257  4677 raft_consensus.cc:2468] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 19.
I20260709 12:27:16.691893 10841 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:16.679529 10610 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.680461 10580 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.675915 10576 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.732728  5347 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:16.732971  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:16.733150  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:16.676044 10820 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.731094  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:16.745343  4680 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: [13.7-15.8]
I20260709 12:27:16.692417 10727 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.728688 10780 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:16.753545 10924 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:27:16.765688  4681 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:16.753295 10747 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.755718 10918 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.777361  4679 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:16.779175  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 393065us
I20260709 12:27:16.783298  4678 raft_consensus.cc:2393] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 21: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:16.785353  4674 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:16.747727 10714 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:16.803263  4673 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:16.768468 10757 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:16.806761  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 74653us
I20260709 12:27:16.810082  4719 leader_election.cc:400] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
I20260709 12:27:16.810626  4719 leader_election.cc:403] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:27:16.768186 10780 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.808748 10714 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.819797 10780 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:16.820132 10714 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:16.821287 10927 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:27:16.822407 10553 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:16.823127 10571 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:16.824683  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:16.825454  5348 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 16: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:16.825817 10576 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:16.830506  4673 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:27:16.833649  4598 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
W20260709 12:27:16.836084 10934 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:27:16.837869 10580 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:16.840391  4485 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e8a74137ba104fe0b465cab6399c4940 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica dbe960c35c84451faf4e5b5c9233fa39 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:16.842861 10727 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:16.792049 10615 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:16.757637 10841 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.804024 10851 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:16.807943 10925 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:27:16.847312  5312 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:16.859582  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:16.742722 10917 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:16.862474  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.865506  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:16.866356  4676 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:27:16.808665 10723 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:16.919709  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.857921 10851 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.945644  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:16.947089  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:16.851852 10553 consensus_queue.cc:1048] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 12:27:16.951082  5345 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:16.867697 10756 raft_consensus.cc:2804] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:16.806079 10918 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:16.977154 10910 log.cc:927] Time spent T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:27:16.977929 10820 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:16.977805 10932 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.128s
I20260709 12:27:16.978663 10939 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.979225 10939 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.980677 10763 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:16.981098 10763 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:16.981204 10933 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.981518 10763 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:16.948351 10576 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:16.981824 10933 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.982606 10576 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.984071 10933 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:16.852881 10875 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:16.986205 10875 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:16.950930 10851 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:16.993753 10938 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:16.994828  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:16.994490 10938 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:16.995507  4673 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:16.996521  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:16.997187  4673 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:16.997987  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:16.998680  4672 raft_consensus.cc:2368] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18.
I20260709 12:27:16.998975  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:16.985608 10930 log.cc:927] Time spent T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 12:27:17.002012  4644 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:17.011157  4679 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:16.975528  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:17.020738  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.028640  4677 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:17.030854  4600 raft_consensus.cc:2393] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:27:16.999915 10925 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:27:17.031966  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:17.047781  4752 raft_consensus.cc:3060] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:17.049835  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 241083us
I20260709 12:27:17.050799  4719 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:17.052048  4680 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 20.
I20260709 12:27:16.997391 10938 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:17.054692  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 107706us
I20260709 12:27:16.961612 10723 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.058902  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:17.063714  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:17.065591 10841 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:17.067663  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 296275us
I20260709 12:27:17.049010 10820 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:27:17.075184  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 99730us
I20260709 12:27:17.083735  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:17.104265  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 2257) took 1235 ms. Trace:
I20260709 12:27:17.110881  4675 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 19: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:17.066677 10860 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:27:17.114092  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 63362.1us
I20260709 12:27:17.110944 10747 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.082543 10453 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:17.116653  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 57874us
I20260709 12:27:17.093475 10939 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.113402 10780 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.114429 10932 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:17.113297 10610 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:17.105121  4601 rpcz_store.cc:276] 0709 12:27:15.853031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.859592 (+  6561us) service_pool.cc:224] Handling call
0709 12:27:15.859991 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:15.933527 (+ 73536us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:15.936636 (+  3109us) log.cc:844] Serialized 14 byte log entry
0709 12:27:16.342033 (+405397us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b1800236aa0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3972a6 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1
0709 12:27:16.342731 (+   698us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b1800236aa0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:27:16.463989 (+121258us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:16.480539 (+ 16550us) log.cc:844] Serialized 26 byte log entry
0709 12:27:16.480761 (+   222us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:16.480809 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.088245 (+607436us) spinlock_profiling.cc:243] Waited 64 ms on lock 0x7b5400354dd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:17.088321 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:17.088388 (+    67us) raft_consensus.cc:1685] finished
0709 12:27:17.088408 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:17.088579 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49747,"prepare.queue_time_us":3056,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":1742,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":25865,"spinlock_wait_cycles":221059840,"thread_start_us":426522,"threads_started":1,"wal-append.queue_time_us":604856}
I20260709 12:27:17.125378  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:17.115597 10870 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:17.118278 10860 raft_consensus.cc:697] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 19 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:17.134032 10453 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:17.143682 10950 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.027s
I20260709 12:27:17.145300 10946 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:17.140178 10932 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:27:17.231643  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 110656us
W20260709 12:27:17.236037  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.236253 10132 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:17.236960 10132 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.230662 10950 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.238922 10950 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.240111 10132 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:17.241644 10528 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:17.127604 10780 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.243638  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:17.129827 10870 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:17.254437  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 118235us
I20260709 12:27:17.255237  4569 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:17.256492 10860 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.257463 10946 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:17.258543 10946 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:17.272558  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 332721us
W20260709 12:27:17.274137  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 159136us
I20260709 12:27:17.283903  4569 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:17.241559 10950 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:17.250312 10723 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.250519 10917 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.308480 10560 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:17.309069 10560 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:17.309557 10560 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:17.316088 10961 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:17.316744 10961 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:17.317855 10961 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.320608 10955 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:17.321424 10955 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.321833 10932 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.299108 10918 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.319650 10961 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:17.303460 10528 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:17.339498  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.340394  4604 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
W20260709 12:27:17.349020  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.355952  4752 raft_consensus.cc:2468] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:27:17.357600  5314 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:27:17.368973  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 99701.4us
I20260709 12:27:17.369874  4569 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:17.372131 10831 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.372656 10831 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.374420 10870 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.382620 10851 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.292909 10756 raft_consensus.cc:697] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 19 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:27:17.354244 10781 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.014s	sys 0.014s
I20260709 12:27:17.402923  4569 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:17.400653 10851 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.407531  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:17.408456  5346 raft_consensus.cc:2410] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 12:27:17.408696  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 51221.3us
W20260709 12:27:17.411736  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 144918us
I20260709 12:27:17.414707  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:17.415422  5348 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 12:27:17.415858  5348 pending_rounds.cc:85] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 9
I20260709 12:27:17.416062  5348 pending_rounds.cc:107] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:27:17.416304  5348 raft_consensus.cc:2889] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:17.416486 10923 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:17.417090 10923 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.385753 10942 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
I20260709 12:27:17.420145  4754 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:27:17.401960 10251 raft_consensus.cc:2804] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:17.426044  4754 pending_rounds.cc:85] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 11
I20260709 12:27:17.401495 10528 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:17.425952 10756 consensus_queue.cc:237] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.429071 10961 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:17.426388  4754 pending_rounds.cc:107] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:27:17.429792  4754 raft_consensus.cc:2889] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:17.421749  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.430454  4750 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 20.
W20260709 12:27:17.410014 10781 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.014s	sys 0.014s
I20260709 12:27:17.439154  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:17.439958  4752 raft_consensus.cc:2393] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 16: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:17.439968  4569 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:17.431137 10870 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:17.436940 10251 raft_consensus.cc:697] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:17.446919  4569 leader_election.cc:400] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18.
I20260709 12:27:17.443682 10916 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:27:17.441147 10528 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:27:17.450507 10870 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:17.451957 10870 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.449405  5340 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:17.447690  4569 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:17.435681 10132 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:17.448630 10251 consensus_queue.cc:237] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.456125 10132 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.458395 10132 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:17.459263 10964 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:17.447106 10723 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:17.466931 10723 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:17.467370 10723 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:17.466022  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:17.463835 10964 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.468312 10916 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:17.468712 10916 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.469058 10916 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:17.469362 10955 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.470160 10964 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:17.465427 10780 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:17.465750 10528 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:17.471366 10528 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:17.471791 10528 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:17.429410 10763 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.437810 10961 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.488349 10560 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.493968 10971 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:17.495330 10929 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.813s	user 0.023s	sys 0.039s
I20260709 12:27:17.501617 10831 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.494571 10971 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.506167 10971 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.506651 10923 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:17.511600  4596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 2262) took 1053 ms. Trace:
I20260709 12:27:17.511894  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:17.512344 10747 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.512913 10747 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.514271  4677 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:17.515000 10747 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.513139  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:17.516917  4676 raft_consensus.cc:1240] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:17.512040  4596 rpcz_store.cc:276] 0709 12:27:16.457942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.458045 (+   103us) service_pool.cc:224] Handling call
0709 12:27:16.458347 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:16.466292 (+  7945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:16.466320 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:16.466824 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 12:27:16.672860 (+206036us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:16.672905 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.511171 (+838266us) spinlock_profiling.cc:243] Waited 7.61 ms on lock 0x7b5400302854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:17.511220 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:17.511269 (+    49us) raft_consensus.cc:1685] finished
0709 12:27:17.511283 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:17.511534 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13636,"raft.queue_time_us":7566,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":13738,"spinlock_wait_cycles":17514240,"thread_start_us":157006,"threads_started":1,"wal-append.queue_time_us":265044}
I20260709 12:27:17.520383  4675 raft_consensus.cc:2393] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 19: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:17.491704  4569 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:27:17.534361  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 406051us
I20260709 12:27:17.538221  4754 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:17.538566  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 2264) took 1079 ms. Trace:
I20260709 12:27:17.543845  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.545697  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.545994  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.546994  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:17.551275  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.560763  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:17.562021  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 70445us
I20260709 12:27:17.564409  4747 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:27:17.538915  4599 rpcz_store.cc:276] 0709 12:27:16.459310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.469720 (+ 10410us) service_pool.cc:224] Handling call
0709 12:27:16.487664 (+ 17944us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:16.511411 (+ 23747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:16.511452 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:16.511980 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 12:27:16.579570 (+ 67590us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:16.580928 (+  1358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.581049 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:17.494563 (+913514us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:27:17.518166 (+ 23603us) raft_consensus.cc:1685] finished
0709 12:27:17.518199 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:17.538505 (+ 20306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20257,"raft.queue_time_us":21470,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":5662,"spinlock_wait_cycles":9673344,"thread_start_us":67197,"threads_started":1,"wal-append.queue_time_us":146711}
I20260709 12:27:17.565944  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.568086  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.568982  4751 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:17.545286 10814 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.569988 10814 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.570281  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.570519 10229 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:17.572351 10814 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.558470  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:17.552305  4681 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:27:17.593647  4681 raft_consensus.cc:606] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:17.565791 10765 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.594715  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:17.598691  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.599463  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.551808 10360 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:17.602244  4676 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:17.605223  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.607609  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.608219  4749 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:17.610767  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.611200  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.616242  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.621100  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:17.621450  4604 raft_consensus.cc:2410] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 12:27:17.623247  4675 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:17.623438 10727 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:17.624225  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:17.626374 10697 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:17.626917 10697 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.628101 10916 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.629030 10697 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.630653 10916 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.631808 10763 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:17.633353  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:17.643294  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:17.643987  4680 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:17.634280  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:17.623291  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.617259 10870 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.637931  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.638891  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:17.639317 10961 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:17.653193  4749 raft_consensus.cc:2410] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:17.653399  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:17.655207  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.652334  4569 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:17.649250 10968 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:27:17.614725 10567 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.675549  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:17.676733  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.676894  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.668561 10814 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:17.677932 10814 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:17.678036  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:17.678390 10814 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:17.679237  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.680286 10714 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:17.663089 10780 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:17.682955  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.683571 10567 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.684324  4605 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:17.685185  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:17.685907 10567 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.610787 10765 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.686885 10741 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:17.687280 10741 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.687654 10741 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:17.689853 10727 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:17.690321 10727 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:17.690884 10727 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:17.683611  4596 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:17.682792 10528 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.689249 10765 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:17.716472  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:17.725008  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 282591us
I20260709 12:27:17.725684  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.669685 10860 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:17.726893  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.730341  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.732791  4755 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:17.677225 10723 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.728056  4645 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:17.731714  4756 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:17.736846  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.737895  4596 raft_consensus.cc:2410] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 12:27:17.740563  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:17.741428  4674 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:17.742422  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:17.764663  5340 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:17.663010 10870 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.769929 10951 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 12:27:17.772986  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:17.778447  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.768968  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:17.894038 10567 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.896472  4674 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:17.916988 10870 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.767789 10820 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:17.882135 10922 consensus_queue.cc:1059] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.183s
W20260709 12:27:17.883733 10977 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:27:17.939023 10576 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.939275  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:17.844987  4678 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:17.900534 10528 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:17.905898 10723 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:17.908691 10982 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:27:17.780448 10990 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.161s
I20260709 12:27:17.943089  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:27:17.943271  4603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 2266) took 1422 ms (client timeout 1770 ms). Trace:
W20260709 12:27:17.943754  4603 rpcz_store.cc:269] 0709 12:27:16.521332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.528309 (+  6977us) service_pool.cc:224] Handling call
0709 12:27:17.943068 (+1414759us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54003967d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:17.943209 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":305588480}
I20260709 12:27:17.948029  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:17.948733  4678 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 20.
I20260709 12:27:17.955983  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:17.743721 10560 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:17.927788  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 158891us
I20260709 12:27:17.929428  4569 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:17.932107  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 172534us
I20260709 12:27:17.964823  4720 leader_election.cc:400] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
I20260709 12:27:17.931977 10705 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:17.934392 10567 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:17.940799  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:17.964839  4645 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:17.957954  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.968421  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:17.963352  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:17.943778  4677 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:17.966265  4569 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:17.967549  4674 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:17.965378  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:17.971839  4569 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:17.970515  4645 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:17.977942  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:17.976469 10723 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:17.978713 10723 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.979544  4598 raft_consensus.cc:2393] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:17.980283  4645 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:17.983552  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:17.973480 10990 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:17.983047  5340 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:18.018543  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:17.984398  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:18.020780  4751 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:18.021757  4645 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:17.982406 10705 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:18.022620 10229 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.024616  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:17.990376  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:17.979275 10553 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:17.966753 10814 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:17.979140 10528 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:18.025959  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:18.036329  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:18.024981 10917 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:18.058962  5340 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:17.998271 10567 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:17.998278 10852 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:18.069499  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 79250.3us
I20260709 12:27:18.069967 10991 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:18.067864 10964 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:18.073362 10964 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:18.079613  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:18.080513  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 51944.2us
I20260709 12:27:18.066620  4645 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:18.037788 10553 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:18.026726 10703 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:18.029066 10990 raft_consensus.cc:697] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:17.985077 10560 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:17.973513 10765 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:17.979089 10723 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:18.059541  4677 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 21.
I20260709 12:27:18.081926  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:18.102718  4749 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:18.090181  4674 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:18.058439 10528 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.106516  4672 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:18.107298  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.062049 10820 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:18.093613  4645 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:18.053210 10727 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.091848 10553 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:18.057353 10229 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:18.106057  4569 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:18.134347 10990 consensus_queue.cc:237] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.134523  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:18.102298 10874 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:27:18.123509 10988 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:27:18.083850 10964 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:18.091960 10917 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:18.108322 10528 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:18.142360  4678 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:27:18.091167 10949 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:27:18.146237 10814 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:18.146850  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:18.095537 10360 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:18.147930  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:18.152371 10781 raft_consensus.cc:3055] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:18.154904  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:18.159349  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:18.167150  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 61194.4us
I20260709 12:27:18.206349  4754 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:18.083639 10741 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:18.168597  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 75232.3us
W20260709 12:27:18.171568  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.173864 10852 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:18.173777 10841 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:18.173812 10916 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:18.175956 10727 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:18.176968  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 105793us
I20260709 12:27:18.201917  4674 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:18.214228  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:18.238416  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:18.223536  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:18.206735  4569 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:18.240372  4600 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:18.242614  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:18.336992  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:18.337711  4605 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
W20260709 12:27:18.338873  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 132215us
I20260709 12:27:18.341766 10814 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:18.342307 10814 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:18.343250 10814 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:18.346024  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 104300us
I20260709 12:27:18.347285  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:18.348932  4571 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:18.355038 10644 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:18.355652 10644 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:18.356091 10644 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:18.358496  4597 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:27:18.359920  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:18.364714 10916 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 21 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:18.366647 10916 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.371402  4681 raft_consensus.cc:2368] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 18. Current term is 19.
I20260709 12:27:18.225626 10360 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:18.377902 10229 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:18.379412 10360 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:18.236675 10852 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:18.182757 10874 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:18.384174 10874 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:18.386637  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:18.388157  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:18.389089  4677 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:18.391145  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:18.392297  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:18.393544  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:18.401858  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 163482us
I20260709 12:27:18.405112  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:18.400265  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:18.410337  4674 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 18: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:18.205976 10781 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:18.414880  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:18.422513  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:18.416586  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 67778.9us
I20260709 12:27:18.427809  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:18.439953  4749 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 21.
I20260709 12:27:18.423242  4602 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:18.336540 10964 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:27:18.465286  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 60260.3us
I20260709 12:27:18.476652  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:18.478008  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:18.473229  5314 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:18.484184  4603 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:18.425737 10571 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:18.433022 10360 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:18.432505 10229 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:18.500937  4679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3567) took 1804 ms (client timeout 1686 ms). Trace:
W20260709 12:27:18.501739  4679 rpcz_store.cc:269] 0709 12:27:16.688853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.688962 (+   109us) service_pool.cc:224] Handling call
0709 12:27:18.493006 (+1804044us) spinlock_profiling.cc:243] Waited 84.8 ms on lock 0x7b540014fad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:18.493144 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195046784}
I20260709 12:27:18.475808 10610 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:18.490126  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:18.392328 10553 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:18.511490  5345 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:27:18.512230  5345 raft_consensus.cc:606] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:18.512594  5345 raft_consensus.cc:614] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:18.493528  4673 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:18.420174 10780 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:18.519673 10780 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:18.522119 10780 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:18.531947 10960 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.532109 11008 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:18.532778 11008 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.533366 11008 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:18.532718 10960 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:18.530010  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:18.479836 11001 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:18.563158  4596 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:18.565505  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:18.495121 10360 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.569823  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:18.570775  4679 raft_consensus.cc:2393] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 21: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:18.477121 10964 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:18.509479 10229 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:18.574450  4679 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:18.576004 10229 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:18.582540 10360 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:18.584460  4604 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:18.586935 10960 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:18.588033 10610 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:18.592649  4676 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 12:27:18.594551  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:18.595358  4602 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:18.602134  4677 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:27:18.568286 11001 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:18.491945 10571 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:18.593818 10964 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:18.619903  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 146799us
W20260709 12:27:18.618916  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 141248us
I20260709 12:27:18.622840  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:18.616652 11000 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.623842 11000 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.639318  4677 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:18.618112 11001 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:18.667347 10852 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:18.670054 10723 consensus_meta.cc:287] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.001s
I20260709 12:27:18.670594 10723 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.673058 10723 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:18.633723 10716 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:18.564826 10781 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.684062 10781 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:18.680328 10741 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:18.703274  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:18.704743  4596 raft_consensus.cc:1240] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:18.705623  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:18.678237 10716 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.694916 10874 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:18.712875  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:18.713892  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:18.716825  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:18.720965  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:18.713581  5345 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:18.729260  4604 raft_consensus.cc:1240] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:18.729452  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:18.711459 10874 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.742995 11000 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:18.747462 10571 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:18.748769 10229 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:18.704803 11008 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:18.753669 11008 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:18.743700  4568 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:18.757268  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:18.754817 10874 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:18.769528  4754 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:18.709136 10644 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.766917  5346 raft_consensus.cc:1240] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:27:18.769289  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:18.739861 10919 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:18.761605 10723 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:18.787568  4597 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:18.785772 10723 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.808672  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:18.809504  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:18.820142  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:18.791258 11011 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:18.781924 10814 consensus_queue.cc:1059] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:27:18.783350 10919 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:18.813768 10723 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:18.834770  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:18.796293 10360 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.854260  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:18.823117 10857 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:18.855341  4754 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:18.857044  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:18.864919  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:18.824425 10741 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:27:18.820755 11014 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:27:18.859856 10857 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.884203 10716 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:18.884831 10360 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:18.885793 10964 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.859127 10757 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:18.869086 10875 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:18.904961  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:18.895555  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:18.855388 11010 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:27:18.871306 11018 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:27:18.922276  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:18.902599  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:18.902746 10919 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:18.891233 10528 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.682s	user 0.011s	sys 0.007s
I20260709 12:27:18.882840 10780 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.903532 11001 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.906046  4719 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:18.911028  4600 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:18.911259  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:18.916020 11009 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:18.831066 11011 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.894335 10857 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:18.930194  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 75139.2us
I20260709 12:27:18.927254  4600 raft_consensus.cc:606] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:18.941823  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:18.910111 10875 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:18.946750  4676 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:27:18.936424 11015 log.cc:927] Time spent T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:27:18.959609  4674 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:18.963888  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:18.964742  4679 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:18.957906  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:18.941745 10933 raft_consensus.cc:3055] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:18.969131  4677 raft_consensus.cc:2376] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19. Re-sending same reply.
I20260709 12:27:18.926600 10757 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:27:18.926532 10528 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:18.952944 10875 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:18.981424  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 124452us
I20260709 12:27:18.970409 11010 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:18.962831 10644 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:18.987725  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:18.973955 10933 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:18.999719  4600 raft_consensus.cc:1240] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:18.981773 10964 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:18.998453 11013 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
W20260709 12:27:18.993911 11012 log.cc:927] Time spent T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.172s	user 0.001s	sys 0.001s
I20260709 12:27:19.075832 10528 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:19.080183 10723 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.081919 10933 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.083323 10933 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:19.010748 11001 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:19.012786 11011 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:19.016906  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:19.017290 10747 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:27:19.022470 11021 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:27:19.089135  4752 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42632 (request call id 1191) took 1359 ms (client timeout 1734 ms). Trace:
I20260709 12:27:19.089032 10747 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:19.089583  4752 rpcz_store.cc:269] 0709 12:27:17.729550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.016617 (+1287067us) service_pool.cc:224] Handling call
0709 12:27:19.089069 (+ 72452us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:19.024199  4672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 2262) took 1512 ms. Trace:
I20260709 12:27:19.090328  4672 rpcz_store.cc:276] 0709 12:27:17.511936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.513665 (+  1729us) service_pool.cc:224] Handling call
0709 12:27:18.062295 (+548630us) spinlock_profiling.cc:243] Waited 3.72 ms on lock 0x7b58000d4798. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff348ffc0 00007efff347a929 00007efff347a3d2 00007effec9f1bd4
0709 12:27:18.103554 (+ 41259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:18.107018 (+  3464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:19.023974 (+916956us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b54001bd350. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:19.024132 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102928000}
W20260709 12:27:19.027621 11020 log.cc:927] Time spent T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
W20260709 12:27:19.028255 11016 log.cc:927] Time spent T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:27:19.092809  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 3009) took 1349 ms. Trace:
I20260709 12:27:19.093144  4756 rpcz_store.cc:276] 0709 12:27:17.742950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.743079 (+   129us) service_pool.cc:224] Handling call
0709 12:27:17.743400 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.762401 (+ 19001us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.762449 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:18.590805 (+828356us) spinlock_profiling.cc:243] Waited 5.8 ms on lock 0x7b2c000d0108. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:27:18.591278 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.758018 (+166740us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:18.758058 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.758127 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:19.092487 (+334360us) raft_consensus.cc:1685] finished
0709 12:27:19.092519 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.092729 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15125,"raft.queue_time_us":1808,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":67856,"spinlock_wait_cycles":13329792,"thread_start_us":166349,"threads_started":1,"wal-append.queue_time_us":221825}
I20260709 12:27:19.034833 10780 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:19.037024  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51856 (request call id 3604) took 1382 ms. Trace:
I20260709 12:27:19.095485  4681 rpcz_store.cc:276] 0709 12:27:17.654174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.967893 (+313719us) service_pool.cc:224] Handling call
0709 12:27:19.036799 (+1068906us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b54001bce54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:19.036959 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":924892032}
I20260709 12:27:19.038409 10874 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.039579  5342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50644 (request call id 873) took 1625 ms. Trace:
I20260709 12:27:19.096803  5342 rpcz_store.cc:276] 0709 12:27:17.414220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.414339 (+   119us) service_pool.cc:224] Handling call
0709 12:27:19.039365 (+1625026us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400640354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:19.039494 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321339264}
I20260709 12:27:19.099534 10874 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:19.067523 11025 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:19.100255 11025 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.102622 11025 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:19.068537  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:18.939067 11009 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.103914  5312 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:19.105052  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:27:19.044368  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.106220 11009 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:19.106823  4722 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.108683 10960 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:19.109171 10948 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:19.112596 10955 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:19.113210 10955 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.116089 10955 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:19.121910 10560 raft_consensus.cc:3055] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:19.122704 10560 raft_consensus.cc:740] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:19.123817 10560 consensus_queue.cc:260] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.128854 11018 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:19.129433 11018 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:19.130492 11018 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.132268 11018 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:19.135097  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:19.136256  4755 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 3620) took 1401 ms (client timeout 1709 ms). Trace:
W20260709 12:27:19.136665  4755 rpcz_store.cc:269] 0709 12:27:17.734970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.134748 (+1399778us) service_pool.cc:224] Handling call
0709 12:27:19.136185 (+  1437us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:19.138249 10560 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:19.146915 10831 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:19.150363  5348 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 883) took 1711 ms. Trace:
I20260709 12:27:19.150827  5348 rpcz_store.cc:276] 0709 12:27:17.439212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.443911 (+  4699us) service_pool.cc:224] Handling call
0709 12:27:17.444163 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:17.459288 (+ 15125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.459317 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:17.459357 (+    40us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:17.459400 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:19.150103 (+1690703us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b54005e33d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:19.150150 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.150275 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34303872}
W20260709 12:27:19.158023  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.210680 11007 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.520s	user 0.004s	sys 0.000s
I20260709 12:27:19.168185 10580 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:19.172333  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.173739  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.174655  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3588) took 2178 ms. Trace:
I20260709 12:27:19.215838  4673 rpcz_store.cc:276] 0709 12:27:16.996446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.055024 (+ 58578us) service_pool.cc:224] Handling call
0709 12:27:18.493152 (+1438128us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:18.513322 (+ 20170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:19.174333 (+661011us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b540014fad0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:19.174543 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3306533248}
I20260709 12:27:19.176347  4679 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:27:19.219652 10747 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:19.201380  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.221920  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.225976 11028 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:19.229039 10874 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:19.229882 10874 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.178799 10723 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:19.184348 10851 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:19.232447 10851 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.185169 11025 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.193775  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53448 (request call id 1046) took 1238 ms. Trace:
I20260709 12:27:19.233795 11025 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.233923  4598 rpcz_store.cc:276] 0709 12:27:17.955446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.955584 (+   138us) service_pool.cc:224] Handling call
0709 12:27:19.193487 (+1237903us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b5400218c54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:19.193685 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51699072}
I20260709 12:27:19.235770  4681 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 22.
I20260709 12:27:19.239768  4749 raft_consensus.cc:1275] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 12:27:19.240561  4749 pending_rounds.cc:85] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 14
I20260709 12:27:19.240880  4749 pending_rounds.cc:107] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 12:27:19.241165  4749 raft_consensus.cc:2889] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:19.243498 10956 raft_consensus.cc:993] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:19.244241 10956 raft_consensus.cc:1081] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:19.194767 10949 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:19.247371 10991 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:19.248366  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:19.248999  4755 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 22.
I20260709 12:27:19.252719 10870 consensus_queue.cc:1059] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:19.231716  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:19.254985 10814 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:19.255549 10814 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:19.256842 10814 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.196720  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:19.259434 10949 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:19.260102 10949 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.260658 10949 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:19.266558  4569 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:19.287675 10795 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:19.296478 10723 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:19.296684  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.297271 10723 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.297951 10723 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:19.288672  5312 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:19.300151 10795 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:19.300760 10795 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.301306 10795 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:19.334116 10553 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:19.335111 10874 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:19.335767  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 348119us
I20260709 12:27:19.347605  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:19.336994  4749 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:27:19.348342  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 187316us
W20260709 12:27:19.336210  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 139582us
I20260709 12:27:19.352967  4674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:27:19.353754  4674 raft_consensus.cc:606] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:19.354208  4674 raft_consensus.cc:614] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:19.354267  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:19.147679 10831 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.359131  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:19.360231  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:19.360971  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 44480.9us
I20260709 12:27:19.361238 10831 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:19.364497  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:19.444355  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.364631 10857 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.365417  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:19.365890  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:19.367355 10851 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:19.368523 11026 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:19.368999 10874 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:19.368841 10870 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:19.456878  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.214820 10580 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:19.374150  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 468136us
I20260709 12:27:19.460440  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:19.460788  4719 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:19.461215  4676 raft_consensus.cc:2393] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 19: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:19.376616  4645 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:19.378880 11030 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:27:19.394493 10951 consensus_queue.cc:1059] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:27:19.469125 10951 raft_consensus.cc:3055] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:19.469496 10951 raft_consensus.cc:740] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:19.469743 10580 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:19.470503 10951 consensus_queue.cc:260] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.471895 10951 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:19.473762  4672 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
W20260709 12:27:19.475121 10922 consensus_queue.cc:1175] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5c3201048f8346009aefca07428828f3" responder_term: 14 status { last_received { term: 14 index: 10 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:19.462314 10870 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.404760 10961 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
W20260709 12:27:19.417822  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 55199.6us
I20260709 12:27:19.418133 11028 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:19.479111 11028 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.479576  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:19.480145  4597 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:19.418756  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.481063  4598 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:27:19.482120  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.482910  4598 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 15: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:19.483986  4598 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:19.418677 11029 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:19.419507 11014 consensus_queue.cc:1059] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:19.486024 11014 raft_consensus.cc:3055] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:19.486395 11014 raft_consensus.cc:740] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:19.487280 11014 consensus_queue.cc:260] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.488550 11014 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:19.419757  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.420768  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.473821 10932 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:27:19.425520  4602 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:19.426774  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:19.427834  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:19.492996 10610 consensus_queue.cc:1175] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5c3201048f8346009aefca07428828f3" responder_term: 19 status { last_received { term: 19 index: 12 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:27:19.485240 11029 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.495983 10610 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:19.497030 10970 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:19.497486 10970 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:19.498458 10970 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.499795 10970 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:19.429031  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:27:19.431103  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 60498.6us
I20260709 12:27:19.456485 10831 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:19.502441  4569 leader_election.cc:400] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 18. Current term is 19.
I20260709 12:27:19.432354  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.408229  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:19.505625  4600 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:19.433823  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:19.436764  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.442721  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:19.513790  4675 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:19.516358  4605 raft_consensus.cc:1275] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:19.444154  4756 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:19.513978  4602 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 12:27:19.453102  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 87768.7us
I20260709 12:27:19.370088 10875 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.457628  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:27:19.458866  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 59371.2us
I20260709 12:27:19.465217  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:27:19.467103  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 90662.8us
I20260709 12:27:19.467587 10841 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:19.467836  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:19.468382 10950 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:19.475478 10942 consensus_queue.cc:1059] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:19.496850 10610 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:27:19.539539  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 74423.1us
I20260709 12:27:19.384152 11005 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:19.383613 11025 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:19.395411 10814 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:19.525645  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:19.531783  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.533707  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:19.534400  4674 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:19.523976 10991 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:19.451905 10723 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.557600  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:19.558276  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:19.538102  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.558542  4679 raft_consensus.cc:3060] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:19.558631  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:19.568936  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:19.569669  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:19.570436  4678 raft_consensus.cc:2368] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 18. Current term is 19.
I20260709 12:27:19.572181  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:19.573526  4680 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:27:19.570003 10960 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:19.579779 10960 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.583989  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:19.585180  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:19.532227 10955 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:19.573249 10942 consensus_queue.cc:1243] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:19.590888 10132 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:19.592936  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:19.593776  4755 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 19: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:19.595280  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:19.596030  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:19.596657  4752 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:19.597455  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:19.595455  4596 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:27:19.604748  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:19.569101 10922 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.605163  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:19.568682 10956 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:19.617508  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:19.618360  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:19.619074  4674 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:19.557026 10916 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:19.609722  4755 raft_consensus.cc:2410] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 12:27:19.602982  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.621006  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:19.615486  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:19.626816  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:19.628506  4747 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:19.630549  4756 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43876
I20260709 12:27:19.589648 10950 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:19.595948 10955 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.635232  5340 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:27:19.635958 10955 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:19.635975  5340 raft_consensus.cc:606] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:19.637244  5340 raft_consensus.cc:614] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:19.636777  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 55208.3us
I20260709 12:27:19.640084 10870 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:19.641070  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:19.642652  4597 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:19.582693 10567 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:19.646219 10567 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.650789 10916 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.653669 10916 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:19.655381  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:19.657720 10922 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.658780  4677 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 18: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:19.609200 11026 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:19.663316 10723 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:19.664191 10723 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:19.666401 10999 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:19.667562  4747 raft_consensus.cc:1240] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:19.667040 10999 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.582796 10874 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:19.611591 11030 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:19.587309 10919 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:19.671305 11030 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.568954 11009 consensus_queue.cc:1048] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 12:27:19.674198  4756 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:19.674594  4756 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:19.671703 10961 raft_consensus.cc:3055] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:19.676263  4604 raft_consensus.cc:2410] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
W20260709 12:27:19.645493 11018 consensus_meta.cc:287] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.513s	user 0.015s	sys 0.001s
I20260709 12:27:19.681366  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.681985  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:19.683138  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.668784 10874 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.685444  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:19.685866  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:19.687732  4752 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:19.690748  4748 raft_consensus.cc:2368] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 19. Current term is 20.
I20260709 12:27:19.679596  4720 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:19.694294  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:19.695932  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:19.696906  4749 raft_consensus.cc:2368] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 18. Current term is 19.
I20260709 12:27:19.703323 10874 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:19.688947 10961 raft_consensus.cc:740] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:19.704947 10961 consensus_queue.cc:260] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.706246 10961 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:19.709904 11028 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:19.713747 10942 raft_consensus.cc:2804] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:19.714924 10942 raft_consensus.cc:697] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:19.715981 10942 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.689270 10951 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:19.718700 10951 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.722934 10951 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:19.722357  4501 catalog_manager.cc:5697] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 6 to 12, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 12 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:19.724414  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:19.725934  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:19.727640 10875 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:19.664961 10723 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.731010  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.740384 10956 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.732054  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.741000  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.731781  4501 catalog_manager.cc:5697] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:19.744153  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:19.734750  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.735440  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.745179  4501 catalog_manager.cc:5697] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 12 to 14, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 14 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:19.746429  4601 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:27:19.661307 11029 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:19.747486  4501 catalog_manager.cc:5697] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 6 to 9, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 9 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:19.740962 10956 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.748970 10857 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:19.688911 10795 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:19.750286  4756 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:19.751299 10933 consensus_meta.cc:287] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.668s	user 0.008s	sys 0.006s
I20260709 12:27:19.753698  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.738430  4645 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:19.738642  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.754329 10956 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:19.711861 10949 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:19.756934  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.761610  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:19.770215  4679 raft_consensus.cc:2468] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:19.735569  4600 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:27:19.774483  4675 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:19.786309  4672 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: []
W20260709 12:27:19.789636  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 165351us
W20260709 12:27:19.780771  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 195318us
I20260709 12:27:19.756186  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:19.772866  4750 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:19.799016  5345 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 23: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:19.800038  4674 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:19.790475  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:19.804857  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:19.808161  4756 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:19.763634 11030 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:19.815819  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:19.826478  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.827791  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.828984  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:19.830992  4676 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:27:19.782073 10580 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:19.836452  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:19.837309  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:19.846562  4499 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52808 (request call id 277) took 50283 ms (client timeout 15000 ms). Trace:
I20260709 12:27:19.816102 10922 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:19.835769 10580 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:19.838622 10560 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:19.815115 10841 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:27:19.847376 10576 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:19.891528  4719 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:19.854585 10571 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:19.845676 10919 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:27:19.893889 10580 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:19.899904  4719 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:19.891038 10918 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:19.895609 10990 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:19.945832  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:19.946859  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 279880us
I20260709 12:27:19.902853 11030 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:19.960705  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 123461us
I20260709 12:27:19.962419  4569 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:19.967734  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 150376us
I20260709 12:27:19.969980  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:19.970708  4751 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:19.969755  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.972438  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.973349  4601 raft_consensus.cc:2393] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 19: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:19.973256  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:19.974407  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:19.974992  4600 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:19.946019 10857 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:19.957315 10841 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:19.978453  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 127977us
I20260709 12:27:19.989924  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:19.983848  4569 leader_election.cc:400] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 18. Current term is 19.
W20260709 12:27:19.983460  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 213452us
I20260709 12:27:19.990775  4679 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:19.991195  4569 leader_election.cc:403] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:27:19.970582 10795 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:19.990147  4571 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:19.995326  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.995640  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:19.996973  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:19.927636 10918 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.001868  4645 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:20.015703  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:20.019023  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:20.021735  4600 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:20.025846  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:20.026535  4673 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:20.026827  4673 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:20.027598  4673 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:19.994761 10990 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:20.028769  4673 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:20.031279  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.035959  4751 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 23.
W20260709 12:27:20.038197  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 59779.5us
W20260709 12:27:20.038758  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.040412  4679 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 23.
I20260709 12:27:20.004451 10723 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:20.072146  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:27:20.074235  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 84193.5us
W20260709 12:27:19.847075  4499 rpcz_store.cc:269] 0709 12:26:29.447992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.448283 (+   291us) service_pool.cc:224] Handling call
0709 12:27:19.731426 (+50283143us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:14.624931 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:14.640992 (+ 16061us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:14.641046 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:27:14.641068 (+    22us) tablet.cc:662] Decoding operations
0709 12:27:14.641998 (+   930us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:14.642065 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:14.642096 (+    31us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:27:14.642808 (+   712us) tablet.cc:701] Row locks acquired
0709 12:27:14.642828 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:27:14.642965 (+   137us) write_op.cc:270] Start()
0709 12:27:14.643096 (+   131us) write_op.cc:276] Timestamp: P: 1783600034642921 usec, L: 0
0709 12:27:14.643118 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:27:14.643843 (+   725us) log.cc:844] Serialized 3619 byte log entry
0709 12:27:15.231884 (+588041us) op_driver.cc:464] REPLICATION: finished
0709 12:27:15.369969 (+138085us) write_op.cc:301] APPLY: starting
0709 12:27:15.370134 (+   165us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:15.425720 (+ 55586us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:15.425859 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:15.672681 (+246822us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:15.673038 (+   357us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 12:27:15.673097 (+    59us) write_op.cc:312] APPLY: finished
0709 12:27:15.673620 (+   523us) log.cc:844] Serialized 89 byte log entry
0709 12:27:15.673870 (+   250us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:15.674072 (+   202us) write_op.cc:454] Released schema lock
0709 12:27:15.674136 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:15.674716 (+   580us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:15.674746 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":310646,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":129707,"apply.run_cpu_time_us":18738,"apply.run_wall_time_us":304763,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41693,"num_ops":9,"prepare.queue_time_us":317270,"prepare.run_cpu_time_us":68514,"prepare.run_wall_time_us":198190,"raft.queue_time_us":378906,"raft.run_cpu_time_us":38794,"raft.run_wall_time_us":70544,"replication_time_us":588438,"thread_start_us":617721,"threads_started":4,"wal-append.queue_time_us":146119,"wal-append.run_cpu_time_us":33694,"wal-append.run_wall_time_us":1888501}]]}
I20260709 12:27:20.011775 10919 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 18. Current term is 19.
I20260709 12:27:20.051017  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:20.159911  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 91709.5us
I20260709 12:27:20.319293  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:20.166535  4677 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 19: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:20.000662 10870 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 12:27:20.167223 11005 consensus_queue.cc:1048] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:27:20.149626 10933 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.174793 10918 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:20.183413 10949 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:20.181170 11051 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:20.186504 10999 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:20.195051 10851 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.197634 10560 raft_consensus.cc:993] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:20.000310 10841 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.044608 10916 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.166s
I20260709 12:27:20.222851 10837 consensus_queue.cc:1059] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.133s
I20260709 12:27:20.215451 10857 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:27:20.222683  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 51954.5us
I20260709 12:27:20.234863 10970 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:20.223444 11030 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:20.224303 10922 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:27:20.206185 11014 consensus_meta.cc:287] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.716s	user 0.009s	sys 0.007s
I20260709 12:27:20.244977 10875 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:27:20.237200 10961 consensus_meta.cc:287] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.531s	user 0.007s	sys 0.004s
I20260709 12:27:20.241345  4599 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:20.315419  4673 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:20.321480  5312 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:20.323462 10132 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:27:20.323853  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 272916us
W20260709 12:27:20.387949  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 386088us
I20260709 12:27:20.349961 10933 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.169027 10991 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:20.325138 11008 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:20.387130  4571 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:20.527806  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:20.436129  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:20.356010 11052 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:20.381863 10999 raft_consensus.cc:1081] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:20.569669 10970 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.376125 10851 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.583845 10970 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:20.584549 10851 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:20.513111  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 87974.7us
I20260709 12:27:20.585439  4647 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:20.585687 11052 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:20.586621 11052 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:20.419294  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 387766us
I20260709 12:27:20.537597  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:20.349692  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 323408us
W20260709 12:27:20.552951  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 165935us
I20260709 12:27:20.530551  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.562368  4600 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:27:20.568948  4596 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 17: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:20.552336 11030 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:20.552052  4748 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:20.526933 10567 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:20.599359  4676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 2294) took 1060 ms. Trace:
I20260709 12:27:20.606710  4676 rpcz_store.cc:276] 0709 12:27:19.539181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.540127 (+   946us) service_pool.cc:224] Handling call
0709 12:27:19.540589 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:19.834565 (+293976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.591354 (+756789us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b54000b0254. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:20.599301 (+  7947us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":653188992}
I20260709 12:27:20.607434  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:20.599820  4571 leader_election.cc:400] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 18. Current term is 19.
I20260709 12:27:20.608827  4571 leader_election.cc:400] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 19. Current term is 20.
I20260709 12:27:20.609184  4571 leader_election.cc:403] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:27:20.365216 11051 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.613821  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:20.542461 10875 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:20.562796 10837 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:20.406443 10857 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.622082  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:20.623737  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3642) took 1004 ms. Trace:
I20260709 12:27:20.542677 10922 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.588670 10961 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:20.598999 10970 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:20.651326  4680 rpcz_store.cc:276] 0709 12:27:19.619650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.619921 (+   271us) service_pool.cc:224] Handling call
0709 12:27:19.620362 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:19.672693 (+ 52331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:19.672744 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:19.673352 (+   608us) log.cc:844] Serialized 26 byte log entry
0709 12:27:19.936093 (+262741us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:19.936149 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:19.936219 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:20.623389 (+687170us) raft_consensus.cc:1685] finished
0709 12:27:20.623417 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:20.623659 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51772,"raft.queue_time_us":59285,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":9671,"thread_start_us":251851,"threads_started":1,"wal-append.queue_time_us":859171}
W20260709 12:27:20.625679  4681 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.009s	sys 0.026s
I20260709 12:27:20.619414 10795 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:20.511003 11050 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:27:20.621397 10580 consensus_meta.cc:287] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.664s	user 0.011s	sys 0.005s
I20260709 12:27:20.675357 10852 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:20.668249 11064 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:20.673015 10961 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.700392 10857 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:20.703171 10841 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:20.700846 11064 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:20.702662 11058 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:27:20.704403 10961 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:20.647380 11053 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 19. Current term is 20.
I20260709 12:27:20.701879 10580 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:20.706172 10922 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:20.707911 10875 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:20.708168  4672 raft_consensus.cc:1240] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:20.707968 10580 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:20.709721 11064 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:20.681514 10970 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.712745 10970 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:20.723194 11050 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.727160 11051 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:20.693516 10852 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:20.733765 10852 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:20.769861  4676 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:27:20.776206  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.777518  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.778760  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.779912  4598 raft_consensus.cc:2410] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:20.780586  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.781555  4604 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 18: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:20.782325  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.798789  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:20.809826 10933 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:20.798851 11050 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:20.834774 11030 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:20.824682 10795 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:20.840318 10132 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 23 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:27:20.845769  4721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.005s	sys 0.016s
W20260709 12:27:20.849010  4721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.005s	sys 0.016s
I20260709 12:27:20.846589 10132 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:20.870935 10933 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:20.871693 10933 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.844638 10795 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:20.873723 11030 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:20.874315 11030 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:20.875346 10795 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:20.882027  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3648) took 1128 ms. Trace:
I20260709 12:27:20.882236 10795 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:20.882786 10795 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:20.882426  4681 rpcz_store.cc:276] 0709 12:27:19.753637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.799121 (+ 45484us) service_pool.cc:224] Handling call
0709 12:27:19.799483 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:19.825172 (+ 25689us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:19.841123 (+ 15951us) log.cc:844] Serialized 14 byte log entry
0709 12:27:20.071143 (+230020us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b1800401ec0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:27:20.752131 (+680988us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:20.752756 (+   625us) log.cc:844] Serialized 26 byte log entry
0709 12:27:20.752921 (+   165us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:20.753197 (+   276us) log.cc:844] Serialized 14 byte log entry
0709 12:27:20.753458 (+   261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.753586 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:20.881717 (+128131us) raft_consensus.cc:1685] finished
0709 12:27:20.881748 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:20.881949 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99932,"prepare.queue_time_us":24470,"prepare.run_cpu_time_us":273,"prepare.run_wall_time_us":2663,"raft.queue_time_us":124152,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":43427,"spinlock_wait_cycles":19668992,"thread_start_us":605814,"threads_started":1,"wal-append.queue_time_us":1008352}
I20260709 12:27:20.885037 10795 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:20.885037 11030 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:20.881182 10852 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.891604 11030 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:27:20.881811 11057 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.263s	user 0.001s	sys 0.004s
I20260709 12:27:20.848091 10918 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.896238 10852 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:20.898972  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3649) took 1137 ms. Trace:
I20260709 12:27:20.901737  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:20.900485  4748 rpcz_store.cc:276] 0709 12:27:19.761433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.761674 (+   241us) service_pool.cc:224] Handling call
0709 12:27:19.772203 (+ 10529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.604497 (+832294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.898667 (+294170us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b54001b0b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:20.898889 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1771585792}
W20260709 12:27:20.926354  4720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.006s	sys 0.023s
I20260709 12:27:20.900808 11030 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:20.930099 10560 raft_consensus.cc:3055] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:20.929986 10852 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:20.930590 10560 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:27:20.931514  4720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.006s	sys 0.023s
W20260709 12:27:20.928828  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 607423us
I20260709 12:27:20.933956  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:20.936285  4672 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:20.903134 10918 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.934823  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.935456  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1212) took 1131 ms. Trace:
W20260709 12:27:20.934978  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 398255us
I20260709 12:27:20.940930  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:20.941303  4599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 2290) took 1515 ms (client timeout 1960 ms). Trace:
W20260709 12:27:20.941787  4599 rpcz_store.cc:269] 0709 12:27:19.426346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.426501 (+   155us) service_pool.cc:224] Handling call
0709 12:27:20.941112 (+1514611us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b5400301954. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:20.941238 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":932558592}
I20260709 12:27:20.942278  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.943226  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:20.946957  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.947554  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:20.950508  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.951476  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:20.940017  4719 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:20.952831  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.953536  4604 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:20.954000  4719 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:20.954780  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:20.955546  4680 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:20.938050  4753 rpcz_store.cc:276] 0709 12:27:19.803964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.804191 (+   227us) service_pool.cc:224] Handling call
0709 12:27:19.804479 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:19.811078 (+  6599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:19.811123 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:19.817670 (+  6547us) log.cc:844] Serialized 26 byte log entry
0709 12:27:20.014299 (+196629us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:20.014335 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.014400 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:20.833355 (+818955us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:27:20.934345 (+100990us) raft_consensus.cc:1685] finished
0709 12:27:20.934366 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:20.935389 (+  1023us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":15486848,"thread_start_us":123799,"threads_started":1,"wal-append.queue_time_us":749625}
I20260709 12:27:20.937764 10956 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:20.973017  4605 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:27:20.960254  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.960387  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.959834  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.975565  4604 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:20.972307  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:20.966497  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.976601  4676 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
W20260709 12:27:20.977994  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 390531us
W20260709 12:27:20.978176  4756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.016s	sys 0.021s
W20260709 12:27:20.978509  4756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.017s	sys 0.022s
I20260709 12:27:20.978732  4647 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:20.979674  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:20.979904  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:20.980496  4647 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:27:20.979476  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 391169us
I20260709 12:27:20.959848  5312 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:20.957000 10999 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:20.958791 10970 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:20.996438  4647 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:20.957760 10933 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:20.998657  4647 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:20.980836 10956 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:20.958818 10875 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:21.006157  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:21.007097  5345 raft_consensus.cc:2393] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 18: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:21.007808  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:21.000111  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 3034) took 1056 ms. Trace:
I20260709 12:27:21.001812  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:20.957173 10990 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.998607 11071 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:21.060297  4756 rpcz_store.cc:276] 0709 12:27:19.943766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.943948 (+   182us) service_pool.cc:224] Handling call
0709 12:27:19.950444 (+  6496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:19.950692 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:20.351802 (+401110us) log.cc:844] Serialized 14 byte log entry
0709 12:27:20.999678 (+647876us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:20.999742 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:20.999802 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.999894 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:21.000034 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4704,"prepare.queue_time_us":756074,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":86,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":2411,"thread_start_us":1009362,"threads_started":2}
I20260709 12:27:21.063691  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:21.003046 11042 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:20.994136 10999 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:20.996388 11072 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:21.004848  4647 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:20.982404 10560 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.108409  4755 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:27:21.007556 11078 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:21.069156  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.004081 11070 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.009490 10949 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:21.076761 11073 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:21.002136 10852 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:27:21.082686 11079 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:21.071012 10990 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.079141  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.080628 10933 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:21.081100 10922 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.081761  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.082659  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:21.083496  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.084342  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.063755 11054 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:27:21.094347 10999 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.098677 11083 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:21.104501  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.132503  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.109462  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.002360 10714 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:21.113629  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 44566.6us
W20260709 12:27:21.118808  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 117659us
I20260709 12:27:21.119752  4749 raft_consensus.cc:2410] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:27:21.134006  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:21.133962  4756 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 20: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:21.134440  4752 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
W20260709 12:27:21.143334  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 129798us
I20260709 12:27:21.108640 10956 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.146334 11005 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:21.114531 11078 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.117206 11079 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.147575 10814 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:21.148792  4755 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:21.149211  4755 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:21.144591  4571 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:21.149693  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:21.144932  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.116074 10875 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:21.115411 10852 raft_consensus.cc:3055] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:21.123979 10922 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.150525 10956 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:21.118743 11070 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.157857 11075 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:21.172049 11075 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.151866  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.175019 11075 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.175289 10875 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:21.114771 11073 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.176630  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:21.154186 11077 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:21.180619  4571 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:21.181308 11077 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:21.123674 10933 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.194789 10933 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:21.182417  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:21.189505 10875 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:21.190120 11073 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:21.164314 11070 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:21.157648 10870 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:21.154577  4569 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:27:21.202615  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 48151.1us
I20260709 12:27:21.202705 10837 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:21.203558 10837 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:21.204102  4569 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:21.159435 10852 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:21.160619 10916 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:21.162391  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:21.207052 10814 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:21.207598 10814 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:21.207929  4755 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.208642 10814 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.166304 11078 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:21.168601 11079 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:21.168957  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:21.212610  5347 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:21.213266  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.169433  4751 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:27:21.157142 10922 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:21.171344 10990 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:21.223290 10990 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:27:21.224057  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 43534.8us
I20260709 12:27:21.225041  4571 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:21.225929 10990 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:21.232350 11077 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:21.233553 11077 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:21.237951  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 45345us
I20260709 12:27:21.247579  5347 raft_consensus.cc:2410] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:21.243479  4571 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:21.252388 10875 raft_consensus.cc:3055] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:27:21.310638  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 306048us
I20260709 12:27:21.309893  4644 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:21.254081 10918 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.204777 10837 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.254884 10852 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:21.255837  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.259763 10560 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:21.260854 10841 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:21.261447 11079 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:21.261586 11065 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:21.262938  4755 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:21.263357 10714 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:21.266331 11025 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:21.266644 11083 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:27:21.274297  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 63838.9us
W20260709 12:27:21.284574  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 45531.3us
I20260709 12:27:21.365908  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:21.367179  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:21.289144  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:21.368444  4675 raft_consensus.cc:2393] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 17: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:27:21.142910  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 141260us
I20260709 12:27:21.361776 11079 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.291110  5348 raft_consensus.cc:1240] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:21.287989  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:21.317421  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.378439  4598 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:21.373119  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:27:21.324761  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 81386.4us
I20260709 12:27:21.379804  4571 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:21.343706 11030 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:21.363303  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 51800.4us
I20260709 12:27:21.361636  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:21.370314  5312 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:27:21.371323  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 61557.2us
I20260709 12:27:21.290524  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.322794 10990 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:21.391322 10132 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:21.361613 10714 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.310392 10875 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:21.352195 10916 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:21.319255 10970 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:27:21.386668  4647 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.383953  4644 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:21.387719  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:21.390129  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:21.385197 11052 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:21.377949 11078 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:21.378643 10922 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:21.409802 10132 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:21.429306 10132 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:21.429796 10132 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:21.414397  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.416468  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:21.364436 11065 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.381409 11070 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.363406 10841 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.389319 11073 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:21.422430  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.402467 11081 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:21.442987  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:21.405474 10970 consensus_queue.cc:1243] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:21.403714 10916 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:21.375388 11025 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.451723  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:21.452662  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:21.375540 11083 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.456487 10714 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.427402 11030 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:21.456825 10956 consensus_queue.cc:1059] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:21.484357 10956 raft_consensus.cc:3055] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:21.459260 10841 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.484755 10956 raft_consensus.cc:740] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:21.397775 10990 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.423599 10933 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.492404 10956 consensus_queue.cc:260] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.488461  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.457861  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:21.458278 11052 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:21.459056 10942 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:21.508983 10942 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:21.459249 10851 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:21.459087  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 72527.1us
I20260709 12:27:21.510171  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.434031 10580 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:21.434355 11078 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:21.444015 11073 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.476281  4605 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:21.478551  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.482328 11030 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:21.482563 11067 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:21.469254 10916 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:21.514530 10851 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:21.517923  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.527570  4678 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 17: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:21.471735 11087 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:21.442430 11070 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.471235 10795 consensus_queue.cc:1059] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:27:21.555917  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.520174 11078 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:21.560041  4597 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 17: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
W20260709 12:27:21.490108 11086 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:21.527375 11030 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:21.519538 11073 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:21.580653  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 70586.1us
I20260709 12:27:21.532516 11067 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:21.613710  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.613885  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:21.580111  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 94801.9us
I20260709 12:27:21.612046  4599 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 16. Current term is 17. Ops: []
W20260709 12:27:21.615072  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.612852  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.613049  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.613608  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.615935  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:21.616246  4755 raft_consensus.cc:2410] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:27:21.614791  4751 raft_consensus.cc:2393] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 17: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:21.623559  4751 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:21.626096  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.512456 10933 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.538980 10851 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:21.615895  4749 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:21.635830  4751 raft_consensus.cc:2410] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:27:21.635833  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.547350 11063 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:21.467934 10875 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.640026  4749 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:21.641177  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.646908  4753 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:27:21.513722 10942 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.639214  4755 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 11.
I20260709 12:27:21.647831  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:21.638633  5312 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:21.648505  4679 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:27:21.644655  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.645454  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.651466  4647 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.553220 10580 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 12:27:21.634760 11067 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:21.656327  4672 raft_consensus.cc:1240] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:21.638417 10132 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.641209 10851 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:21.518404 11052 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:21.638088 11057 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:27:21.656813  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 271046us
I20260709 12:27:21.559139 11087 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:21.664618  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 242248us
I20260709 12:27:21.674499  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.655629 10956 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:21.644132 11054 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:27:21.668293  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 273463us
I20260709 12:27:21.656333 11025 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.657794 11051 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:21.684772  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.671128 11090 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:21.647589  4485 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e8a74137ba104fe0b465cab6399c4940 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica dbe960c35c84451faf4e5b5c9233fa39 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:21.695825 11065 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:21.695583  4485 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 51851.3us
I20260709 12:27:21.673142 11079 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.706169 11070 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:21.706569 10942 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:21.711647  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 60254.3us
I20260709 12:27:21.648696 11063 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.718497  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.723093  4647 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.724058 10714 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:21.724390 11066 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:21.724469  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.724967 11066 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:21.725888 11079 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:21.726133 11066 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:21.726379 11079 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:21.727084 11079 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:21.691886 10970 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:27:21.746042  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:21.749382  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.797374  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.794626  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.689246 11054 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.788467 10814 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:21.787170 10852 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:21.757663 11050 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:21.785972 10990 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:21.788866 11087 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:21.773237 11083 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:21.746922 11110 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:21.820453  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 152123us
I20260709 12:27:21.824365  4597 raft_consensus.cc:2393] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 21: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:21.830435  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.832871  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:21.745707 10942 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:21.835572  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 86305.9us
I20260709 12:27:21.845986  4647 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.848387  4647 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.833652 10553 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:21.854915 11087 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:21.859105 11113 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:21.859732 11113 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:21.859848 11073 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.870752  4756 raft_consensus.cc:2393] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 18: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:21.872479 11073 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:21.862628 11063 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.841840 11111 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:21.863348 10933 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:21.863778 11114 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.835579 11076 raft_consensus.cc:3055] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:21.864861 11054 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:21.868239 11066 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.869839 11083 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:21.860702 11113 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:21.887180  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 164173us
W20260709 12:27:21.903077  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 157102us
I20260709 12:27:21.904127  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.865419 10851 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.892069 11111 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:21.893927 11076 raft_consensus.cc:740] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:21.911203  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:21.911911  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.918280  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.914026 10933 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:21.920241  5340 raft_consensus.cc:3055] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:21.920650  5340 raft_consensus.cc:740] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:21.920713  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:21.921412  5340 consensus_queue.cc:260] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.922351  5340 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:21.927299 11102 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:21.927070 10916 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.927699 11063 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.928260 11063 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.927927 11102 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.929703 10851 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.929695 10916 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.930411 11102 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.907608 11051 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:21.939738  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:21.948091  4720 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:21.948436  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 309868us
I20260709 12:27:21.910017 11111 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:21.953197 11123 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:27:21.947117 10942 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:21.966408 10942 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:21.966853 10942 raft_consensus.cc:3060] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:21.927095 11117 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:21.969959 11117 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:21.970259 11067 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.972487 11067 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:21.972903 11117 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.973930  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.975276 11066 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.976576 11063 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:21.930039 11073 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:21.988612  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:21.980679  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.989831  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.989989  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.991510  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:21.958979  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:21.983069  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:21.992936  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.993290  4678 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:21.994390  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:21.945871 10933 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:21.987600  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.997932  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:21.998955  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:22.000039  5312 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:21.988168  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:21.980711  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:21.898532 11114 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:21.995889  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:21.995965  5340 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 12:27:22.004849  5340 pending_rounds.cc:85] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Aborting all ops after (but not including) 13
I20260709 12:27:22.005167  5340 pending_rounds.cc:107] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:27:22.005003  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:22.005622  5340 raft_consensus.cc:2889] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:21.998756  4600 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:22.007732  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:22.000276  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:22.008579  4604 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 21.
I20260709 12:27:21.997095  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:22.008755  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:22.009454  4672 raft_consensus.cc:2393] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 17: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:22.003666  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:22.009614  4602 raft_consensus.cc:3055] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:22.007578  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:22.002619  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:22.010110  4602 raft_consensus.cc:740] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:22.012633  4675 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:22.014369  4677 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:21.925268 11079 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.013993 11088 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:22.015939  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:22.016222 11127 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.004s
I20260709 12:27:22.027024  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:22.018343 11114 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:22.020292 11110 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:22.007232 10960 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:21.999439 11072 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.023799  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:22.032120  4678 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:22.025864  4599 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:21.985785 11123 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:22.028069  4602 consensus_queue.cc:260] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.030395  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:22.030017  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:22.030855  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.033725  4751 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:27:22.037077  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:22.035650  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:22.037984  4751 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 15.
I20260709 12:27:22.040117  4602 raft_consensus.cc:3060] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:22.039379  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:22.043883  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:22.056864  4673 raft_consensus.cc:1217] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Deduplicated request from leader. Original: 15.7->[16.8-20.10]   Dedup: 16.8->[18.9-20.10]
I20260709 12:27:22.057415  4673 pending_rounds.cc:85] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 8
I20260709 12:27:22.059146  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:22.060024  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:22.061259  4673 pending_rounds.cc:107] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:27:22.061640  4673 raft_consensus.cc:2889] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:22.067354  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:22.070655  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:22.075999  4678 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:22.076076  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:22.081167  4602 raft_consensus.cc:2468] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:22.083187  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:22.084398 11110 raft_consensus.cc:2804] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:22.085094 11110 raft_consensus.cc:697] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:22.086467 11110 consensus_queue.cc:237] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:22.035614 10837 consensus_queue.cc:1175] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3a5aeedd11fe49938079fe86cb01232c" responder_term: 17 status { last_received { term: 17 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 17 index: 10 } }
I20260709 12:27:22.097644  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:22.100549  4678 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:22.105497  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:22.085184 11134 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:22.051047 11114 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:22.106550  4674 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:22.060104 11073 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:22.071053 11070 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.107374 11073 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.106650 11114 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.108220 11114 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:22.107934 11070 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.111173 11070 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:22.112052  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:22.107935 11073 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:22.124836 11073 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.130781 11134 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.137609  4599 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:22.152678  4678 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:22.160318  4674 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:22.078112 11116 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:22.173801  4676 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:22.173790  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:22.192095  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 116446us
I20260709 12:27:22.192296  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:22.202639  4679 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 23.
I20260709 12:27:22.095579 11119 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:22.060003 10942 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:22.038614 11072 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.220734  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:22.221793  4678 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:22.079258 11078 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.094763 11030 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:22.228257  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 143091us
I20260709 12:27:22.230625  5342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50638 (request call id 901) took 1057 ms. Trace:
I20260709 12:27:22.230954  4569 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:22.023180 11076 consensus_queue.cc:260] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:22.194212 11134 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:22.208674 11119 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:22.231210  5342 rpcz_store.cc:276] 0709 12:27:21.173482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.173659 (+   177us) service_pool.cc:224] Handling call
0709 12:27:21.173975 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:21.174189 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:21.513671 (+339482us) log.cc:844] Serialized 14 byte log entry
0709 12:27:22.061028 (+547357us) spinlock_profiling.cc:243] Waited 8.11 ms on lock 0x7b1800349960. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee05623a 00007effee4c0a2a
0709 12:27:22.230152 (+169124us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:22.230231 (+    79us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:22.230301 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.230390 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:22.230549 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4840,"prepare.queue_time_us":294316,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":7108,"spinlock_wait_cycles":18663296,"thread_start_us":501564,"threads_started":2}
I20260709 12:27:22.244122  4752 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:22.210218 10961 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:22.229205 10916 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.251780  4751 raft_consensus.cc:1240] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:22.209038 11042 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:27:22.254575 11076 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:22.242847 11109 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:22.245594 11119 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:22.263959  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:22.272560 11114 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.277370  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:22.278304  4599 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:22.288960  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:22.289850  4675 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:22.297339 11114 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:22.257548 10916 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.307806 10916 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:22.269907 11111 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.257993 10961 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:27:22.321906 11130 log.cc:927] Time spent T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:27:22.325678 11072 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:22.322860  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 59031.8us
I20260709 12:27:22.325150  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3654) took 1052 ms. Trace:
I20260709 12:27:22.327399  4681 rpcz_store.cc:276] 0709 12:27:21.272830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.276100 (+  3270us) service_pool.cc:224] Handling call
0709 12:27:21.276399 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:21.425051 (+148652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:21.425079 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:21.425117 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:21.425144 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.324808 (+899664us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b540017f554. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:22.324882 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:22.325071 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341140864}
I20260709 12:27:22.330385 10961 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.345429  4675 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:22.304853  4599 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
W20260709 12:27:22.348374  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.350997 11109 consensus_queue.cc:1059] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:22.352903 11109 raft_consensus.cc:3055] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:22.357246 11073 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:22.322938 11117 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:22.346755  4569 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:22.363731  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:22.271692 11127 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:22.353397 11109 raft_consensus.cc:740] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:22.387250  4597 raft_consensus.cc:2468] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:22.397854  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:22.395606  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:22.400918  4599 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:22.399140  4720 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:22.412472  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:22.415177  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:22.415861  4675 raft_consensus.cc:3060] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:22.420548  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:22.424419 10875 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:22.424036 11148 consensus_queue.cc:1059] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:22.421461  4748 raft_consensus.cc:2410] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:27:22.426328 11148 raft_consensus.cc:3055] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:22.426846 11148 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:22.427804 11148 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.374341 11116 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.505903  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:22.429425  4675 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 22.
I20260709 12:27:22.491281  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:22.448935  4748 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:27:22.508136  4602 raft_consensus.cc:2368] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19.
I20260709 12:27:22.429368 11148 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:22.509582  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:22.491737  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:22.511332  4604 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:22.512923  4599 raft_consensus.cc:2468] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 22.
I20260709 12:27:22.514201  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:22.458359  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 43594us
I20260709 12:27:22.375500 11140 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:22.406499 11123 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:22.517648  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:27:22.468127 11132 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:22.454643 10942 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:22.458943 11078 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:22.494181 11134 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.442097 11052 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:22.443995 11109 consensus_queue.cc:260] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.773789 11109 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:22.775012  4597 raft_consensus.cc:2368] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16.
I20260709 12:27:22.776388 11134 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:22.430375 11111 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:22.440744 11030 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:22.514860 11116 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.400134 11127 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:22.780972 11127 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:22.781553  4602 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:22.782290 11111 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:22.782848 11111 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.785135 11111 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:22.793249  4720 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:22.794611  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:22.771011 10916 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.795837 10916 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.797070 10918 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.771135  4600 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:22.798749  4647 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:22.468479 11077 raft_consensus.cc:2804] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:22.451154 11088 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:27:22.709550  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 183618us
I20260709 12:27:22.710788 11076 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.011s
W20260709 12:27:22.749284  5311 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.769423  4604 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:27:22.811698 10132 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:22.804169  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:22.806562  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:22.807246  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 3040) took 1186 ms. Trace:
I20260709 12:27:22.807952  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:27:22.821313 11092 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 1.100s	user 0.005s	sys 0.008s
I20260709 12:27:22.813411  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:22.816937 11052 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.802022 11070 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:22.809360 10851 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.816851  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:22.809528 11075 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.805104 10795 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.816732 11116 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:22.806720 11078 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:22.811084 10942 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:22.809307 11088 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:22.896227 11070 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:22.903779 10795 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:22.904577 11088 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:22.811506 11123 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 19 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:22.799347 10970 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:22.812799 11072 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:22.811342 11111 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:22.846879 11077 raft_consensus.cc:697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:22.847411 10916 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:22.902029 11125 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:22.854635  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.924558  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:22.857647  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:22.886620  4750 raft_consensus.cc:2410] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:22.892109  4568 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:22.895825  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 91724.7us
I20260709 12:27:22.891538 10560 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:22.885061 10580 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:22.803340 10918 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:22.897650 11052 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.905735 11088 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:22.896272 10875 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:22.905854 11070 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:22.876605  4605 raft_consensus.cc:2468] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:22.914222  4645 leader_election.cc:400] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19.
I20260709 12:27:22.978958  4645 leader_election.cc:403] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 12:27:22.979471  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 71318.6us
I20260709 12:27:22.980475  4645 leader_election.cc:400] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16.
I20260709 12:27:22.980764  4645 leader_election.cc:403] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:27:22.809798 11119 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.852370 11054 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:22.918767 10970 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:22.982877 10714 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19.
I20260709 12:27:22.984530 10714 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:22.984031 11054 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:22.985559 10970 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:22.987447 11054 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:22.905838 10795 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:22.897076  4756 rpcz_store.cc:276] 0709 12:27:21.612399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.612525 (+   126us) service_pool.cc:224] Handling call
0709 12:27:22.798485 (+1185960us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b5400264154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:22.798609 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":586140800}
I20260709 12:27:22.989346  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:22.989625 11123 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.991102  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:22.991458  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:22.992681  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:22.992738  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:22.971697 10580 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:22.996701 10580 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:22.996824  4675 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:22.997747 11077 consensus_queue.cc:237] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:22.933084 11125 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.006479 11125 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:22.928661  4599 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:22.933404  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:22.931831  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.007752  4749 raft_consensus.cc:2368] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 21. Current term is 22.
I20260709 12:27:22.928561  4568 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:23.008849  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:23.009238  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 80780.5us
I20260709 12:27:23.010041 11118 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:23.010453  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:22.931953  4720 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.011214 11118 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:23.011755  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:23.012069  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 85385.2us
I20260709 12:27:23.012806  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.013708 10875 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:23.014060  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:23.014539 10875 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:23.014918 11070 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:23.013036  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:22.934952 11081 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:23.016772  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:22.993264  4751 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:22.901356 11075 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:22.902637  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:22.902381 10851 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.019663  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.022593  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:23.021613  4675 raft_consensus.cc:2468] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:23.024822  4645 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:22.920266 11072 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.015636  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:27:23.025599  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 44047.1us
I20260709 12:27:23.027760  4645 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:23.027968  4750 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43876 (request call id 3038) took 1416 ms (client timeout 1513 ms). Trace:
I20260709 12:27:23.035580  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.037307  4756 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:23.039608  4597 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 13. Current term is 14. Ops: []
W20260709 12:27:23.031149  4750 rpcz_store.cc:269] 0709 12:27:21.611693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.615714 (+  4021us) service_pool.cc:224] Handling call
0709 12:27:23.023682 (+1407968us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540004cc54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:23.027894 (+  4212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2920012032}
I20260709 12:27:23.041280  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.046372 10970 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:23.053072 11052 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:23.053206 10851 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:23.054203 10918 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:23.055419  4748 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42632 (request call id 1235) took 1041 ms. Trace:
I20260709 12:27:23.055881  4748 rpcz_store.cc:276] 0709 12:27:22.014471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.613531 (+599060us) service_pool.cc:224] Handling call
0709 12:27:22.801673 (+188142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.984440 (+182767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:22.984470 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:22.984524 (+    54us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:22.984559 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.055052 (+ 70493us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b5400183154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:23.055105 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.055343 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419871488}
I20260709 12:27:23.059456 11119 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:23.063274  4647 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.039160 11111 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:23.059513  4752 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3678) took 1089 ms. Trace:
I20260709 12:27:23.068359  4752 rpcz_store.cc:276] 0709 12:27:21.970604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.973662 (+  3058us) service_pool.cc:224] Handling call
0709 12:27:21.974025 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.245541 (+271516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.059287 (+813746us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54000bf4d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:23.059459 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618639232}
I20260709 12:27:23.076064  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.076757  4604 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 20.
I20260709 12:27:23.077870  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.078460  4597 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:27:23.080166  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.080891  4748 raft_consensus.cc:2393] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 18: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:23.073781  4571 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:23.082612  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.086877 10918 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:23.086984 11111 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:23.087797  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:23.088044 11111 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.089056  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.089048 11072 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:23.090317 11075 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:22.972468 10560 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.090905  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:23.092029  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.092363 10560 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:23.090793  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.086880  4647 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.094594  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.095739  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.096596  4597 raft_consensus.cc:2410] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:27:23.093317  4752 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:27:23.097445  4647 leader_election.cc:400] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 21. Current term is 22.
I20260709 12:27:23.097894  4647 leader_election.cc:403] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:23.099004  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.095009  4720 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:23.101541  4754 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3667) took 1999 ms. Trace:
I20260709 12:27:23.104112  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.101944  4754 rpcz_store.cc:276] 0709 12:27:21.102115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.122569 (+ 20454us) service_pool.cc:224] Handling call
0709 12:27:21.140286 (+ 17717us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:21.140665 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:21.140701 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:21.150876 (+ 10175us) log.cc:844] Serialized 26 byte log entry
0709 12:27:21.425124 (+274248us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:21.425156 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.425212 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:22.721912 (+1296700us) spinlock_profiling.cc:243] Waited 4.86 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:27:23.069782 (+347870us) raft_consensus.cc:1685] finished
0709 12:27:23.069804 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.101483 (+ 31679us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11169536,"thread_start_us":267647,"threads_started":1,"wal-append.queue_time_us":515652}
I20260709 12:27:23.106328  4647 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.108381  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.102741  4645 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:23.101765 11072 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:23.081579 10875 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.081748 10580 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:23.101403 10851 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 21. Current term is 22.
I20260709 12:27:23.128883  4720 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:23.101397 11078 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:23.111785 10970 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:23.114782 11052 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:23.114737 11117 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:23.141902  4605 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:23.116931 10933 raft_consensus.cc:2804] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:23.119216 11072 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:23.149910 11030 raft_consensus.cc:2804] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:23.132464 11081 consensus_queue.cc:1059] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:27:23.150889 10851 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:23.151369 10851 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.151926 10851 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:23.152575 11079 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:23.104349 11113 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:27:23.155296  4601 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:23.156621 10942 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:23.138082 10970 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:23.158041 10970 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:23.118753 10580 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.161841 10580 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:23.163184 11117 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.163610 11117 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:23.114822 10918 consensus_queue.cc:1059] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:27:23.136947 11078 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.166038 11078 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:23.176051  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.153339 11079 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:23.166332 10918 raft_consensus.cc:3055] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:23.149931 11072 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:23.130450 10875 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.194270 10918 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:23.194686 10714 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 15. Current term is 16.
I20260709 12:27:23.219149  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:23.243037  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.221174 10132 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:23.246804  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.262161 10132 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.263952  5340 raft_consensus.cc:2393] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 18: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:23.228412 10960 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:23.258397 10795 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:23.266434  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.267441  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.272969  4601 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:23.267976  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.204937 11073 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:27:23.271534  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 51514.7us
I20260709 12:27:23.212713 11070 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:23.272168  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.219930 11042 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:23.279194 10795 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.276571 10960 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.287199 10795 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:23.280596  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.287608 11042 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.288488  5342 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:23.290287 11042 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:23.265426 10132 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:23.290226 10960 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:23.317716 11115 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:23.318389 11115 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.280762 10875 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:23.201854 11079 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.338385  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.320858 11115 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:23.349233  4753 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 17: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
W20260709 12:27:23.359805  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 257152us
I20260709 12:27:23.360664  4645 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:23.382310  4672 raft_consensus.cc:1275] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:23.385020  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.410864  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.412041  5342 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 17: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:23.352605 11115 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:23.284335 11073 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:23.425314  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.265933 10918 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.386234 10714 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:23.430374  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 309336us
I20260709 12:27:23.384387 10933 raft_consensus.cc:697] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:27:23.442970  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.438789 10714 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.455271  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.454242  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.439741 11030 raft_consensus.cc:697] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:23.454154  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:23.461999  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.462877  4673 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:27:23.445006 10870 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.466162  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.454695 10918 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:23.447289 10916 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:23.430976 11115 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.472702  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.473297  4755 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3676) took 1503 ms. Trace:
I20260709 12:27:23.447407 10580 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.474740  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.457729 10714 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:23.476508  4677 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:23.473625  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.481475  5314 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.473697  4755 rpcz_store.cc:276] 0709 12:27:21.970009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.972690 (+  2681us) service_pool.cc:224] Handling call
0709 12:27:21.973038 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:21.973422 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:21.973457 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:22.885743 (+912286us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b2c000d0108. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:27:22.886207 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:27:23.212154 (+325947us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:23.212200 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.212293 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:23.469181 (+256888us) raft_consensus.cc:1685] finished
0709 12:27:23.469210 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.473229 (+  4019us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4397,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":12489600,"thread_start_us":325654,"threads_started":1,"wal-append.queue_time_us":553572}
I20260709 12:27:23.457266 11158 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.478111  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.477885  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.490037  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.490777  4597 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:23.492496  4675 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:23.495783  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.498801  4597 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:23.481158 11159 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:23.483894 11156 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:23.503599  4600 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:23.506484  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.505069  4680 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:23.484225 11114 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.511790  5345 raft_consensus.cc:1240] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:23.487859 10942 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:23.506240 11042 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:23.522970 10942 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.523882 11119 raft_consensus.cc:3055] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:23.524278 11119 raft_consensus.cc:740] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:23.526640 11079 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:23.485674 11158 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.530185 11158 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:23.530894  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.531566  4604 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:23.531983  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.482650 10916 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.520249 11113 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:23.535019 11113 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.518270 10960 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.531996 11030 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.536849 10960 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.512837 11156 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.538870 11156 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:23.542910  4644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.014s	sys 0.012s
W20260709 12:27:23.543367  4644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.014s	sys 0.012s
I20260709 12:27:23.547367  4756 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:23.474063 10870 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.525319 10942 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:23.509815 11076 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:23.520133 11159 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:23.544783  4681 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:23.574841  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:23.543663 11169 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:23.491218 10933 consensus_queue.cc:237] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:23.596683  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 119297us
I20260709 12:27:23.519021 10851 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.601365  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.607188  5346 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:23.582265 11158 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:23.604063  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.517270 11114 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.584654 11078 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.663390  4673 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:27:23.662876  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:23.611063  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.693497  4600 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:23.610483  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.663714  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:23.629127  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 175136us
I20260709 12:27:23.627919  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.696570  5347 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:23.583402 11168 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.669412 10560 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:27:23.683243 11092 log.cc:927] Time spent T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:27:23.708439  4754 raft_consensus.cc:1240] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
W20260709 12:27:23.711480 11164 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:27:23.717010  4674 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:27:23.718840  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:27:23.720543 11165 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.188s	user 0.000s	sys 0.003s
I20260709 12:27:23.687664 10960 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:23.669070 11073 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:23.695688 10970 consensus_meta.cc:287] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.537s	user 0.014s	sys 0.000s
I20260709 12:27:23.588685 11159 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.731189 10870 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:23.735040 10851 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:23.709331 11162 log.cc:927] Time spent T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:27:23.738999  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:23.707490 10916 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:23.740536  5345 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
W20260709 12:27:23.742225  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.744868  4720 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.745244  4600 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:27:23.746946 11170 raft_consensus.cc:2804] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:23.726720  4674 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:23.751518  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.752213  4674 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:23.613479 11172 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:23.835382 11172 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.761332 11078 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 24 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:23.762102 11171 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:23.709673 10580 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:23.723888 11115 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:23.709537 11168 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.854230  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.708316 11113 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:23.777873 11159 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:23.778102 11117 consensus_meta.cc:287] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.614s	user 0.010s	sys 0.003s
I20260709 12:27:23.724233 10560 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.705760 11158 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.600363 11169 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.753891 10970 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.779438 10961 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:27:23.759400 11170 raft_consensus.cc:697] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:23.789839  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.772990 10714 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.795859 11119 consensus_queue.cc:260] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.799289 11123 consensus_queue.cc:1059] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:23.800207 11166 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 12:27:23.877025 10942 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:27:23.802348 11081 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:27:23.886687  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.887388  5343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.803364  4569 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:27:23.888684  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 85405us
W20260709 12:27:23.889467  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.889968  4750 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:27:23.803292 11180 consensus_queue.cc:1059] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:23.807849  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.808249  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.874315 11115 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.808995  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:23.815208  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 117558us
I20260709 12:27:23.817266  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.817914  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:23.820155  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.820708  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.822185 11075 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.825903  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.896989  4604 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 19.
I20260709 12:27:23.826490  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.826556  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.827257  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:23.828794  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.834599  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.899509  4677 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:23.775727 11114 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:23.865731  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:23.866446  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:23.859776  4644 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:23.876379  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.892143  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 3061) took 1100 ms. Trace:
I20260709 12:27:23.903968  4675 raft_consensus.cc:2393] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 19: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:23.807512  4719 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:23.894562  4678 raft_consensus.cc:2393] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 20: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:23.905203  4672 raft_consensus.cc:1240] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:23.905323  4677 raft_consensus.cc:1240] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:23.906744  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 99394.7us
I20260709 12:27:23.892928  4754 raft_consensus.cc:2410] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:23.903748  4747 rpcz_store.cc:276] 0709 12:27:22.791292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.064043 (+272751us) service_pool.cc:224] Handling call
0709 12:27:23.064311 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:23.064645 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.064674 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:23.065158 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 12:27:23.483158 (+418000us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:23.483194 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.483256 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:23.799577 (+316321us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:27:23.887893 (+ 88316us) raft_consensus.cc:1685] finished
0709 12:27:23.887923 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.892076 (+  4153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3038,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"spinlock_wait_cycles":7764224,"thread_start_us":417651,"threads_started":1,"wal-append.queue_time_us":477209}
I20260709 12:27:23.900498  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:23.906479  4672 raft_consensus.cc:1240] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: []
W20260709 12:27:23.908264  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 109448us
W20260709 12:27:23.908394  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 48746.5us
I20260709 12:27:23.872722 11117 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.910023  4719 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:23.912489  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.913038  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:23.913877  4644 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:23.914984  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.916476  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.917150  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:23.917546  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:23.918503  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:23.919186  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:23.917001  4752 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:23.922601  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:23.924592  4644 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:23.927000  4571 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:23.882889 11077 consensus_queue.cc:1048] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:27:23.894101 11102 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:23.922470  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:23.901741 10795 raft_consensus.cc:3055] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:23.884194 10916 consensus_queue.cc:1059] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:27:23.900661 11167 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.934686  4647 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:23.900704 10852 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:23.933415 10795 raft_consensus.cc:740] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:23.922005 10918 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:23.922680 11114 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:23.940248  4677 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:23.931275 10875 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:23.930135 11178 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:23.930356 11118 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:23.902740 11115 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.935943 11079 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:23.936193 10851 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:23.969774 10795 consensus_queue.cc:260] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.971263 10795 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:23.972000 11163 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.444s	user 0.001s	sys 0.001s
I20260709 12:27:23.926330 10922 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:27:23.979997  4679 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:27:23.951231 11170 consensus_queue.cc:237] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.937139 11072 consensus_queue.cc:1059] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 12:27:23.909068 11075 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:23.990794  5348 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:23.951579 10875 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:23.957789 10990 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:23.978726 10851 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.992857 10851 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:23.940106 10852 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:23.991631 10875 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:23.940328 11167 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:24.010242  4674 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:24.016443 11119 raft_consensus.cc:3060] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:24.040889 11185 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:27:24.051568  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 132452us
I20260709 12:27:24.052116 11073 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:24.056945  5312 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:24.072705  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:24.073863 10132 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:24.074496 10132 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:24.075626  5312 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:24.078114  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:24.079380  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:24.080024  4603 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
W20260709 12:27:24.060818 11156 consensus_meta.cc:287] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.521s	user 0.020s	sys 0.000s
I20260709 12:27:24.061362 11117 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.061882 10970 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:24.066382  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 131766us
I20260709 12:27:24.062186 11072 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:24.071410  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:24.085340 10132 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:24.085788 10132 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:24.086122 10132 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:24.087214  5312 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:24.083105  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:24.071261 11079 raft_consensus.cc:3055] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:24.091012 11079 raft_consensus.cc:740] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:24.091329 10961 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:24.091730 10961 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.092132 10961 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:24.091881 11079 consensus_queue.cc:260] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:24.093287 11079 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:24.106308  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.084622 11105 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:24.111449 10970 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:27:24.143385  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:24.115844  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 176893us
I20260709 12:27:24.153692 11189 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.154239 11189 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:24.155614 11111 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:24.156545 11189 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:24.156855 11167 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:24.142076 11169 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:24.142511 11078 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:24.157917 11078 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:24.158102 11115 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.158852 11078 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.159854 11078 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:24.160022 11115 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:24.160632 11115 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.141585 10714 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.163431  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:24.163725  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:24.164214  4680 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:24.164767  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:24.114579 11155 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:24.165395  4603 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:24.105187 11072 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.084826 11156 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:24.168948 11072 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.169925  5344 raft_consensus.cc:2368] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17.
I20260709 12:27:24.170948 11072 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.171559 11156 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:24.172235 11075 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.174256 11192 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:24.175006 11192 consensus_queue.cc:1243] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:24.175540 11117 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:24.175909 11117 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:24.176400  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:24.177018  4750 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:24.176671 11117 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.178262 11117 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:24.178547 11115 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.179351 11158 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.180549 10560 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.181408 10851 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.183511 10851 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.188618  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:24.191004  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:24.191680  4603 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:27:24.185500  5314 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:24.199271  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 265587us
I20260709 12:27:24.204157  4604 raft_consensus.cc:1275] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:24.205309 10132 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:24.094816 11030 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 12:27:24.180974  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:24.211325  4604 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:27:24.211714  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:24.236249  4750 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:24.212852  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:24.213963  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:24.237887  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 52466.8us
I20260709 12:27:24.220511  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:24.222185  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:24.239342  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:24.081606 11185 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.222960  4644 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:24.215459 10132 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.243994  4644 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:24.244743  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:24.158052 11168 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:24.270313  5346 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:27:24.248347  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:24.259126  4568 leader_election.cc:400] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17.
I20260709 12:27:24.272293  4681 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:27:24.281988  4568 leader_election.cc:403] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:24.281876  4673 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:24.284293  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:24.284904  4673 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
I20260709 12:27:24.284586  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:24.269868  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:24.287106  5348 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:24.247169  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:24.289209  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:24.284519  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:24.289870  4681 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:24.246836  4674 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:24.297147  4603 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:24.198464 10852 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.299686  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:24.302644  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:24.306087  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:24.307624  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:24.300647  4569 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:24.310093  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:24.310597  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:24.310741  4674 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:24.233986 11102 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:24.311985  4644 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:24.313015  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 229967us
I20260709 12:27:24.322408  4645 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:24.323199  4756 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:24.257370 11172 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.326829  4676 raft_consensus.cc:3060] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:24.256754 10851 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:24.266806 11127 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:27:24.328112  4501 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52818 (request call id 304) took 53244 ms (client timeout 15000 ms). Trace:
I20260709 12:27:24.330757  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:24.336977  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:24.333389  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:24.331418  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:24.338205  4680 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:24.333608  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:24.340134  4678 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:24.332794  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:24.334882  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:24.341656  4747 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:24.337774  4751 raft_consensus.cc:2410] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:24.332048  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:24.331043  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:24.341292  4681 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:24.342970  4749 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:24.343453  4672 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 17. Current term is 18. Ops: [13.7-17.9]
I20260709 12:27:24.333822  4571 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:24.334812  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:24.329416  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:24.344930  4645 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:24.346513  4645 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:24.348140  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:24.348789  4681 raft_consensus.cc:3060] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:24.350008  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:24.354521  5312 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:24.354771  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:24.345988  4647 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:24.366876  4681 raft_consensus.cc:2468] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
W20260709 12:27:24.338717  4501 rpcz_store.cc:269] 0709 12:26:31.084159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.086435 (+  2276us) service_pool.cc:224] Handling call
0709 12:27:24.328036 (+53241601us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:20.535806 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:20.766290 (+230484us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:20.789022 (+ 22732us) write_op.cc:435] Acquired schema lock
0709 12:27:20.793655 (+  4633us) tablet.cc:662] Decoding operations
0709 12:27:20.807141 (+ 13486us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:20.807236 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:20.807276 (+    40us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:27:20.807818 (+   542us) tablet.cc:701] Row locks acquired
0709 12:27:20.807870 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:27:20.808055 (+   185us) write_op.cc:270] Start()
0709 12:27:20.808230 (+   175us) write_op.cc:276] Timestamp: P: 1783600040807982 usec, L: 0
0709 12:27:20.808256 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:27:20.811116 (+  2860us) log.cc:844] Serialized 1694 byte log entry
0709 12:27:21.170023 (+358907us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b1e28 00007efff1eec581 00007efff1eebd58 00007efff1ef09f1 00007efff1ef09a9 00007efff1ef0939 00007efff1ef0901 00007efff1eefb2d 0000000000388634 0000000000388479
0709 12:27:21.682848 (+512825us) op_driver.cc:464] REPLICATION: finished
0709 12:27:21.895597 (+212749us) write_op.cc:301] APPLY: starting
0709 12:27:21.898357 (+  2760us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:21.970862 (+ 72505us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:21.970905 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:21.973241 (+  2336us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:21.973473 (+   232us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:27:21.973515 (+    42us) write_op.cc:312] APPLY: finished
0709 12:27:22.013657 (+ 40142us) log.cc:844] Serialized 49 byte log entry
0709 12:27:22.022304 (+  8647us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:22.022418 (+   114us) write_op.cc:454] Released schema lock
0709 12:27:22.022460 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:22.029915 (+  7455us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:22.032488 (+  2573us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1771039,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":161885,"apply.run_cpu_time_us":6954,"apply.run_wall_time_us":131992,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20823,"num_ops":4,"prepare.queue_time_us":349322,"prepare.run_cpu_time_us":22668,"prepare.run_wall_time_us":754533,"raft.queue_time_us":480519,"raft.run_cpu_time_us":13904,"raft.run_wall_time_us":180694,"replication_time_us":855209,"spinlock_wait_cycles":4868352,"thread_start_us":854232,"threads_started":4,"wal-append.queue_time_us":316663,"wal-append.run_cpu_time_us":26644,"wal-append.run_wall_time_us":1874846}]]}
I20260709 12:27:24.283854 11073 consensus_queue.cc:1059] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
W20260709 12:27:24.500926  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.501670  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 147220us
I20260709 12:27:24.321714 11167 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:24.524271  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 190524us
I20260709 12:27:24.518646  4678 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:24.506928  5312 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:24.532135  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 174013us
W20260709 12:27:24.534354  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 199642us
I20260709 12:27:24.617326  4569 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:24.547016 11168 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:24.539984 10852 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:24.335927 11118 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17.
I20260709 12:27:24.533413 11114 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:24.479983 11125 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:24.533484 11158 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:24.525065 11136 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:24.615286  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 269342us
W20260709 12:27:24.754341 11193 log.cc:927] Time spent T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:27:24.333009 11172 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.218s
I20260709 12:27:24.540256 10916 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:24.337081 11127 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:24.540643  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 194201us
W20260709 12:27:24.618605  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.547760 11192 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:24.787668 11192 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:24.788132 11078 consensus_meta.cc:287] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.628s	user 0.010s	sys 0.000s
I20260709 12:27:24.619712  4747 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 20.
I20260709 12:27:24.711417 11189 consensus_queue.cc:1059] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:24.788323 11192 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.789709 11189 raft_consensus.cc:3055] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:24.790110 11189 raft_consensus.cc:740] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:24.790923 11189 consensus_queue.cc:260] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.598085 10933 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:24.596045 10990 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:24.522454 11042 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:24.674049 11190 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:24.670944 11159 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:24.704042 11195 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:24.739099  4568 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:24.740514  4676 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:24.703138 11168 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:24.375219  4495 catalog_manager.cc:5697] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 12 to 20, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 20 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:24.792505 11127 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:24.665849  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.757313  4750 consensus_meta.cc:287] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.515s	user 0.007s	sys 0.000s
W20260709 12:27:24.764578  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 146189us
W20260709 12:27:24.800850 10961 consensus_meta.cc:287] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.708s	user 0.011s	sys 0.000s
I20260709 12:27:24.545794 11176 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:24.618801 11102 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.561379 10795 raft_consensus.cc:2804] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:24.678223 11170 consensus_queue.cc:1048] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 12:27:24.764479 11114 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.764931 11125 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.794518  4647 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:27:24.773172 11119 consensus_meta.cc:287] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.701s	user 0.010s	sys 0.000s
I20260709 12:27:24.758749 11115 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:24.798234  4677 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:24.767262 11158 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:24.802603  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 63620.6us
I20260709 12:27:24.775077 10916 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.594904 11030 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:24.753510 11079 consensus_meta.cc:287] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.648s	user 0.005s	sys 0.001s
I20260709 12:27:24.806967  4750 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:24.807458 11169 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:24.802001 11159 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.844223 11158 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:24.851146 11159 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:24.821902  4569 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:24.801265 10933 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.809037 11210 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:24.801172 10990 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:24.811445 11185 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:24.834878  4495 catalog_manager.cc:5697] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:24.873759  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:24.839789 11115 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.875028  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:24.843379  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:24.876570  4599 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:24.843601  4677 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 12:27:24.834089 10875 consensus_meta.cc:287] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.835s	user 0.010s	sys 0.007s
I20260709 12:27:24.845520  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:24.838155 11117 consensus_meta.cc:287] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: Time spent flushing consensus metadata: real 0.659s	user 0.008s	sys 0.002s
I20260709 12:27:24.834504 11114 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:24.886261 10875 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:24.886904  4495 catalog_manager.cc:5697] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 8 to 12, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 12 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:24.888514 10875 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:24.849110 11169 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:24.889201 10990 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:24.835565 11125 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:24.889910  4748 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:24.891268  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 150630us
I20260709 12:27:24.891633  4601 raft_consensus.cc:2410] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:27:24.858402  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:24.838306 11172 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:24.861477  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:24.863164 10933 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:24.863539  4676 raft_consensus.cc:1240] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:27:24.854452 11171 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.845443 11030 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.844290 10916 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:24.866852 10560 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.844619 11168 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.844461 11119 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:24.834987 10795 raft_consensus.cc:697] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:24.822775 10961 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.881349  4720 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:24.893976  4603 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:24.873121 11211 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:24.898268 11136 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:24.898325 11189 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:24.926199  4604 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:27:24.902009  4571 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:24.889854 11210 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:24.904662 11172 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 20 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:24.928674 11172 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.915163 10961 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:24.904450 11171 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.915872  4753 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 18: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:24.916656 10795 consensus_queue.cc:237] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:24.917593 11119 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.917789 11030 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:24.888551  4495 catalog_manager.cc:5697] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:24.940989 11117 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:24.920184 11195 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:24.907774  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:24.923658  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:24.960561  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 138769us
I20260709 12:27:24.960738  4603 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:27:24.901527  4599 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 20.
I20260709 12:27:24.890839 11169 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.931558  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:24.932518  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:24.934854  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:24.935760  4645 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:24.909157 11078 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:24.905851 10560 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:24.939550  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:24.940955 10132 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:24.945960  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:24.947467 11171 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:24.964027  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:24.949002  4495 catalog_manager.cc:5697] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 3 to 10, term changed from 16 to 18, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 18 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:24.970000 11115 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.970204 11119 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.970362 10918 raft_consensus.cc:493] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:24.970007 11117 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:24.974869 11158 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:24.976855  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:24.982069  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 74383.9us
W20260709 12:27:24.985507  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49831.9us
I20260709 12:27:24.990150 10132 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:24.990939  4600 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:24.998173  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:24.999234  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:25.006471  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:25.007086  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:25.011160  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:25.010054 11180 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:25.014171 11171 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:25.015106 11176 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:25.015552 11171 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:25.016088 11171 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:25.016112 10916 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.013644  4495 catalog_manager.cc:5697] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:24.989236 11190 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:24.988881 11213 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:25.020778  4645 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:24.990528 11192 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:24.996735 11169 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:25.030212  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 45331.5us
I20260709 12:27:24.927248 11211 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.002987 11079 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.035391  4600 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:25.038023  4605 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 20: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:25.042136  4569 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:25.042613 10132 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:25.017060 11171 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.029402 11111 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:25.005080 11115 consensus_queue.cc:1059] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:27:25.001024 11159 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.036134 10580 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.036042 11178 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:25.017071 11125 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.010511 10918 raft_consensus.cc:515] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.060164  4604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:25.004557 11114 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.061543  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.062474  4603 raft_consensus.cc:1275] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:25.062776  4599 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:25.073789  4569 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:25.074791  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.072347 11219 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:27:25.072556 11178 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.082333 11155 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.082901 11155 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.083841 11178 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:25.044221 10970 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:27:25.086297 11155 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.087401 10916 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.088735  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:25.089296  4749 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:27:25.044978 11077 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:27:25.090713 11125 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:25.091918  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:25.092746  4750 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 24: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:25.092854 10918 leader_election.cc:290] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:25.072387  4604 raft_consensus.cc:606] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:25.094689  5340 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:25.095721  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:25.096323  5347 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 24.
I20260709 12:27:25.097268  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:25.099200  4644 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:25.099462 11077 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:25.100097 11190 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:25.100507 11077 raft_consensus.cc:3055] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:25.100582 11190 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:25.101044 11077 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:25.046298 11217 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:25.013696 11180 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.103814  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8e62ff81eb34853811314530131292b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:25.105367  4647 leader_election.cc:304] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:25.106438 11200 raft_consensus.cc:2749] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:25.049912 11079 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.109066 11079 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.103145 11077 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.112210  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:25.141119  4753 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 24.
I20260709 12:27:25.112607  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:25.046331 10933 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:25.043932 10961 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:27:25.037809 11197 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:25.115679  4749 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 20.
W20260709 12:27:25.117381  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 64262.9us
I20260709 12:27:25.125247  4604 raft_consensus.cc:1275] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:27:25.151044  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.150424  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:25.157315  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 83626.3us
I20260709 12:27:25.064067 11192 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:25.114336  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:25.152154  4672 raft_consensus.cc:2368] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 19. Current term is 20.
I20260709 12:27:25.105171 11180 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.158195  4569 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:25.156809  4571 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:25.146293 11127 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.162773  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.140178 11190 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:25.163460  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:25.046909 11071 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.103s
I20260709 12:27:25.139688 11156 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:25.192009  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:25.171826  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.150501 10918 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:25.072870 10580 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.194651  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:25.176975  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.194715  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.198073  4678 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:25.199065 11070 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:27:25.206121  5344 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 18: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:25.217207  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.219326  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.219791  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:25.220548  4604 raft_consensus.cc:3060] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.221264  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:25.222012  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.222977  4675 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 16: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:25.223824  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.231762 10922 consensus_queue.cc:1048] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:25.199357 10970 raft_consensus.cc:476] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Not starting pre-election -- already a leader
W20260709 12:27:25.233759 11229 log.cc:927] Time spent T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.086s	user 0.002s	sys 0.000s
I20260709 12:27:25.239233  4569 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:25.239997 10580 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.240718  4569 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:25.241254 10560 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.241813 10560 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:25.243257 11159 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.199255 11156 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.190901 10990 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:25.248049 10990 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 20 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:25.239724 11171 consensus_queue.cc:1048] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:25.248929 11171 consensus_queue.cc:1243] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:25.163226 11195 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:25.248942 10990 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.251935  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:25.252803 11156 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:25.253777 11127 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.254359  5345 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:27:25.255002  5345 raft_consensus.cc:606] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:25.255378  5345 raft_consensus.cc:614] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:25.256564 11211 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:25.259151  4604 raft_consensus.cc:2468] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:27:25.073609 11117 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.190047 11105 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:25.190398 11192 raft_consensus.cc:697] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:25.191432 11214 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:25.260453  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:25.228431  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.202454 11119 raft_consensus.cc:993] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:25.262650  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:25.273073  4752 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 18: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:25.157087 10933 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.274355 11194 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:25.279304 11158 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:25.280896 10933 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:25.282045  4601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 2354) took 1095 ms. Trace:
I20260709 12:27:25.281867 11192 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.282399  4601 rpcz_store.cc:276] 0709 12:27:24.187069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.188327 (+  1258us) service_pool.cc:224] Handling call
0709 12:27:25.281856 (+1093529us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b54002e4fd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:25.281988 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1614482304}
I20260709 12:27:25.283315 11218 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:25.268846  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.289675 11217 raft_consensus.cc:3055] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:25.266234  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:25.290184 11217 raft_consensus.cc:740] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:25.290277  5344 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50638 (request call id 919) took 1021 ms. Trace:
I20260709 12:27:25.282666 11119 raft_consensus.cc:1081] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:25.290761  5344 rpcz_store.cc:276] 0709 12:27:24.269215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.269327 (+   112us) service_pool.cc:224] Handling call
0709 12:27:25.290071 (+1020744us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b540055e6d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:25.290213 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2076409600}
I20260709 12:27:25.291054 11217 consensus_queue.cc:260] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.204918  4495 catalog_manager.cc:5697] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 14 to 15, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 15 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:25.293749  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:25.293687  4495 catalog_manager.cc:5697] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 7 to 18. New cstate: current_term: 18 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:25.294574 11218 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:25.295684  5312 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:25.295687  4495 catalog_manager.cc:5697] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:25.263872 11105 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.297802 11105 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:25.277212 11224 log.cc:927] Time spent T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:27:25.292469 11217 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:25.302333  4752 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:27:25.199298 10918 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.302973  4752 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:25.303483  4752 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.303738  4676 raft_consensus.cc:1240] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:25.304693 10918 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.307785 11174 raft_consensus.cc:2804] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:25.308449 11174 raft_consensus.cc:697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:25.309432 11174 consensus_queue.cc:237] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.312819  4644 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:25.314561  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:25.315445  4750 raft_consensus.cc:2393] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 20: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:25.314473  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.318396  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:25.269773  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:25.326025 11114 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:25.321930 11167 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:25.323443 10918 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.327417 10918 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.329913  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:25.332015 11077 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:25.332343  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:25.333158  4605 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:25.334905 11206 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.504s	user 0.000s	sys 0.005s
I20260709 12:27:25.336858  4679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 2352) took 1080 ms. Trace:
I20260709 12:27:25.337234  4679 rpcz_store.cc:276] 0709 12:27:24.261806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.262090 (+   284us) service_pool.cc:224] Handling call
0709 12:27:24.262516 (+   426us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:24.262980 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.263022 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:24.283351 (+ 20329us) log.cc:844] Serialized 52 byte log entry
0709 12:27:24.539929 (+256578us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:24.952943 (+413014us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b1800147a60. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effee05e4bc 00007effeb3ac48b 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c6455 00007efff1eae3c7 00007effee4c09a7 00007effee4d00bf
0709 12:27:24.953075 (+   132us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.953266 (+   191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.953337 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:25.325031 (+371694us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:27:25.336641 (+ 11610us) raft_consensus.cc:1685] finished
0709 12:27:25.336664 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.336798 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":242166,"prepare.queue_time_us":255940,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":39466,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":39325,"spinlock_wait_cycles":7861376,"thread_start_us":455039,"threads_started":2,"wal-append.queue_time_us":487160}
I20260709 12:27:25.339061  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:25.339355 11197 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.342754 11197 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.345037 11197 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.334062 10918 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.348820 11210 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.361205  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.362327  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.348791 11156 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:25.351176  4752 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.351516  4568 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:25.354107  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:25.354854  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:25.367568  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:25.368268  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.370050  4569 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:25.371292  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.382507  4680 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 16: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:25.374413  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:25.377035  4605 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:25.386929  4605 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:25.388417  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.389007  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:25.389810  4599 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 18: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:25.389050  4674 raft_consensus.cc:2393] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 18: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:25.390638  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.393764  4601 pending_rounds.cc:85] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 12
I20260709 12:27:25.394177  4601 pending_rounds.cc:107] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:27:25.394491  4601 raft_consensus.cc:2889] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:25.396538  4752 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:25.396799  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.398545  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.399816  4680 raft_consensus.cc:2393] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:25.397946  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:25.407629  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:25.407897 11235 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:25.408936 10933 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:25.400952 11159 consensus_queue.cc:1059] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:27:25.384701 11210 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.439955 11194 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:25.440258 11210 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.440362 11167 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:25.440603 11194 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:25.447134 11190 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.298457  4495 catalog_manager.cc:5697] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 13 to 17, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 17 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:25.480458  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 110387us
I20260709 12:27:25.485985  4605 raft_consensus.cc:606] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:25.590008  4605 raft_consensus.cc:614] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:25.592904  4569 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:25.490164  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:25.493059  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:25.594260  4674 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:25.595141  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 105604us
I20260709 12:27:25.508864 11052 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:25.533577 11159 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.598304 11174 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:25.572192 11119 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:25.584796  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:27:25.586558  4719 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.598229  4675 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 24.
I20260709 12:27:25.609205  4599 raft_consensus.cc:3055] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:25.609612  4599 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:25.609872  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.610894  4599 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.612716  4599 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.589473  4495 catalog_manager.cc:5697] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 12 to 17, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 17 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:25.598130 11052 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.616899 11052 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:25.617213 11169 consensus_meta.cc:287] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.575s	user 0.010s	sys 0.002s
I20260709 12:27:25.614779  4495 catalog_manager.cc:5697] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:25.599009 11159 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.631805  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:25.651573  4605 raft_consensus.cc:2393] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 16: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:25.659176 11078 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.660181 11078 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:25.666483  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 73890.8us
I20260709 12:27:25.662006  4495 catalog_manager.cc:5697] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 10 to 18, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 18 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:25.675356 11215 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:25.682736 11189 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:25.719363 11242 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:27:25.710218 11079 raft_consensus.cc:3055] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:25.721058 11079 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:25.708595  4604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:25.722509  4604 raft_consensus.cc:606] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:25.724349 11079 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.726116 11079 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:25.732997  4678 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:25.738449  4749 raft_consensus.cc:1240] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:25.740103  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:25.740355  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:25.740410  5312 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:25.741243  4747 raft_consensus.cc:2393] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 24: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:25.740927 11189 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.742010 11127 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:25.742676 11127 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:25.743904 11189 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.744153 11127 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.746166 11190 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:25.748456  4598 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:27:25.749981  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.750989  4603 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
W20260709 12:27:25.740489  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 43965.2us
I20260709 12:27:25.787003  4747 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:25.787143  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:25.761215  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:25.790731 11159 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.792906  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:25.798280 11244 log.cc:927] Time spent T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.066s	user 0.003s	sys 0.001s
I20260709 12:27:25.783977  4720 leader_election.cc:400] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 19. Current term is 20.
I20260709 12:27:25.803237  4679 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:25.806640  4720 leader_election.cc:403] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:25.806842 11078 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:25.806931  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.807909  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.807654 11114 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 19. Current term is 20.
I20260709 12:27:25.806648  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.803859  4680 raft_consensus.cc:1275] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:27:25.817235  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:25.829268  4674 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
W20260709 12:27:25.833122  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.836899  4599 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:25.837978  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:25.840147  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:25.818832 11159 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:25.840685 11114 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:25.841328 11159 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.842567  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:25.843710 10875 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:25.844444  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:25.845989 11169 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.846345  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:25.847046 11114 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:25.847707 11078 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:25.850167  4673 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:25.855141 11192 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:27:25.870990  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.874737 11218 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:25.874202 11194 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:25.878719 11194 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.879716  4598 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:25.891323  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:25.893194 11185 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:25.893965 11185 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:25.893535  4495 catalog_manager.cc:5697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 11 to 15, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 15 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:25.894752 11218 consensus_queue.cc:1048] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:25.895162 11185 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:25.902894 11077 consensus_meta.cc:287] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.014s
I20260709 12:27:25.865605 11159 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:25.904379  4604 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:25.903929  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:25.917538 11169 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:25.937805  4752 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:25.896237  4495 catalog_manager.cc:5697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:25.953857  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:25.969487  4672 raft_consensus.cc:1240] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 15. Current term is 16. Ops: []
W20260709 12:27:25.970449 11105 consensus_meta.cc:287] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: Time spent flushing consensus metadata: real 0.672s	user 0.009s	sys 0.019s
I20260709 12:27:25.971159 11105 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:25.966710  4569 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:25.987191 11192 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:26.004757 11192 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.005282 11192 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:26.000262  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:26.017449  4495 catalog_manager.cc:5697] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:26.025230  4598 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:26.027818  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:26.038437  4600 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abf423ac214b42acbe1696a11279567c"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:26.039281  4600 raft_consensus.cc:606] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:26.039745  4600 raft_consensus.cc:614] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:26.042838  5346 raft_consensus.cc:2368] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 24. Current term is 25.
I20260709 12:27:26.044224  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:27:26.055735  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:26.058749  4647 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.067907  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:26.072541 11105 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 25 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:26.073805  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:26.073949  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:26.074930  4747 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:26.089746 11030 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:27:26.090626 11030 consensus_queue.cc:1243] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 5c3201048f8346009aefca07428828f3 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:26.091954 11211 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:26.094791  4747 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 25.
I20260709 12:27:26.096462  5314 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:26.097718 11088 raft_consensus.cc:2804] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:26.098469 11088 raft_consensus.cc:697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 25 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:26.099704 11088 consensus_queue.cc:237] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:26.064013 11189 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.106005 11189 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:26.108573 11189 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:26.129812 10918 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.130481 10918 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:26.133167 10918 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:26.145368  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:26.142863  4495 catalog_manager.cc:5697] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:26.145457  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:26.117871 11042 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.152772  4681 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:26.066213 11077 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 12:27:26.156944  5344 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:26.157533 11077 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:26.157910 11077 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:26.153043  4680 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:26.156415 11042 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:26.154631  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:26.164011  4672 raft_consensus.cc:3060] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:26.169440 11042 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:26.170387  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:26.159107 11077 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:26.154209 11159 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.179908  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:26.183386  4680 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:26.190511 11159 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:26.194062  4678 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:26.195366  4603 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:26.201478  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:26.196309 11192 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:26.203154 11174 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:26.203317 11077 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:26.203786 11174 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.204209 11174 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:26.215278  4495 catalog_manager.cc:5697] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 11 to 18. New cstate: current_term: 18 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:26.228384  4680 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:26.206610 11192 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:26.218997 10918 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:26.242981 11127 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:26.244702  4598 raft_consensus.cc:1275] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:26.245750  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:26.247618 11159 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:26.262420  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 192336us
I20260709 12:27:26.267594  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:26.264344  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:26.265959 10918 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:26.270932  4495 catalog_manager.cc:5697] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:26.273132 11078 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:26.284391 11210 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:27:26.285667  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:26.288216 10916 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:27:26.287096 11078 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.282886  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:26.299683  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 154526us
I20260709 12:27:26.264494 11215 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:27:26.306734  4750 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:26.313109  4672 raft_consensus.cc:2468] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 25.
I20260709 12:27:26.314851  4747 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:26.309624 11078 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:26.321202 10918 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:26.327766  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:26.330124  4645 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:26.341750  4750 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 11.
I20260709 12:27:26.345927  4644 leader_election.cc:400] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 election: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 24. Current term is 25.
I20260709 12:27:26.346446  4644 leader_election.cc:403] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 12:27:26.354413  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:26.353734 11159 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:26.353955 11190 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067) with higher term. Message: Invalid argument: T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for earlier term 24. Current term is 25.
I20260709 12:27:26.376441  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:26.377084 11254 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.188s	user 0.000s	sys 0.003s
I20260709 12:27:26.377246  4674 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 12.
I20260709 12:27:26.347879 11174 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:26.383287  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 101873us
W20260709 12:27:26.386499  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63137.6us
W20260709 12:27:26.387056 11256 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.200s	user 0.002s	sys 0.000s
I20260709 12:27:26.387344  4645 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:26.388919  4645 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
W20260709 12:27:26.386561 11259 log.cc:927] Time spent T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.098s	user 0.002s	sys 0.000s
I20260709 12:27:26.393347  4747 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:26.395675 11156 consensus_queue.cc:1048] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:26.398854  4571 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:26.401374 11156 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:26.414567  4752 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 20.
I20260709 12:27:26.427361 11262 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.427646 11071 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:26.434396  5348 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 18: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:26.421427 11190 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:26.436463  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:26.448760  4672 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:26.450755  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:26.435940 11211 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:26.452661  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 63811.3us
I20260709 12:27:26.452994 11079 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:27:26.453747 11265 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:27:26.456182 11174 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:26.459494  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.462843 11078 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:26.464787 11210 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:26.465622 11210 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:26.467052  4749 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:27:26.469821  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 57658.7us
I20260709 12:27:26.465653 11077 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.471578 11217 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:26.472198 11217 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:26.471354 11077 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:26.453670  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:26.433241 11071 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.465233 11078 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:26.477608 11169 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:26.479588 10990 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:26.482040  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:26.492415  4603 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:26.483227  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:26.493264  4752 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:26.480949 11071 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:26.512075  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:26.513808  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:26.514421  4677 raft_consensus.cc:3060] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:26.501586 11190 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.515998 11190 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:26.428092 11262 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:26.542829 11210 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:26.561152 11268 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:27:26.575807  4748 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:26.576295  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 177582us
I20260709 12:27:26.578135  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:26.579046  4753 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
W20260709 12:27:26.582317  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 128793us
I20260709 12:27:26.584659  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:26.584731  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:26.586156  4569 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:26.586351 11159 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:26.586995 10990 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:26.587522 10990 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.587719  4645 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:26.588045 10990 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:26.611766  4672 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
W20260709 12:27:26.642187  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 62660.9us
I20260709 12:27:26.655844  4679 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:26.666261 11262 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:26.668016 11190 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:26.668397  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 232718us
I20260709 12:27:26.670475  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:26.678114 11190 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:26.678104  4603 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:26.682252  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:26.682400 10990 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:26.683457  4748 raft_consensus.cc:2410] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:27:26.684635 10990 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:26.685309 10990 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:27:26.715608  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 125937us
I20260709 12:27:26.727946  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:26.728868  4600 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:26.735802  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:26.731314  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:26.736445  5312 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:26.718043 11190 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:26.740873  4603 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19. Ops: [16.9-18.10]
W20260709 12:27:26.741797  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 65969.2us
I20260709 12:27:26.695175 11078 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:26.735299 10875 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.743304 11190 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:26.777839  4605 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:26.782352  4602 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:26.781301 10875 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:26.790656 10875 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:26.792490 11115 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:26.793215 11115 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:26.786691 10916 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.794180 11105 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:26.794270 10916 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:26.796622  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 209015us
I20260709 12:27:26.797709  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:26.803848  4600 raft_consensus.cc:2468] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:26.788146 11190 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:26.797238 11119 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.787563 11282 raft_consensus.cc:2804] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:26.795497 11115 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:26.798659 11078 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:26.826552  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 131689us
I20260709 12:27:26.837740  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:26.838636  5345 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:26.839906  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:26.840631  4600 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:26.842103  4604 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:27:26.845942 11264 log.cc:927] Time spent T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.367s	user 0.002s	sys 0.014s
I20260709 12:27:26.849179  4677 raft_consensus.cc:2468] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:26.825038 11282 raft_consensus.cc:697] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
W20260709 12:27:26.879437  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 131668us
W20260709 12:27:26.882642  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 56935.7us
W20260709 12:27:26.888207  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 125104us
I20260709 12:27:26.898844  4569 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:26.899219 11159 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:26.900086  4721 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:26.901291  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 71671.2us
I20260709 12:27:26.828883 11119 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:26.973304  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 84355.6us
I20260709 12:27:26.994813 11159 consensus_queue.cc:1059] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:26.999224  4605 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:26.994534 11282 consensus_queue.cc:237] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.003552  4747 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 25 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:27:27.004892  4602 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:27.022006  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:27.025876  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 127098us
W20260709 12:27:27.027720  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 70943.9us
I20260709 12:27:27.016607 11115 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:27.030941  4647 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:27.021793 11159 consensus_queue.cc:1243] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:27.036605  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.038925  4600 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 15.
I20260709 12:27:27.035208 11078 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:27.039656  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.032819 11296 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:27:27.070145 11277 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:27:27.050032 11042 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 12:27:27.071744 11298 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
W20260709 12:27:27.073915  4720 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.89.194:42599 timed out after 1.657s (SENT) [suppressed 3 similar messages]
I20260709 12:27:27.070013 11159 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:27.074625 11159 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:27.075209 11159 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:27.082839  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:27.083446 11272 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 12:27:27.084147  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.085021  4604 raft_consensus.cc:2410] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
W20260709 12:27:27.088485 11071 consensus_meta.cc:287] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.576s	user 0.015s	sys 0.000s
I20260709 12:27:27.088922 11071 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.091769 11071 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.098148  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:27.101724  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 59766.5us
I20260709 12:27:27.111389  4677 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:27.109619 11299 log.cc:927] Time spent T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:27:27.117975  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.121032 11194 raft_consensus.cc:993] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:27.127197 11194 raft_consensus.cc:1081] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:27.132150  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:27.132782  4747 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:27:27.128348 11052 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:27:27.139093 11283 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:27:27.112778 11071 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:27.140543  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.142134  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:27.104029 11190 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.121182 11114 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:27.119268 11119 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.141170 11071 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.098495 11296 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:27.172461  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 98579.9us
I20260709 12:27:27.181533  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.182376  4681 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 21.
I20260709 12:27:27.182809  4677 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:27:27.187774  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 205616us
I20260709 12:27:27.197192  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:27.181242 11115 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:27.198959  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.199278 11119 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:27.192405 11114 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.209858  4596 raft_consensus.cc:2393] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 18: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:27.200841 11194 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:27.215236  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.222577 11285 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:27:27.223572 11285 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:27.223920 11285 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:27.224501 11071 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.226354  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.227173  4676 raft_consensus.cc:2368] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
I20260709 12:27:27.229179 11190 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.213495  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:27.231015 11159 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.232182 11071 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:27.232743 11071 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.233335 11159 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.235180 11071 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.235553  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.236267  4751 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:27.239668  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.240420  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:27.244366  4749 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:27:27.246688  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:27.247344  4600 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:27.249107 11289 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:27.252465 11286 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:27.253057 11286 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.256577  4600 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:27.260918  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:27.257120 11289 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.269731  4755 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:27.270058 11289 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:27.262024  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.263360 11115 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.192636 11279 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:27:27.265801 11296 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.230135 11119 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 20 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:27.224809 11285 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.297526  4681 raft_consensus.cc:1275] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 25 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
W20260709 12:27:27.302101  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 56421.9us
I20260709 12:27:27.302220  4719 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:27.306636  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 91675us
I20260709 12:27:27.307717  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:27.312839  4645 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:27.313184  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
W20260709 12:27:27.332737  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.302898 10875 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:27.339835 11119 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.347178 11321 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:27.347632 11321 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.347959 11321 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:27.338840  4747 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:27.339663 11313 consensus_queue.cc:1059] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:27:27.345541  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:27.359488  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 46729.8us
I20260709 12:27:27.317325 11215 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:27:27.361352  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 59200.7us
I20260709 12:27:27.361776 11319 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:27.366050 11318 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:27.366703 11318 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.373615  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.310966 11262 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:27:27.375161  4645 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:27.346179 11285 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:27.382483  4645 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:27.377908 11319 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.393824 11319 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:27.398841  4647 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:27.368800 11296 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:27.406944 11290 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:27.359588 10875 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.413326  4645 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:27.414824  4571 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:27.383739  4751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:27:27.423044  4751 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:27.423527  4751 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:27.429626 11310 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:27:27.367920 11321 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.410300 11313 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:27.435961  4755 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:27.447970  4647 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:27.449628 11311 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.136s
I20260709 12:27:27.450549 11330 raft_consensus.cc:2804] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:27.450889 11307 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:27.451069 11330 raft_consensus.cc:697] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:27.407568 11290 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.451918 11288 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:27.452447 11288 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.452139 11330 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.452891 11288 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:27.453353 11290 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:27.454018 11290 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:27.454638 11290 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.458016  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:27.460407  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.427561 11296 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:27.496323  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.496910  4597 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:27.384711 11215 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.442133 11190 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:27.424551 11194 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:27.499047  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.498728  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:27.465462 11313 raft_consensus.cc:697] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:27.538364  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.542766  4720 leader_election.cc:400] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
I20260709 12:27:27.543184  4720 leader_election.cc:403] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:27.546633  4602 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:27.554042  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.556576 11292 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:27.556999  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.557358 10875 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:27.557173 11292 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.559469 11292 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.562669 11289 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.563279 11042 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:27.565034 11289 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:27.571326  4756 raft_consensus.cc:2368] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 21. Current term is 22.
I20260709 12:27:27.547788 11114 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:27.608789  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.611178 11290 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.612784 11215 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.613698 11291 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:27.615576  4749 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:27:27.613726 11333 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 15. Current term is 16.
I20260709 12:27:27.618279 11321 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:27.622241 11318 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.625634  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.626665 11313 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.541877 11287 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:27.644008  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.645064  4678 raft_consensus.cc:1240] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:27.647320  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:27.649240  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.650699  4748 raft_consensus.cc:1240] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:27.651818  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:27.652897  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:27.662988  4672 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:27.665498  5346 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:27.637503  4596 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:27.662567  4751 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.673837  4675 raft_consensus.cc:2393] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 21: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:27.681324 11157 consensus_queue.cc:1048] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.184s
I20260709 12:27:27.681692  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:27.683578  4596 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:27.676451  4571 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:27.676015  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.688752  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.691017  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.691049 11115 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:27.692368 11115 raft_consensus.cc:697] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:27.692576 11194 consensus_queue.cc:1059] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:27.693614 11194 raft_consensus.cc:3055] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:27.693316 11115 consensus_queue.cc:237] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.694037 11194 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:27.695744 11321 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.694979 11194 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.696215 11321 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.696604  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.697052 11194 raft_consensus.cc:3060] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:27.691799  4754 raft_consensus.cc:2393] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 19: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:27.700034  4571 leader_election.cc:400] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 21. Current term is 22.
I20260709 12:27:27.700482  4571 leader_election.cc:403] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:27.698369  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.703253  4678 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:27.705133  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.706077  4755 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:27.708236 11321 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:27.672264 11169 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:27.710244 11169 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.712503 11169 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:27.715376 11288 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.716641  4600 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:27.675101 11291 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.719637 11291 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.719695  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:27.719848  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:27.721624  4600 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:27:27.724828  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.724318  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.727900  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:27.728536  4676 raft_consensus.cc:2376] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16. Re-sending same reply.
I20260709 12:27:27.736016  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.731935  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:27.753862  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.755535  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.756994  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.759802  4680 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:27.764233  4645 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:27.768167  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:27.768959  4678 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:27.767515  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.774302  4596 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 22.
I20260709 12:27:27.775692  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.781219  4751 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:27.781761  4645 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:27.765184 11215 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:27.689889 11287 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.764569 11285 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:27.781703 11288 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.805171  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.779019 11169 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.777148 11315 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 21. Current term is 22.
I20260709 12:27:27.814191  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.818500  4600 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 21.
I20260709 12:27:27.819587  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:27.822033  4600 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:27:27.822448  4672 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 13.
I20260709 12:27:27.809689 11320 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:27.804075 11285 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.785785 11215 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:27.839159  4720 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:27.803682 11335 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:27.845144 11285 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:27.834954 11215 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:27.850018 11285 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:27.850631 11285 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.815768 11169 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.850657  4749 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:27.853156 11321 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:27.842350 11320 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.825299 11313 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:27.886835 11320 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.875813  4647 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:27.876649 11285 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:27.876888  4678 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 22.
I20260709 12:27:27.889241  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:27.891430  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.891450  4672 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:27.883978  5346 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 21.
I20260709 12:27:27.893261  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:27.880103 11215 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.855072 11307 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:27.894019  4719 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:27.898669 11285 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:27.900147 11321 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:27.900252  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:27.900583 11285 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:27.900955  4596 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:27.899829 11295 raft_consensus.cc:3055] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:27.900763 11321 raft_consensus.cc:697] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 21 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:27.901824 11285 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:27.902587 11312 log.cc:927] Time spent T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.664s	user 0.002s	sys 0.000s
I20260709 12:27:27.902969 11295 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:27.904910 11339 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:27.911445  4755 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:27:27.902160 11321 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.909528  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:27.932658  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:27.907549  4720 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.910993 11077 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:27.901402  4596 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
W20260709 12:27:27.918733  4570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.919308  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 96006.6us
I20260709 12:27:27.934566  4596 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.935453 11119 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:27.936015 11119 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.938628 11119 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:27.935482  4596 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:27.945842  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.904026 11295 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.948865 11295 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:27.951130  4672 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:27.952795  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:27.950344  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.953828 11077 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:27.954521 11077 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:27.955412 11077 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.912559 11287 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.971220  4596 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 16.
I20260709 12:27:27.970605  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:27.972388  4678 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:27.981727 11284 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:27.982498 11287 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:27.982378 11284 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.983167 11287 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:27.984748 11284 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:27.985518 11287 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:27.987020 11052 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:27.987972  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.988238 11052 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:27.989259  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:27.971814 10916 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.967253 11215 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:27.931222 11307 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:28.010989  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:28.012173  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:28.025274  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:28.026144  4747 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:28.031699  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:28.042896  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:28.044754  4605 raft_consensus.cc:1240] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:27:28.047228  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:28.048316  4720 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:28.049619 11325 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:28.049757 11260 raft_consensus.cc:2804] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:28.050063 11325 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:28.050448 11260 raft_consensus.cc:697] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:28.051503 11260 consensus_queue.cc:237] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:28.052362 11333 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:27:28.055583 11305 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.826s	user 0.002s	sys 0.000s
I20260709 12:27:28.057019 11286 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:28.050525 11325 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:28.096128  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:28.097527  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:28.099053  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:28.102216  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:28.106159  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:28.107344  5346 raft_consensus.cc:2468] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:28.118849 11286 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:28.119908 11321 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:28.136475  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:28.133857 11078 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:28.232395  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.155711  4755 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:28.140894  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:28.150628 11331 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:28.249892 11078 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:28.333529 11351 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
W20260709 12:27:28.349826  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 208746us
I20260709 12:27:28.351585  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:28.362826  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:28.369879  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:28.352268 11078 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:28.348935 11119 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:28.352504 11285 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:28.364229 10875 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:28.393975  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 2386) took 1221 ms. Trace:
I20260709 12:27:28.381307 11333 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:28.386392 11287 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:27:28.421228  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.430207 11352 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:27:28.400029 11119 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:28.436970 11119 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:28.442684  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:28.443106  4747 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:28.443717 11345 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:28.444046  4756 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3798) took 1022 ms. Trace:
I20260709 12:27:28.444536  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:28.391670 11217 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:28.452312  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:28.366089 11337 consensus_queue.cc:1059] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 12:27:28.394699 11211 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:28.455067 11211 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.455484 11211 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:28.453462  4751 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:28.459470 11052 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:28.412510  4673 rpcz_store.cc:276] 0709 12:27:27.162723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.162896 (+   173us) service_pool.cc:224] Handling call
0709 12:27:27.174756 (+ 11860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:27.195017 (+ 20261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:27.195067 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:27.201356 (+  6289us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:27.201406 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.380920 (+1179514us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b5400200b54. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:28.380958 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:28.383616 (+  2658us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33428736}
I20260709 12:27:28.458297 11298 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:28.462847 11298 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:28.444437  4756 rpcz_store.cc:276] 0709 12:27:27.426104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.695627 (+269523us) service_pool.cc:224] Handling call
0709 12:27:27.695949 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:27.717254 (+ 21305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:27.717286 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:27.717327 (+    41us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:27.717372 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.443820 (+726448us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b54000555d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:28.443867 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:28.443992 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48522624}
I20260709 12:27:28.468148  4645 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:28.480643  4677 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:27:28.477676  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 132342us
I20260709 12:27:28.476197  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:28.483742  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:28.483949  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:28.465430 11298 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:28.496851  4747 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:28.417393 11260 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.509872  4751 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:28.467465 11287 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:28.481817 11338 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:28.477523 10990 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:28.522395  4569 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:28.526621  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:28.541524  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:28.444387 11345 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:28.544776  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 143658us
I20260709 12:27:28.546160  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:28.548316 10990 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:28.551092 10990 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:28.551709 10990 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:28.552130 10990 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:28.552560 10990 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:28.556106 11345 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:28.556820 11345 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:28.538800 10916 consensus_queue.cc:1059] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:27:28.558600 11169 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:28.559087 10916 raft_consensus.cc:3055] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:28.559482 10916 raft_consensus.cc:740] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:28.560472 10916 consensus_queue.cc:260] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:28.566839 10916 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:28.571986  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 50187.6us
I20260709 12:27:28.526196 11338 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:28.538091 11119 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:28.528757 11260 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:28.582763  4647 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:28.583376 11260 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:28.584545 11293 raft_consensus.cc:3055] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:28.585037 11293 raft_consensus.cc:740] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:28.585853 11293 consensus_queue.cc:260] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:28.587175 11293 raft_consensus.cc:3060] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:28.557384 11345 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:28.584458  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:28.626163 10918 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:28.627763 11287 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:28.634861 11211 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:28.637116 11325 consensus_meta.cc:287] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.541s	user 0.007s	sys 0.000s
I20260709 12:27:28.645306  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:28.646044  4673 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 19.
I20260709 12:27:28.647104  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:28.648025  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:28.649256  4674 raft_consensus.cc:1275] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:27:28.649158  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:28.649899  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:28.650120 11159 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:28.651544  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:28.578418 11338 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:28.656315  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 77255.7us
I20260709 12:27:28.592337  4596 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:28.667200  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:28.669469  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:28.670120  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:28.671636  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:28.670128  5344 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:28.675822  5312 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:28.683672  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:28.692278  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:28.694705  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:28.674569 11360 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:28.656814 11325 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:28.669339 10918 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:27:28.669850 11303 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:27:28.671259 11296 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:27:28.666674 11159 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:28.687654 11287 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:28.704957  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 184149us
I20260709 12:27:28.707152  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:28.707720  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:28.719646  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 184149us
I20260709 12:27:28.708336  5312 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:28.691368 11169 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.698863 11119 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:27:28.715477  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 57181.3us
I20260709 12:27:28.717154  4756 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:27:28.727975  4756 pending_rounds.cc:85] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 9
I20260709 12:27:28.733381  4568 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:28.688673 11298 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:27:28.684310 11313 consensus_queue.cc:1059] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:27:28.728266  4756 pending_rounds.cc:107] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:27:28.749851  4756 raft_consensus.cc:2889] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:28.754356  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:28.762887 11211 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:28.764752  4601 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:28.764860  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:28.765535  4673 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:28.766089  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:28.766566  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:28.767128  4679 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:28.767704  4604 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:28.769044  4605 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:28.774755  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:28.775331  5348 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:27:28.784099  5348 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:28.785961  5346 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:28.785432  4719 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:28.745559 11215 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:28.773321 11325 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:28.814469  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 109294us
W20260709 12:27:28.801853 11356 log.cc:927] Time spent T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 12:27:28.837637  5312 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:27:28.838583  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 49377us
W20260709 12:27:28.840744  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 103375us
I20260709 12:27:28.745746 11169 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:28.801151 11335 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:28.818166 11218 raft_consensus.cc:2764] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:27:28.876765  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:28.797780 10990 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:28.892544  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:27:28.907248  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 82515us
I20260709 12:27:28.916469  4569 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:28.916989  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:28.918591  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 137192us
I20260709 12:27:28.844944 11333 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:28.923461  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:28.925248  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 143319us
W20260709 12:27:28.927323  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 50638.7us
I20260709 12:27:28.928505  5312 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:28.824340 11371 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 12:27:28.893787 11322 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:28.908115 11119 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:28.943890  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:28.854158 11260 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.082s
I20260709 12:27:28.936921 11346 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:28.937244 11345 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:28.970872  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 54528.5us
I20260709 12:27:28.980353  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:27:28.969482  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 77751.6us
I20260709 12:27:28.889011 11335 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:28.982415  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:28.981510  4679 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:28.948946 11332 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:28.941723 10918 raft_consensus.cc:493] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:28.942981 11371 consensus_queue.cc:1243] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:29.008098 11335 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:29.009037  4673 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:28.944554 11376 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.009369 11368 consensus_queue.cc:1048] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 12:27:29.009863 11376 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:29.011240 11365 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 12:27:29.011639  4681 raft_consensus.cc:1275] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:28.949035 11210 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:28.965627 11333 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:29.001883  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
W20260709 12:27:29.006084 11078 consensus_meta.cc:287] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.586s	user 0.010s	sys 0.000s
I20260709 12:27:28.990638 11127 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:28.935699 11185 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:27:28.936493 11262 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 12:27:28.983327 11217 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:29.032120  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:28.996711 11345 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:29.034060  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:29.041085  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:29.050956  4748 pending_rounds.cc:85] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 8
I20260709 12:27:29.051312  4748 pending_rounds.cc:107] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:27:29.051692  4748 raft_consensus.cc:2889] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:29.051259  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:29.075309  4602 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:29.018594 11376 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:29.079324  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:29.079669  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:29.080816  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 102558us
I20260709 12:27:29.087201  4644 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:29.020305 11325 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:27:29.091845  4569 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:29.093187  5346 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:27:29.093842  5346 raft_consensus.cc:606] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:29.094184  5346 raft_consensus.cc:614] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:29.020686 11078 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.048430 11159 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:29.026273 10918 raft_consensus.cc:515] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:29.099826  4645 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:29.101189  4569 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:29.093590 11052 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:29.123301  4753 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 20: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:29.099285 11376 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:29.097705 11210 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:27:29.151363 11297 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.133s
I20260709 12:27:29.131076 11169 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:29.185985  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:29.144513 11293 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:29.201064  4678 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:29.201243  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:29.145540 11295 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:27:29.204485 11303 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.243850 11303 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.246135 11303 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:29.205662 11379 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 12:27:29.213336 11377 log.cc:927] Time spent T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:27:29.173753 11392 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:29.223246 11052 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:29.225857 11313 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:29.228147  4600 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:29.228524  4604 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:29.175010 11333 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:29.107443 11119 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:29.232020 11384 log.cc:927] Time spent T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:27:29.275018 11313 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.282208  4749 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:29.282742  4678 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:29.283233 11052 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:29.233744 11159 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:27:29.235044 11378 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.230s	user 0.003s	sys 0.000s
I20260709 12:27:29.243023 11078 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:29.246573 11375 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.256s	user 0.002s	sys 0.000s
I20260709 12:27:29.254459  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:29.291661  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:29.298496  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 55693.7us
I20260709 12:27:29.308905  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:29.310554  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:29.321640 10918 leader_election.cc:290] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:29.326254  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:29.285558 11052 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:29.291896 11307 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:29.342427  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:29.343838  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:29.313665 11077 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:29.309262 11293 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:27:29.383541  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:29.314010 11159 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.391753  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:29.395457  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:29.427186 11386 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.098s
I20260709 12:27:29.427996 11395 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.106s
I20260709 12:27:29.428628 11393 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 12:27:29.352386 11078 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:29.429876  4603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1133) took 1856 ms. Trace:
I20260709 12:27:29.351985 11077 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:29.430938  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:29.346431 11307 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.356163 11052 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:27:29.440306 11338 consensus_meta.cc:287] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.003s
I20260709 12:27:29.381704 11285 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:29.487021  4645 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:29.348855 11313 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:29.459854  4751 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:29.465390  4753 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43864 (request call id 3809) took 1019 ms. Trace:
I20260709 12:27:29.469079  4672 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:29.468655  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:29.493096  4753 rpcz_store.cc:276] 0709 12:27:28.440805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.442417 (+  1612us) service_pool.cc:224] Handling call
0709 12:27:29.398527 (+956110us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b54000e60d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4ba2fa 00007efff347bb17 00007effec9f66d4 00007effec9f6662 00007effec9f6591 00007effec9f650c 00007effec9f57c2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:29.460294 (+ 61767us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1555401600}
I20260709 12:27:29.468253  4603 rpcz_store.cc:276] 0709 12:27:27.573663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.760833 (+187170us) service_pool.cc:224] Handling call
0709 12:27:27.761128 (+   295us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:27.778991 (+ 17863us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:27.779024 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:27.801489 (+ 22465us) log.cc:844] Serialized 52 byte log entry
0709 12:27:28.718365 (+916876us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b18004a70a0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:27:28.821480 (+103115us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:28.821523 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.821606 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:29.245807 (+424201us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:27:29.429614 (+183807us) raft_consensus.cc:1685] finished
0709 12:27:29.429641 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:29.429791 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6518,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":1781,"spinlock_wait_cycles":86929664,"thread_start_us":102890,"threads_started":1,"wal-append.queue_time_us":1104069}
I20260709 12:27:29.508422  4571 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:29.512828  4672 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:29.519140  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:29.488291 11077 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:29.504029 11303 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:29.482924 11078 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.546582  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:29.517956 11174 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.498678 11338 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.520562 11298 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:29.485865 11052 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.501307 11285 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.560902 11174 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:29.564921 11307 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:29.525514 11391 consensus_queue.cc:1059] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.238s
I20260709 12:27:29.568315 11385 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:29.547017 11345 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:29.550928 11363 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.571904  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:29.593587  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:29.605598  4674 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:29.569382 11192 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:29.568902 11301 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:29.616197  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 320774us
W20260709 12:27:29.616787  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 129830us
W20260709 12:27:29.618206  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 46383.6us
I20260709 12:27:29.622852 11290 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:29.611123 11385 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.663425 11285 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:29.621090 11363 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:29.638633 11192 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.639693 11159 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:29.668113 11174 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:29.669497 11078 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:29.670284 11338 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:29.672348 11397 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.320s	user 0.002s	sys 0.004s
W20260709 12:27:29.675089 11400 log.cc:927] Time spent T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:27:29.726897 11217 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:29.747895  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 171818us
I20260709 12:27:29.755518  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:29.756734  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 138862us
I20260709 12:27:29.706233 11052 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:29.758782  4644 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:29.759616  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 130418us
I20260709 12:27:29.760071  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:29.777087  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:29.778060  4603 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 21.
W20260709 12:27:29.758008 11412 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:27:29.737742 11192 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.793604  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:29.745914 11217 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:29.816845  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 198228us
W20260709 12:27:29.821628  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 62947.3us
I20260709 12:27:29.809986 11159 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.835700  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:29.837358  4569 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:29.849184 11052 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.849956 11052 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.854183  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:29.857832  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:29.858690  4597 raft_consensus.cc:2393] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 17: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:29.863036  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:29.867404  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:29.868187 11392 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:29.829627 11407 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:29.875234  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:29.860253 11385 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:29.861933 11363 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:29.885525 11363 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:29.886125 11363 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.899261 11192 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:29.901129 11217 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:29.902805  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:29.901017  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:29.905337 11335 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:29.905980 11335 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.908351 11335 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:29.911019  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:29.909373 11363 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:29.930305  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:29.934926 10990 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:29.939786 11374 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.914s	user 0.005s	sys 0.005s
I20260709 12:27:29.941476 11360 raft_consensus.cc:3055] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:29.942070 11360 raft_consensus.cc:740] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:29.943189 11360 consensus_queue.cc:260] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:29.948670  5348 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:29.950865 11390 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.804s	user 0.004s	sys 0.009s
W20260709 12:27:29.955523  5313 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.962574 11363 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:29.972359 11210 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:29.979027 11315 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:29.841989 11159 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:29.986991 11372 log.cc:927] Time spent T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 1.019s	user 0.004s	sys 0.002s
I20260709 12:27:29.992552  4681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51858 (request call id 2412) took 1229 ms. Trace:
I20260709 12:27:30.000741  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:30.010464  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.011270  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:30.012421  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.026029  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.027271  4754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.047020  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:27:30.049196  5314 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.066474  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.074726  4681 rpcz_store.cc:276] 0709 12:27:28.762901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.769859 (+  6958us) service_pool.cc:224] Handling call
0709 12:27:28.770159 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:29.012034 (+241875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.992269 (+980235us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b540019f354. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:29.992480 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554519808}
I20260709 12:27:30.099120  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.100714  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.102596  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:30.103598  4603 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 20: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:30.104614  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.017647 11052 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:30.108170  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 61233.6us
I20260709 12:27:30.110347  4597 raft_consensus.cc:1240] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:30.110625  4604 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:30.111337  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:30.073068 11363 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.030146 11398 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:30.114524  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cbe5bef0f514b01ac552ff54d4d13be" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.130740  4603 raft_consensus.cc:1240] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:30.115700 11390 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:30.116817  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:30.132571  4674 raft_consensus.cc:2410] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:27:30.117437 11416 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.118140  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:30.118316 11370 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:30.119650 11078 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:30.125625  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:27:30.068861 11077 consensus_meta.cc:287] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: Time spent flushing consensus metadata: real 0.519s	user 0.010s	sys 0.002s
I20260709 12:27:30.050007 11285 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:30.141222  4644 leader_election.cc:304] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:30.141794  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.142619 11363 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:30.145726 11398 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:30.156692 11373 log.cc:927] Time spent T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 1.217s	user 0.006s	sys 0.004s
I20260709 12:27:30.166858  4747 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3816) took 1668 ms. Trace:
W20260709 12:27:30.169499  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 44013.3us
I20260709 12:27:30.172381  4676 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 12:27:30.206360  4676 pending_rounds.cc:85] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 9
I20260709 12:27:30.206667  4676 pending_rounds.cc:107] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 12:27:30.206952  4676 raft_consensus.cc:2889] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:30.087106 11360 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:30.181465 11395 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:30.136562 11315 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.188002 11159 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:30.171310 11416 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.138902 11210 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:30.229023 11210 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:30.196482 11077 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.231487 11077 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:30.197288 11285 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:30.234714 11285 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:30.235509  4749 raft_consensus.cc:1240] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:30.235592 11416 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:30.205008  4747 rpcz_store.cc:276] 0709 12:27:28.498379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.525023 (+ 26644us) service_pool.cc:224] Handling call
0709 12:27:28.525388 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:28.525737 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:28.525765 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:28.547256 (+ 21491us) log.cc:844] Serialized 26 byte log entry
0709 12:27:28.748277 (+201021us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:28.748315 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.748384 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:29.956215 (+1207831us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:27:30.166467 (+210252us) raft_consensus.cc:1685] finished
0709 12:27:30.166498 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.166762 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":426,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":6196,"spinlock_wait_cycles":253087104,"thread_start_us":124444,"threads_started":1,"wal-append.queue_time_us":298980}
I20260709 12:27:30.220677  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:30.221271  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 80130.8us
I20260709 12:27:30.203492  4645 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:30.240811  4680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48198 (request call id 1089) took 1592 ms. Trace:
I20260709 12:27:30.250818  4672 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 21.
W20260709 12:27:30.265075  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 61705.9us
I20260709 12:27:30.269044  4645 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:30.275038  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.275826  4603 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:30.241328  4680 rpcz_store.cc:276] 0709 12:27:28.645583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.649670 (+  4087us) service_pool.cc:224] Handling call
0709 12:27:28.649996 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:28.650313 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:28.650341 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:28.650801 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 12:27:28.782206 (+131405us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:28.782243 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.782304 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:29.997941 (+1215637us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
0709 12:27:30.237250 (+239309us) raft_consensus.cc:1685] finished
0709 12:27:30.237272 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.237485 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":509,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"spinlock_wait_cycles":269878400,"thread_start_us":130942,"threads_started":1,"wal-append.queue_time_us":203644}
I20260709 12:27:30.220439 11338 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.220012 11395 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:30.294358  4645 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:30.224455 11413 raft_consensus.cc:2749] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:30.260670 11333 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.991s	user 0.005s	sys 0.037s
I20260709 12:27:30.321264  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.285674 11159 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:27:30.289497 11404 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:27:30.289187 11386 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:30.219333 11078 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:27:30.185418 11370 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:30.310500 11395 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:30.327060  4645 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:30.340013  4645 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:27:30.311764  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 252972us
I20260709 12:27:30.285836 11416 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:30.318802  4603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
W20260709 12:27:30.320243 11333 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.051s	user 0.006s	sys 0.037s
I20260709 12:27:30.325855 11407 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:30.322140  4568 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:30.345558  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 123114us
I20260709 12:27:30.310261 11338 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.342247 11388 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:30.371120  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 306912us
W20260709 12:27:30.377122 11366 log.cc:927] Time spent T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 1.467s	user 0.004s	sys 0.000s
I20260709 12:27:30.385264  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:30.387413  4751 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43876 (request call id 3140) took 1847 ms. Trace:
I20260709 12:27:30.387843  4751 rpcz_store.cc:276] 0709 12:27:28.540352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.559083 (+ 18731us) service_pool.cc:224] Handling call
0709 12:27:29.459675 (+900592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:29.492036 (+ 32361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.387239 (+895203us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b54000e60d0. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b58ad 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6 00007effec7f026f
0709 12:27:30.387356 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2070530304}
I20260709 12:27:30.358090 11416 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:30.401906  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:30.390180 11302 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:30.360631 11333 consensus_queue.cc:1059] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:27:30.394642 11388 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.421152  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 67455.4us
I20260709 12:27:30.425758  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
W20260709 12:27:30.444931  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.446455  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.423924 11289 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:30.425786 11078 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:30.429606 11398 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:30.451184 11407 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:30.444175 10990 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:30.450768 11315 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:30.580070 11432 log.cc:927] Time spent T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 12:27:30.554010 11407 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.539604 11289 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:30.636819 11407 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:30.638417 11289 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:30.638970 11289 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.639424 11289 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:30.615439 11439 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:27:30.641760 11440 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:27:30.641475 11289 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:30.544646 11078 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:30.648047 11078 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:30.648895 11435 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:27:30.650938 11416 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:30.651921 11441 log.cc:927] Time spent T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:27:30.584977 10990 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:30.628165 11429 log.cc:927] Time spent T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
W20260709 12:27:30.632261 11427 log.cc:927] Time spent T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
W20260709 12:27:30.633822 11437 log.cc:927] Time spent T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 12:27:30.427682 11388 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.621968 11338 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:30.684520  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 233981us
I20260709 12:27:30.582345 11443 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:30.689415  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:30.694152  4569 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:30.696328 11443 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:30.700598 10990 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:30.702474 11431 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:30.705143 11388 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:30.693982 11217 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:30.711558 11443 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:30.714111  4569 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:30.715334 11215 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:30.718075 11285 raft_consensus.cc:3055] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:30.718660 11285 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:30.720127 11285 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.721666 11285 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:30.725521 11388 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:30.726284 11388 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.728788 11388 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:30.742234 11360 consensus_meta.cc:287] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.521s	user 0.007s	sys 0.005s
I20260709 12:27:30.743163 11302 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
W20260709 12:27:30.751881  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 621240us
I20260709 12:27:30.778414  4755 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.754678 11325 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:30.796885 11418 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:30.785408  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:30.801865 11418 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.802853  4749 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43864 (request call id 3837) took 1133 ms. Trace:
I20260709 12:27:30.803232  4749 rpcz_store.cc:276] 0709 12:27:29.669740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.669900 (+   160us) service_pool.cc:224] Handling call
0709 12:27:29.670208 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:30.236105 (+565897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.802615 (+566510us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b54000726d4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:30.802792 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1299495168}
I20260709 12:27:30.792027 11345 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:30.806825 11345 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:30.807348 11418 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:30.798969 11389 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 1.652s	user 0.000s	sys 0.007s
I20260709 12:27:30.810046  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.810242 11345 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:30.811062 11325 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.812407 11297 consensus_queue.cc:1059] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:30.813275 11297 raft_consensus.cc:3055] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:30.813642 11297 raft_consensus.cc:740] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:30.814105 11325 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:30.814460 11297 consensus_queue.cc:260] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.814990 11302 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:30.815711 11297 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:30.755256  5314 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:30.757486  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.759681  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.760509  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.761365  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.764062 11444 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:30.766997  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.827548 11444 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.768289  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.770848  4753 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:30.772665  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.773487  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.830180  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:30.774560  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.800523  4673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51856 (request call id 3790) took 1620 ms. Trace:
I20260709 12:27:30.831357 11215 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:30.831595 11444 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:30.821720 11337 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:30.831323  4673 rpcz_store.cc:276] 0709 12:27:29.180192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.180343 (+   151us) service_pool.cc:224] Handling call
0709 12:27:29.190661 (+ 10318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:29.224088 (+ 33427us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:29.353592 (+129504us) log.cc:844] Serialized 14 byte log entry
0709 12:27:29.765395 (+411803us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:29.765585 (+   190us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:29.765649 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.800215 (+1034566us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b540009a454. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:30.800281 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.800462 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":284941,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":5555,"raft.queue_time_us":61305,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":9102,"spinlock_wait_cycles":76307712,"thread_start_us":540123,"threads_started":3,"wal-append.queue_time_us":446916}
I20260709 12:27:30.834842 11337 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:30.787568 11295 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:30.837759 11295 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:30.838806 11215 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:30.838696 11295 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.840565 11295 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:30.842844  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.844697 11337 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:30.845451 11302 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:30.789691  4602 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:30.848176  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:30.851330  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 96150.9us
I20260709 12:27:30.853821 11127 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:30.859566 11446 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.860213 11446 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.875344  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.877805  4750 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:30.878405 11370 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:30.878553  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.880008  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.880774  4753 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 16.
I20260709 12:27:30.884802  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.886408  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.887074  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.887828  4600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 3165) took 1043 ms. Trace:
I20260709 12:27:30.888230  4600 rpcz_store.cc:276] 0709 12:27:29.844855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.845399 (+   544us) service_pool.cc:224] Handling call
0709 12:27:30.886938 (+1041539us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b5800002798. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:27:30.887771 (+   833us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4200832}
I20260709 12:27:30.891328  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:30.896930  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.897742  4600 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 19.
I20260709 12:27:30.899298 11077 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:30.899957 11077 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:30.901788 11210 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:30.902331 11433 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:30.902379 11210 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:30.903024 11433 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:30.903715  4569 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:30.904703 11119 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:30.906111  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.906837 11433 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:30.907440 11077 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:30.905428 11210 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:30.908267  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.909550  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.912067  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.912155  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.912786  4748 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:30.914126  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:30.914660  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.915908  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:30.916093 11174 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:30.917598  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.917683  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:30.918820 11174 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:30.919281 11174 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:30.919926 11174 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:30.924381 11381 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:30.919243 11210 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:30.927649  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:30.753645 11395 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:30.932348  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 484596us
I20260709 12:27:30.936336  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.942772  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:30.935645  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:30.950826  4748 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:30.943852  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.943303  4569 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:27:30.958738  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.959964  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.963686  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.963212  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.966547  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.965512  4645 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:30.964578  4600 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
I20260709 12:27:30.975188  4496 catalog_manager.cc:5697] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 13 to 18, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 18 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:30.958123 11433 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:30.977731 11337 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:30.977826  4496 catalog_manager.cc:5697] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 6 to 14, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 14 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:30.978356 11337 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:30.980434 11337 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:30.982833  4496 catalog_manager.cc:5697] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 11 to 14, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 14 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:30.984930 11447 consensus_queue.cc:1048] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:27:30.985668 11447 consensus_queue.cc:1243] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer 3a5aeedd11fe49938079fe86cb01232c log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:30.985980  4496 catalog_manager.cc:5697] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:30.988765  4647 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:30.990983  4756 raft_consensus.cc:1240] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:31.005971  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:30.968590 11295 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.993783  4496 catalog_manager.cc:5697] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:30.993705 11302 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:31.015807  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:30.990455 11363 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:31.016153  4645 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:31.019176  4496 catalog_manager.cc:5697] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:30.972831  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:27:31.022616  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49861.3us
I20260709 12:27:31.024534  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:31.025120  4678 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
W20260709 12:27:31.017074  4495 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 167) took 14260 ms (client timeout 15000 ms). Trace:
I20260709 12:27:31.022290  4645 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:31.040773  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:31.059605  4496 catalog_manager.cc:5697] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 12 to 15, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 15 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:31.028105  4603 raft_consensus.cc:606] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:31.061880  4603 raft_consensus.cc:614] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:31.058795 11391 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:31.064244 11391 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:31.064687 11386 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:31.064869 11391 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.058305 11417 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:31.070894  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:31.073508  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:31.044113  4596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:31.087741  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 65513.2us
I20260709 12:27:31.028275  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:31.092443  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:31.095201  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:31.051285  4569 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:31.116858  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:31.118502  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:31.015029 11337 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:31.118538  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:31.021325 11303 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:31.121155  4496 catalog_manager.cc:5697] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:31.117631  4605 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:31.121927  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:31.122764  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:31.127177  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:31.065234 11386 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.021440 11052 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:31.058741 11320 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:31.177664  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 52625.1us
W20260709 12:27:31.176932  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 114277us
I20260709 12:27:31.180109  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:31.181968  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:31.189103  4647 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:27:31.173612  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 122432us
W20260709 12:27:31.183249  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 329883us
I20260709 12:27:31.189606  4678 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:31.193157  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:31.018965 11295 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:31.015282 11119 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.155s
I20260709 12:27:31.121734 11415 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:31.066287 11451 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:31.203498  4756 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:31.125955 11077 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:31.215031  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:31.200803  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:31.216745  4681 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:31.216857  4647 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:31.066483 11217 consensus_queue.cc:1059] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.142s
I20260709 12:27:31.222674  4647 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:31.223418  4496 catalog_manager.cc:5697] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:31.224581  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:31.226573  4679 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:31.214519 11451 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:31.213927 11415 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.228442  4496 catalog_manager.cc:5697] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:31.228999  4647 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:31.237500  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:31.244629  4647 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:31.219395 11307 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:31.122560 11417 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.251134  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:31.231436 11296 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:31.257272  4597 raft_consensus.cc:2468] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:31.234642 11411 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:31.025592  4495 rpcz_store.cc:269] 0709 12:27:16.750946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.776909 (+ 25963us) service_pool.cc:224] Handling call
0709 12:27:31.007766 (+14230857us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:26.438749 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:26.439973 (+  1224us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:26.440056 (+    83us) write_op.cc:435] Acquired schema lock
0709 12:27:26.440129 (+    73us) tablet.cc:662] Decoding operations
0709 12:27:26.442561 (+  2432us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:26.442692 (+   131us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:26.442736 (+    44us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:27:26.445495 (+  2759us) tablet.cc:701] Row locks acquired
0709 12:27:26.445554 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:27:26.445827 (+   273us) write_op.cc:270] Start()
0709 12:27:26.446063 (+   236us) write_op.cc:276] Timestamp: P: 1783600046445743 usec, L: 0
0709 12:27:26.446121 (+    58us) op_driver.cc:348] REPLICATION: starting
0709 12:27:26.447061 (+   940us) log.cc:844] Serialized 7498 byte log entry
0709 12:27:26.576791 (+129730us) op_driver.cc:464] REPLICATION: finished
0709 12:27:26.837961 (+261170us) write_op.cc:301] APPLY: starting
0709 12:27:26.838082 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:27.565627 (+727545us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:27.565756 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:28.556530 (+990774us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b180027d720. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effee1f50ec 00007effee241c0a 00007effee2414f7 00007effee241490 00007effee1f8f48 00007efff1f0d1bb 00007efff1f545d4 00007efff1e008da 00007efff1e033cb
0709 12:27:28.777255 (+220725us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:28.824308 (+ 47053us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 12:27:28.856424 (+ 32116us) write_op.cc:312] APPLY: finished
0709 12:27:29.036262 (+179838us) log.cc:844] Serialized 172 byte log entry
0709 12:27:29.147287 (+111025us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:29.147464 (+   177us) write_op.cc:454] Released schema lock
0709 12:27:29.147537 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:29.150223 (+  2686us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:29.150266 (+    43us) write_op.cc:454] Released schema lock
0709 12:27:29.671735 (+521469us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effee05186e 00007effee050bb6 00007effee0507d8 00007effee05a221 00007effee05a1d9 00007effee05a169
Metrics: {"mutex_wait_us":159661,"reported_tablets":26,"spinlock_wait_cycles":8064,"child_traces":[["op",{"apply.queue_time_us":260859,"apply.run_cpu_time_us":52006,"apply.run_wall_time_us":2312246,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83433,"num_ops":19,"prepare.queue_time_us":26020,"prepare.run_cpu_time_us":28858,"prepare.run_wall_time_us":55397,"raft.queue_time_us":24983,"raft.run_cpu_time_us":16848,"raft.run_wall_time_us":169119,"replication_time_us":130448,"spinlock_wait_cycles":13917312,"thread_start_us":308382,"threads_started":5,"wal-append.queue_time_us":357185,"wal-append.run_cpu_time_us":16448,"wal-append.run_wall_time_us":2599890}]]}
I20260709 12:27:31.242067 11415 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:31.234189 11451 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:31.252110 11407 consensus_meta.cc:287] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: Time spent flushing consensus metadata: real 0.612s	user 0.007s	sys 0.004s
I20260709 12:27:31.269714  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:31.287755  4756 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 16.
I20260709 12:27:31.271162  4598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 3162) took 1439 ms. Trace:
I20260709 12:27:31.283073  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:31.272845  4604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 2447) took 1216 ms. Trace:
I20260709 12:27:31.273849 11438 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:31.256680 11450 raft_consensus.cc:2749] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:31.277045 11295 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:31.280181  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.244412 11337 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:31.290544  4604 rpcz_store.cc:276] 0709 12:27:30.056169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.125755 (+ 69586us) service_pool.cc:224] Handling call
0709 12:27:30.141102 (+ 15347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:30.172346 (+ 31244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.172393 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:30.891137 (+718744us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b18003e6020. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3972a6 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a
0709 12:27:30.891680 (+   543us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b2c000014c8. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:27:30.892361 (+   681us) log.cc:844] Serialized 26 byte log entry
0709 12:27:31.020645 (+128284us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:31.020679 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.020767 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:31.272635 (+251868us) raft_consensus.cc:1685] finished
0709 12:27:31.272656 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.272785 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30812,"raft.queue_time_us":31333,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":3415,"spinlock_wait_cycles":11369472,"thread_start_us":127819,"threads_started":1,"wal-append.queue_time_us":377719}
I20260709 12:27:31.258420 11388 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:31.302099  4673 pending_rounds.cc:85] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Aborting all ops after (but not including) 12
I20260709 12:27:31.288831  4598 rpcz_store.cc:276] 0709 12:27:29.831848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.832012 (+   164us) service_pool.cc:224] Handling call
0709 12:27:29.841300 (+  9288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:30.172901 (+331601us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:30.172955 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:30.173528 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 12:27:30.706696 (+533168us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b1800430900. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3abe07 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007effee05020b 00007effee055d23 00007effee0560d1 00007effee471a0f
0709 12:27:31.027922 (+321226us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:31.027979 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.116904 (+ 88925us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54002e4ad4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:27:31.116966 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:31.270906 (+153940us) raft_consensus.cc:1685] finished
0709 12:27:31.270928 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.271098 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23675,"raft.queue_time_us":28120,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":720286336,"thread_start_us":321015,"threads_started":1,"wal-append.queue_time_us":1031835}
I20260709 12:27:31.308189  4673 pending_rounds.cc:107] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:27:31.310213  4720 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:31.311466  4720 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:31.295939 11407 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.312237 11333 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:31.312384  4720 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:31.312875 11333 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:31.280784 11446 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:31.308812  4673 raft_consensus.cc:2889] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:31.302829  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:31.316686  4677 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:27:31.318845  4720 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:31.322820  4496 catalog_manager.cc:5697] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 10 to 14, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 14 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:31.256577 11448 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.136s
I20260709 12:27:31.313478 11333 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:31.329483 11298 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:31.332924 11386 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:31.306815 11392 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:31.335569 11174 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:31.341187 11391 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:31.331923  4720 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:31.349484  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 147076us
I20260709 12:27:31.344518 11298 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.326009 11395 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:31.355913  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:31.358008  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:31.345976 11285 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:31.360839  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:31.345712 11370 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.363258 11370 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.365450 11370 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:31.366940  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:31.367779  4756 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 20.
I20260709 12:27:31.365365  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:31.372526  4496 catalog_manager.cc:5697] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 8 to 9, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 9 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:31.369939  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:31.362025 11078 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:31.377825  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:31.351426 11407 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:31.358435 11298 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:31.377422 11395 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:31.361622 11333 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:31.352695 11392 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:31.417366  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 179986us
W20260709 12:27:31.388450 11453 log.cc:927] Time spent T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:27:31.420063  4645 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:31.394829 11078 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:31.426936  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:31.397790 11210 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:31.434334  4605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:31.438716  4605 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:31.434672  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:31.435041  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:31.436331  4496 catalog_manager.cc:5697] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:31.443907  4496 catalog_manager.cc:5697] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:31.445268  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:31.447930  4645 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:31.455428  5345 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53808 (request call id 788) took 1338 ms. Trace:
I20260709 12:27:31.447584 11285 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:31.455964  5345 rpcz_store.cc:276] 0709 12:27:30.116517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.116769 (+   252us) service_pool.cc:224] Handling call
0709 12:27:30.117095 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:30.751828 (+634733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.751862 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:30.752400 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:27:31.022803 (+270403us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:31.022843 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.024052 (+  1209us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:31.455184 (+431132us) raft_consensus.cc:1685] finished
0709 12:27:31.455207 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.455337 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":638827,"raft.queue_time_us":635746,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":269991,"threads_started":1,"wal-append.queue_time_us":476461}
I20260709 12:27:31.461805  4496 catalog_manager.cc:5697] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 15 to 19, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 19 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:31.463541  4496 catalog_manager.cc:5697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:31.464607 11470 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.465602 11470 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.472738  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:31.427737 11433 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:31.474390 11433 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:31.476845 11433 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:31.437083 11210 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.479252 11210 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:31.482416 11447 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:31.483165 11447 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:31.456280 11285 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:31.486321 11285 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.425989 11418 raft_consensus.cc:2749] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:31.496317  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48994.4us
I20260709 12:27:31.431602 11078 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:31.494133 11417 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:31.502550 11407 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:31.522974  4605 raft_consensus.cc:2468] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 17.
W20260709 12:27:31.548705  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 112880us
I20260709 12:27:31.534961 11454 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:31.549377 11447 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:31.555488 11174 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:31.571180 11392 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:31.534142 11451 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.584031 11333 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:31.584604 11333 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.586901 11298 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.587261 11333 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:31.587944 11333 raft_consensus.cc:493] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:31.588522 11333 raft_consensus.cc:515] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:31.588872 11210 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.555776 11290 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:27:31.591763  4751 raft_consensus.cc:1240] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:31.581866 11454 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:31.596441 11454 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:31.620115 11415 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.622967 11447 consensus_queue.cc:1059] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:27:31.623296 11415 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:31.624112 11448 raft_consensus.cc:3055] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:31.624725 11448 raft_consensus.cc:740] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:31.624739  4672 raft_consensus.cc:2368] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
I20260709 12:27:31.625655 11448 consensus_queue.cc:260] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.634596 11451 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:31.643802 11320 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:31.644439 11320 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:31.620059 11285 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.648947 11285 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:31.650754 11078 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.651417 11210 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:31.652558 11470 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:31.653689 11078 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:31.669344  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:31.670049  5344 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Rejecting Update request from peer 3a5aeedd11fe49938079fe86cb01232c for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:31.679383  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:31.682627  4678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:31.682967  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:31.683332  4678 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 20.
I20260709 12:27:31.693874  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:31.704371  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:31.709452  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:31.717592  4603 raft_consensus.cc:1240] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:27:31.721885  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:31.728188  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 152008us
I20260709 12:27:31.736153  4720 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:31.722481 11431 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:31.753073  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:31.770835 11333 leader_election.cc:290] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:31.750072  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:31.751775 11078 raft_consensus.cc:2804] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:31.771343 11303 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:27:31.749294  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 225502us
I20260709 12:27:31.779372  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:31.776650 11333 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:31.801476  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:31.812664  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:31.817361  4720 leader_election.cc:400] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599) with higher term. Message: Invalid argument: T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
I20260709 12:27:31.751605 11431 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.825652  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:31.823863  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:31.830538  4747 raft_consensus.cc:3060] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:31.833604  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:31.834272  4751 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 20.
I20260709 12:27:31.783368 11078 raft_consensus.cc:697] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:31.835794 11078 consensus_queue.cc:237] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:31.836275 11487 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:27:31.817611 11303 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.843415 11431 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:31.843802 11303 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:31.783512 11320 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:31.847724 11489 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:27:31.848577 11320 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:31.848749  4644 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:31.849363 11470 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:31.850684  5349 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:31.852689  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:31.829149 11325 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:27:31.853751 11360 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:31.864795 11360 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:31.851794 11470 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:31.865752 11360 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:31.867066 11360 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.874245  4647 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:31.884682  4647 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:31.888780 11337 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:31.895272  4747 raft_consensus.cc:2468] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:31.889334 11337 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.902243 11417 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:31.903856 11417 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.894426 11391 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:31.911808  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:31.906533  4646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.908200  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:31.912923  4752 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:31.909914  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:31.896431 11407 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:31.915215  4647 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:31.899802 11337 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:31.930651 11303 consensus_queue.cc:1059] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:31.932024 11303 raft_consensus.cc:3055] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:31.932423 11303 raft_consensus.cc:740] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:31.933102 11192 raft_consensus.cc:3055] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:31.922472 11470 raft_consensus.cc:2804] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:31.933633 11192 raft_consensus.cc:740] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:31.934114 11470 raft_consensus.cc:697] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:31.934520 11192 consensus_queue.cc:260] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.935079 11470 consensus_queue.cc:237] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.936025 11192 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:31.941361  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:31.950555  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:31.957887  4645 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:31.962517  4752 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 21.
I20260709 12:27:31.967260  4571 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:31.933225 11303 consensus_queue.cc:260] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:31.972234 11368 raft_consensus.cc:2955] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index 4 to 12, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:31.973186  4753 raft_consensus.cc:2955] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 4 to 12, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:31.979120  4597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:31.981719  4752 raft_consensus.cc:1240] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:31.979024  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:31.984483 10990 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:31.987295  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:31.988117  4602 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:31.959210 11407 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:31.966068 11392 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.987169 10990 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 21 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:31.995731 11215 consensus_queue.cc:1059] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:31.959611 11169 consensus_queue.cc:1059] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:27:31.941987 11391 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:32.011101  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:32.011641 11169 raft_consensus.cc:3055] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:32.011968  4598 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 18: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:32.012055 11169 raft_consensus.cc:740] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:32.012609 11391 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:32.013079 11169 consensus_queue.cc:260] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:32.014245  4680 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:32.015473 11337 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.015933  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:31.969170 11446 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:32.005364  4602 raft_consensus.cc:2468] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 22.
I20260709 12:27:32.006511 11215 raft_consensus.cc:3055] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:32.005510 10990 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.009976 11363 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:32.035180  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:32.035660 11363 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:32.036204 11363 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:32.036731 11448 raft_consensus.cc:3060] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:32.041996 11446 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.044802 11446 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:32.020650 11444 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.054138 11444 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.056670 11444 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:31.968195 11483 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:32.058259 11483 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.059415 11298 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:32.060802 11483 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:32.020074  4599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53448 (request call id 1165) took 1077 ms. Trace:
I20260709 12:27:31.988379 11418 raft_consensus.cc:2804] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:32.030834  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:27:32.071866  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.031783  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 50366.2us
I20260709 12:27:32.029172 11215 raft_consensus.cc:740] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:32.077968  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:32.078856  4756 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 21: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:32.078486 11215 consensus_queue.cc:260] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.079969 11215 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:32.033232  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 75433.9us
I20260709 12:27:32.014385 11169 raft_consensus.cc:3060] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:32.082921  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:32.084043  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 53311.8us
I20260709 12:27:32.085040  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:32.085806 11157 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:32.086895 11157 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:32.087102 11418 raft_consensus.cc:697] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:32.088025 11418 consensus_queue.cc:237] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.009413 11392 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.091049  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:32.091637  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:32.093779  4604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:32.094336  4604 raft_consensus.cc:606] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
W20260709 12:27:32.095753  4568 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.099148  4568 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:32.104478 11217 raft_consensus.cc:2749] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:32.117264 11417 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:32.119292  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:32.120872  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:32.121829  4752 raft_consensus.cc:2393] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 18: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:32.121968  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:32.122609  4645 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:32.065927  4599 rpcz_store.cc:276] 0709 12:27:30.943120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.944314 (+  1194us) service_pool.cc:224] Handling call
0709 12:27:30.944941 (+   627us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:30.945423 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:30.968021 (+ 22598us) log.cc:844] Serialized 14 byte log entry
0709 12:27:31.253083 (+285062us) log.cc:844] Serialized 14 byte log entry
0709 12:27:31.265636 (+ 12553us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:31.673020 (+407384us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b180020ed80. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3c93f3 00007effeb3cb7c1 00007effeb3cb779 00007effeb3cb709 00007effeb3cb6d1 00007effeb3ca9cd 0000000000388634 0000000000388479 00007effeb3ac83a
0709 12:27:32.006390 (+333370us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b180020ed80. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb3972a6 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a
0709 12:27:32.007031 (+   641us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b2c000014c8. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effecdafa11 00007effecdbf372 00007effee4f1a01 00007effee4b643a 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2
0709 12:27:32.007822 (+   791us) log.cc:844] Serialized 494 byte log entry
0709 12:27:32.007994 (+   172us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:32.008031 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.008102 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:32.019854 (+ 11752us) raft_consensus.cc:1685] finished
0709 12:27:32.019876 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.020013 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17815,"prepare.queue_time_us":22516,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":91,"raft.queue_time_us":34540,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":21729,"spinlock_wait_cycles":40722176,"thread_start_us":263791,"threads_started":1,"wal-append.queue_time_us":341256}
I20260709 12:27:32.129738  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:32.132462  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 64378.4us
I20260709 12:27:32.135560 11451 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.076494  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:32.138543  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:32.139307  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:32.137409  4645 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:32.120754  5312 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6
I20260709 12:27:32.074952 11337 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:32.149854  4672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:32.150205  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:32.150882  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:32.151269  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:32.151340  4647 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:32.152315  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdfc68cfdd1411cad1b4e974e65067a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:32.154349 11451 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.155012  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:32.171428  4676 raft_consensus.cc:2393] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 21: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:27:32.182243  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 54683.1us
I20260709 12:27:32.178547 11338 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:32.183219  4568 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:32.142198 11417 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:32.155736 11484 raft_consensus.cc:2804] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:32.199872  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:32.204715  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:32.179977 11498 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:32.208081 11451 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:32.208590 11303 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:32.201189 11469 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:32.210996 11484 raft_consensus.cc:697] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:32.212989  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:32.215093  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:32.216684  4604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:32.217167  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:32.220948  4603 raft_consensus.cc:1240] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:32.223999  4599 raft_consensus.cc:2368] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
I20260709 12:27:32.226742  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:32.227734  4680 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:32.236186  4603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:32.237283  4603 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:32.245980  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:32.246654  4753 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:32.248032  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:32.248852  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:32.249748  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:32.250362  4681 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 21.
I20260709 12:27:32.250840  4571 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:32.259665  4720 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:32.260602  4571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.261778  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:32.262589  4569 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:32.262420  4681 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:32.269579  4720 leader_election.cc:304] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:32.278913  4569 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:32.281822 11370 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:32.244983 11392 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:32.282578 11370 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:32.282677  4569 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:32.283457 11395 raft_consensus.cc:2749] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:32.283524 11370 raft_consensus.cc:2749] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:32.243199 11363 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:32.287685  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:32.267961 11469 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:32.286125 11438 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:32.268429 11368 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:41342 (request call id 11) took 2302 ms. Trace:
I20260709 12:27:32.304312  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:32.295603  4569 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:32.313984  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:32.300305 11469 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.316525 11469 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:32.324333 11363 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:32.325325  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:32.307452 11260 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:32.329389 11469 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.332433 11469 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:32.334254  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:32.307962 11368 rpcz_store.cc:276] 0709 12:27:29.946629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.946797 (+   168us) service_pool.cc:224] Handling call
0709 12:27:30.065423 (+118626us) log.cc:844] Serialized 494 byte log entry
0709 12:27:30.747999 (+682576us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b18003e3320. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff4291ab1 00007efff429e281 00007effeb343c0a 00007efff4a83219 00007effeb3cba40 00007effeb3c64e4 00007effeb3c6455 00007effee40fb0b 00007effee494b63 00007effee4bc752 00007effee4b49fa
0709 12:27:32.246398 (+1498399us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":596108,"raft.queue_time_us":686809,"raft.run_cpu_time_us":1533,"raft.run_wall_time_us":77541,"spinlock_wait_cycles":33964416}
I20260709 12:27:32.330091 11260 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:32.339589  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 44126us
I20260709 12:27:32.344976  4569 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:32.345773  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:32.346807  4674 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:32.340256 11444 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:32.349493 11260 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:32.382189 11469 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:32.363770  4485 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e8a74137ba104fe0b465cab6399c4940 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
W20260709 12:27:32.491288  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 141093us
I20260709 12:27:32.510097  4603 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Rejecting Update request from peer dbe960c35c84451faf4e5b5c9233fa39 for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:32.511442  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 135494us
W20260709 12:27:32.427659 11503 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
W20260709 12:27:32.500157  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 130426us
I20260709 12:27:32.505788  4601 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:32.367906 11444 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.369349 11303 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:32.296591 11484 consensus_queue.cc:237] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:32.494987  4602 raft_consensus.cc:2468] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 22.
I20260709 12:27:32.508639  4674 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 18.
I20260709 12:27:32.512265  4603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:32.516353  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:27:32.507993 11489 log.cc:927] Time spent T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.144s	user 0.000s	sys 0.005s
I20260709 12:27:32.522928  4603 raft_consensus.cc:606] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:32.526430  4603 raft_consensus.cc:614] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:32.526824  4720 leader_election.cc:400] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
I20260709 12:27:32.527146  4720 leader_election.cc:403] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:27:32.527684 11386 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:27:32.521646  4485 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 175658us
I20260709 12:27:32.521039 11360 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:32.539588 11360 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.544142 11360 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:32.545658  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:32.550671  4601 raft_consensus.cc:2393] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 20: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:32.546536 11215 raft_consensus.cc:993] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:32.551506 11215 raft_consensus.cc:1081] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:32.552073  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:32.535051 11481 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:32.558784 11446 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:32.559377 11446 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.521595 11444 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:32.562047 11446 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:32.565131  4569 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:32.566931  4751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:27:32.567559  4751 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:32.567919  4751 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:32.568601  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:32.569742 11388 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:32.570358 11388 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.573892 11446 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:32.574810 11446 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:32.575397  4751 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.584329  4752 raft_consensus.cc:1275] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:27:32.586905  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 60213us
I20260709 12:27:32.575800 11446 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.578202  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:32.593756  4751 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:32.594004  4602 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 21.
I20260709 12:27:32.591049 11476 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077) with higher term. Message: Invalid argument: T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 for earlier term 17. Current term is 18.
I20260709 12:27:32.593168  4681 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:32.594473  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:32.596961  4568 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:32.598968  4645 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:32.602628  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:32.603439  4750 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 21: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:32.604236  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:32.604874  4677 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 21.
I20260709 12:27:32.599036  4604 raft_consensus.cc:2468] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:32.577672 11388 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:32.611444  4602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:32.615882  4602 raft_consensus.cc:606] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:32.619623  4647 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:27:32.619863  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 94959.7us
I20260709 12:27:32.620656 11307 raft_consensus.cc:3060] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:32.626049 11502 log.cc:927] Time spent T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: Append to log took a long time: real 0.267s	user 0.000s	sys 0.003s
I20260709 12:27:32.629676 11217 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:32.610386 11337 consensus_queue.cc:1048] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:27:32.640158  4720 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:32.644480  4603 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:32.660226  4601 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:32.611590 11469 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:32.666107  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:32.632542 11313 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:32.672303  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:32.670486  4598 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:32.673089  4677 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:32.678719  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:32.684782  4598 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:32.694423  4720 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:32.620119 11346 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:27:32.663345 11169 raft_consensus.cc:2804] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:32.672941 11313 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:32.725698 11169 raft_consensus.cc:697] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:32.715099  4598 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.677323 11388 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:27:32.765364  4598 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:32.691512 11469 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:32.740767 11307 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:32.691370 11438 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:27:32.744174 11325 raft_consensus.cc:2804] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:32.746620 11313 raft_consensus.cc:3060] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:32.772653 11169 consensus_queue.cc:237] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.705389 11260 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.789033  4681 raft_consensus.cc:1275] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:32.768381 11469 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:32.793114  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:32.794982  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:32.796264  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:32.804526  4748 raft_consensus.cc:2393] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 18: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:32.809095 11260 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:32.792774 11325 raft_consensus.cc:697] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:32.814497 11325 consensus_queue.cc:237] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.825551  4677 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
W20260709 12:27:32.826210  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 120021us
I20260709 12:27:32.796064 11469 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:32.878823  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 184533us
I20260709 12:27:32.929503 11444 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:32.994418  4753 raft_consensus.cc:2368] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 21. Current term is 22.
I20260709 12:27:32.996048 11444 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:32.859786  4749 raft_consensus.cc:1275] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:27:32.997927  4749 pending_rounds.cc:85] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 10
I20260709 12:27:32.999064  4749 pending_rounds.cc:107] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:27:33.000007  4749 raft_consensus.cc:2889] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:32.844091  4756 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:32.968927  4598 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:33.007083 11393 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:33.007496 11388 consensus_queue.cc:1048] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:33.015543  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:33.018149  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:33.018985  4681 raft_consensus.cc:2393] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 18: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:33.034641  4571 leader_election.cc:400] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 21. Current term is 22.
I20260709 12:27:32.993306 11313 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:33.036139 11469 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:33.043160 11469 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:33.044114  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:33.063891  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:33.071707 11169 consensus_queue.cc:1048] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:33.072932  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:33.074447  4600 raft_consensus.cc:2468] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 21.
I20260709 12:27:33.076140  4681 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 21.
I20260709 12:27:33.082113  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:33.083032  4645 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:33.085973  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:33.074013 11325 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:33.062143 11407 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:33.079715 11192 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:33.104939 11127 raft_consensus.cc:3055] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:33.087352 11370 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:33.087571 11388 raft_consensus.cc:2804] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:33.129745  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:33.130537  4751 raft_consensus.cc:2368] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 21. Current term is 22.
I20260709 12:27:33.136701  4571 leader_election.cc:400] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 election: Vote denied by peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) with higher term. Message: Invalid argument: T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 for earlier term 21. Current term is 22.
I20260709 12:27:33.106141 11407 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:33.105450 11127 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:33.143611 11127 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:33.145113 11127 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:33.145630 11192 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:33.148278 11192 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:33.160009  4751 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:33.163753  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:33.164836  4750 raft_consensus.cc:2393] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 21: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:33.165468 11370 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 21 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:33.166474 11370 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:33.167611 11192 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:33.173641 11313 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:33.193913  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:33.196476  5345 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 20: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:33.197090  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:33.197788  4673 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 20.
I20260709 12:27:33.189436 11192 raft_consensus.cc:993] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:33.209378  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:33.217224  4598 raft_consensus.cc:1813] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Handling vote request from an unknown peer 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:33.217835  4598 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 19: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:33.218479  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:33.214921  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:33.227391  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:33.228120  4681 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:33.196110 11388 raft_consensus.cc:697] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 21 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:33.197265 11325 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:27:33.242051  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 155811us
I20260709 12:27:33.237556 11388 consensus_queue.cc:237] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:33.228163 11298 raft_consensus.cc:1081] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:33.276832  4681 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 22.
I20260709 12:27:33.242568 11192 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:33.290807 11192 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:33.291155  4599 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:33.291275 11192 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:33.292754 11517 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:33.280639 11325 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:33.285868  4681 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:33.322415 11407 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:33.357708  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:33.358601  5345 raft_consensus.cc:1813] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Handling vote request from an unknown peer 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:33.359346  5345 raft_consensus.cc:2410] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:27:33.361799 11371 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:33.367614  4644 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:33.371892 11470 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:33.373054 11192 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:33.388317  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:33.389173 11260 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:33.389288  5345 raft_consensus.cc:2393] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c3201048f8346009aefca07428828f3 in current term 20: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:33.399282  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:33.407563  4495 catalog_manager.cc:5697] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:33.410295  4495 catalog_manager.cc:5697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 15 to 17, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 17 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:33.413129  4495 catalog_manager.cc:5697] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 12 to 18, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 18 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:33.397001 11192 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:33.433769  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 175805us
W20260709 12:27:33.432657  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 173738us
I20260709 12:27:33.437288  4681 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 20.
I20260709 12:27:33.450847  4569 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:33.454378 11393 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:33.455009 11393 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 20 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:33.456343 11393 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:33.462143 11444 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.462786 11444 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:33.464329  4753 raft_consensus.cc:1275] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:33.466738 11444 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:33.469677  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:33.469848 11290 consensus_queue.cc:1048] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:33.471323 11169 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:33.471902 11169 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:33.472886 11169 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:33.484469  4681 raft_consensus.cc:3060] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:33.489151  4599 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:33.491829  4681 raft_consensus.cc:1275] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:33.492134  4599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
W20260709 12:27:33.498898 11527 log.cc:927] Time spent T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:27:33.511441  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 57848.5us
I20260709 12:27:33.545723  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:33.567409 11260 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:33.581723  4496 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52812 (request call id 293) took 32737 ms (client timeout 15000 ms). Trace:
I20260709 12:27:33.590852  4495 catalog_manager.cc:5697] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 15 to 17, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 17 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:33.593466  4495 catalog_manager.cc:5697] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:33.572804 11470 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:33.585825  4496 rpcz_store.cc:269] 0709 12:27:00.844747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.844916 (+   169us) service_pool.cc:224] Handling call
0709 12:27:33.581618 (+32736702us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:31.560039 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:31.560646 (+   607us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:31.560682 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:27:31.560703 (+    21us) tablet.cc:662] Decoding operations
0709 12:27:31.562224 (+  1521us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:31.562316 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:31.562348 (+    32us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:27:31.563680 (+  1332us) tablet.cc:701] Row locks acquired
0709 12:27:31.563702 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:27:31.563864 (+   162us) write_op.cc:270] Start()
0709 12:27:31.564015 (+   151us) write_op.cc:276] Timestamp: P: 1783600051563807 usec, L: 0
0709 12:27:31.564042 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:27:31.564862 (+   820us) log.cc:844] Serialized 6309 byte log entry
0709 12:27:32.045806 (+480944us) op_driver.cc:464] REPLICATION: finished
0709 12:27:32.271266 (+225460us) write_op.cc:301] APPLY: starting
0709 12:27:32.271352 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:32.437739 (+166387us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:32.511189 (+ 73450us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:32.858398 (+347209us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:32.871199 (+ 12801us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 12:27:32.871261 (+    62us) write_op.cc:312] APPLY: finished
0709 12:27:32.871754 (+   493us) log.cc:844] Serialized 148 byte log entry
0709 12:27:32.871943 (+   189us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:32.872127 (+   184us) write_op.cc:454] Released schema lock
0709 12:27:32.872188 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:32.872648 (+   460us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:32.872686 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1088715,"reported_tablets":51,"spinlock_wait_cycles":339328,"child_traces":[["op",{"apply.queue_time_us":225168,"apply.run_cpu_time_us":29059,"apply.run_wall_time_us":601344,"cfile_cache_hit":66,"cfile_cache_hit_bytes":70911,"num_ops":16,"prepare.queue_time_us":77279,"prepare.run_cpu_time_us":17395,"prepare.run_wall_time_us":168093,"raft.queue_time_us":175585,"raft.run_cpu_time_us":25410,"raft.run_wall_time_us":150193,"replication_time_us":481584,"thread_start_us":412181,"threads_started":4,"wal-append.queue_time_us":202007}]]}
I20260709 12:27:33.595330  4495 catalog_manager.cc:5697] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 9 to 10, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 10 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:33.622666  4598 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:33.624583 11532 consensus_queue.cc:1048] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:33.632308 11169 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:33.638537  4753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:27:33.639243  4753 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:33.639670  4753 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:33.650949  4753 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:33.613683  4495 catalog_manager.cc:5697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 11 to 18, now has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 18 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:33.654783  4495 catalog_manager.cc:5697] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 18 to 20, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 20 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:33.655930  4495 catalog_manager.cc:5697] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 19 to 25, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 25 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:33.657037  4495 catalog_manager.cc:5697] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:33.658252  4495 catalog_manager.cc:5697] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:33.669839  4756 raft_consensus.cc:1240] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:33.671155  4753 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:33.672813  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:33.673429  4600 raft_consensus.cc:3055] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:33.673509  4677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:33.673866  4600 raft_consensus.cc:740] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:33.674220  4677 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:33.674688  4600 consensus_queue.cc:260] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:33.675865  4600 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:33.684434  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:33.687258  4677 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:27:33.691325  4600 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:27:33.702278  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:33.703526 11169 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:33.704051 11169 raft_consensus.cc:697] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:33.704924 11169 consensus_queue.cc:237] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:33.713882  4677 raft_consensus.cc:3060] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:33.721928  4719 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:33.723738 11260 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:33.728286  4677 raft_consensus.cc:1275] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:33.728966 11418 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.729599 11418 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:33.738178  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:33.738798 11418 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:33.740952 11532 consensus_queue.cc:1048] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:33.741551  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:33.742789  4647 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:33.748441 11418 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:33.753875 11368 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.754863 11368 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:33.757169 11368 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:33.758709  4676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:33.759472  4676 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 20.
I20260709 12:27:33.760855  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:33.762665  4753 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:33.763099  4676 raft_consensus.cc:1240] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:33.764539 11392 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:33.764997 11368 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:33.765328 11392 consensus_queue.cc:1243] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer 49277a7653524787a715232ab908bbc6 log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:33.765535 11368 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.765964 11392 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:33.766422 11392 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:33.766175 11368 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:33.767254 11469 consensus_queue.cc:1059] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:33.769001  4600 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:33.771517  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:33.774086 11392 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:33.776273 11392 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:33.779311 11337 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:33.779989 11368 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:33.832180  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:33.832772 11368 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:33.833220  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:33.833935  4681 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:33.834864  4599 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:33.836953 11169 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:33.841147  4681 raft_consensus.cc:2468] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 21.
I20260709 12:27:33.842407  5312 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:33.843575 11368 raft_consensus.cc:2804] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:33.844282 11368 raft_consensus.cc:697] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:33.845155 11368 consensus_queue.cc:237] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:33.848767  4753 raft_consensus.cc:1275] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:33.855484 11407 consensus_queue.cc:1048] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:33.903760  4681 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:33.921010  5349 raft_consensus.cc:1240] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 LEADER]: Rejecting Update request from peer 5c3201048f8346009aefca07428828f3 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:33.929400 11169 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:33.937450 11469 consensus_queue.cc:1059] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:33.938570 11469 consensus_queue.cc:1243] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [LEADER]: Peer dbe960c35c84451faf4e5b5c9233fa39 log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:27:33.939174 11469 raft_consensus.cc:3055] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:33.939563 11469 raft_consensus.cc:740] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:33.940672 11469 consensus_queue.cc:260] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:33.942739 11469 raft_consensus.cc:3060] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:34.020947  4681 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:34.026640 11169 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:34.127864  4500 catalog_manager.cc:5697] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 15 to 17, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 17 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:34.131192  4500 catalog_manager.cc:5697] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.132994  4500 catalog_manager.cc:5697] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 11 to 14, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 14 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.135746  4500 catalog_manager.cc:5697] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.148339  4495 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 168) took 2045 ms. Trace:
I20260709 12:27:34.162985  4495 rpcz_store.cc:276] 0709 12:27:32.103310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.103484 (+   174us) service_pool.cc:224] Handling call
0709 12:27:32.211095 (+107611us) spinlock_profiling.cc:243] Waited 89.6 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee420090 00007effee4bef3a 00007efff4274e9c 00007efff437304e 00007efff28342a4 00007efff2834232 00007efff2834161 00007efff28340dc 00007efff2833392 00007effec7f148c 00007effec7f08c6
0709 12:27:34.148237 (+1937142us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:33.733953 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:33.734504 (+   551us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:33.734544 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:27:33.734563 (+    19us) tablet.cc:662] Decoding operations
0709 12:27:33.735590 (+  1027us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:33.735669 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:33.735696 (+    27us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:27:33.737028 (+  1332us) tablet.cc:701] Row locks acquired
0709 12:27:33.737047 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:27:33.737186 (+   139us) write_op.cc:270] Start()
0709 12:27:33.737305 (+   119us) write_op.cc:276] Timestamp: P: 1783600053737139 usec, L: 0
0709 12:27:33.737329 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:27:33.746979 (+  9650us) log.cc:844] Serialized 4577 byte log entry
0709 12:27:33.752355 (+  5376us) op_driver.cc:464] REPLICATION: finished
0709 12:27:33.855395 (+103040us) write_op.cc:301] APPLY: starting
0709 12:27:33.855480 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:33.890438 (+ 34958us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:33.890559 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:33.905877 (+ 15318us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:33.906189 (+   312us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 12:27:33.906248 (+    59us) write_op.cc:312] APPLY: finished
0709 12:27:33.906584 (+   336us) log.cc:844] Serialized 105 byte log entry
0709 12:27:33.906759 (+   175us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:33.907233 (+   474us) write_op.cc:454] Released schema lock
0709 12:27:33.907287 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:33.907734 (+   447us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:33.907766 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":213238,"reported_tablets":19,"spinlock_wait_cycles":206096640,"child_traces":[["op",{"apply.queue_time_us":102803,"apply.run_cpu_time_us":15027,"apply.run_wall_time_us":52302,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50041,"num_ops":11,"prepare.queue_time_us":63887,"prepare.run_cpu_time_us":4138,"prepare.run_wall_time_us":16422,"replication_time_us":14904,"thread_start_us":86343,"threads_started":2}]]}
W20260709 12:27:34.156312  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 130645us
I20260709 12:27:34.177270  4600 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:27:34.177838  4600 pending_rounds.cc:85] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 12
I20260709 12:27:34.178130  4600 pending_rounds.cc:107] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:27:34.178432  4600 raft_consensus.cc:2889] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:34.192095  4500 catalog_manager.cc:5697] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 18 to 19, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 19 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.194223  4500 catalog_manager.cc:5697] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.195664  4500 catalog_manager.cc:5697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.217912  4598 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:34.230155  4500 catalog_manager.cc:5697] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 12 to 15, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 15 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.231837  4500 catalog_manager.cc:5697] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.233307  4500 catalog_manager.cc:5697] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 10 to 17, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 17 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.235735 11078 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:34.235544  4500 catalog_manager.cc:5697] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.242570  4500 catalog_manager.cc:5697] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 13 to 14, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 14 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:34.246855  4500 catalog_manager.cc:5697] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 12 to 13, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 13 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.256999 11438 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:34.268409  4500 catalog_manager.cc:5697] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 12 to 13, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 13 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:34.283229  4500 catalog_manager.cc:5697] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 14 to 20, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 20 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.302661  4500 catalog_manager.cc:5697] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 14 to 15, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 15 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.308466  4676 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:34.310618  4500 catalog_manager.cc:5697] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 11 to 13, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 13 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.312867  4500 catalog_manager.cc:5697] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.314611  4500 catalog_manager.cc:5697] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 9 to 13, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 13 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.316395  4500 catalog_manager.cc:5697] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 14 to 15, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 15 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:34.318173  4500 catalog_manager.cc:5697] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.319442  4500 catalog_manager.cc:5697] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 9 to 10, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 10 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:34.320880  4500 catalog_manager.cc:5697] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 8 to 13, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 13 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.329996 11558 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:34.444088  4681 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:27:34.445420 11419 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:34.461521  4600 raft_consensus.cc:1275] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:27:34.461987  4600 pending_rounds.cc:85] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Aborting all ops after (but not including) 15
I20260709 12:27:34.462236  4600 pending_rounds.cc:107] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:27:34.462529  4600 raft_consensus.cc:2889] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:34.464835 11368 consensus_queue.cc:1048] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:34.567816  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:34.609946 11078 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:34.610605 11078 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:34.612798 11078 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:34.613929  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:34.614804  4598 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 19: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:34.617833  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:34.618563  5349 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:34.628091  4719 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:34.628888 11078 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:34.629289 11078 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:34.629709 11078 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:34.658222 11078 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:34.678607  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:34.678895 11078 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:34.679427  4598 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 19: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:34.679958  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:34.697943  5349 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:34.705646  4719 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:34.707860 11559 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:34.708587 11559 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 19 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:34.709496 11559 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:34.834379 11392 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.834909 11392 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:34.838022 11392 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:34.863665  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:34.864293  5349 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
W20260709 12:27:34.876268  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 155635us
I20260709 12:27:34.935258  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:34.943885  4598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:34.944502  4598 raft_consensus.cc:606] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:34.944832  4598 raft_consensus.cc:614] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:34.929356  4497 catalog_manager.cc:5697] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 11 to 14, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 14 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.946691  4568 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:34.952107 11393 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:34.952549 11393 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.952888 11393 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:34.957496  4497 catalog_manager.cc:5697] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 9 to 14, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 14 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:34.962963  4497 catalog_manager.cc:5697] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 10 to 15, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 15 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:34.965071  4497 catalog_manager.cc:5697] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 9 to 12, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 12 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
W20260709 12:27:34.966841  4500 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52818 (request call id 306) took 23362 ms (client timeout 15000 ms). Trace:
I20260709 12:27:34.968837  4598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:34.969854  4598 raft_consensus.cc:606] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:34.970518  4598 raft_consensus.cc:614] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:34.968941  4500 rpcz_store.cc:269] 0709 12:27:11.604205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.604635 (+   430us) service_pool.cc:224] Handling call
0709 12:27:34.966761 (+23362126us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:34.368922 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:34.369450 (+   528us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:34.369486 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:27:34.369503 (+    17us) tablet.cc:662] Decoding operations
0709 12:27:34.395191 (+ 25688us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:34.395275 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:34.395301 (+    26us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:27:34.399392 (+  4091us) tablet.cc:701] Row locks acquired
0709 12:27:34.399412 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:27:34.399590 (+   178us) write_op.cc:270] Start()
0709 12:27:34.399713 (+   123us) write_op.cc:276] Timestamp: P: 1783600054399541 usec, L: 0
0709 12:27:34.399736 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:27:34.400423 (+   687us) log.cc:844] Serialized 9009 byte log entry
0709 12:27:34.433402 (+ 32979us) op_driver.cc:464] REPLICATION: finished
0709 12:27:34.493400 (+ 59998us) write_op.cc:301] APPLY: starting
0709 12:27:34.493490 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:34.578637 (+ 85147us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:34.578731 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:34.722815 (+144084us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:34.847985 (+125170us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=0,mrs_lookups=0
0709 12:27:34.848031 (+    46us) write_op.cc:312] APPLY: finished
0709 12:27:34.848442 (+   411us) log.cc:844] Serialized 204 byte log entry
0709 12:27:34.849208 (+   766us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:34.849395 (+   187us) write_op.cc:454] Released schema lock
0709 12:27:34.849459 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:34.876156 (+ 26697us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:34.876186 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1023325,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":45104,"apply.run_cpu_time_us":43397,"apply.run_wall_time_us":382717,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100129,"num_ops":23,"prepare.queue_time_us":31734,"prepare.run_cpu_time_us":6092,"prepare.run_wall_time_us":39112,"replication_time_us":33488,"thread_start_us":39803,"threads_started":2}]]}
I20260709 12:27:35.007449  4599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:35.008183  4599 raft_consensus.cc:606] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
W20260709 12:27:35.018245  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 51931.7us
I20260709 12:27:35.019426  4599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:35.020092  4599 raft_consensus.cc:606] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:35.020510  4599 raft_consensus.cc:614] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:35.025027  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 54118.9us
I20260709 12:27:35.029623  5349 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b"
dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:39524
I20260709 12:27:35.030215  5349 raft_consensus.cc:606] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
W20260709 12:27:35.040549  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 64298.2us
I20260709 12:27:35.043890  4599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:35.044658  4599 raft_consensus.cc:606] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:35.045831  4599 raft_consensus.cc:614] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:35.058341  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 45011.8us
I20260709 12:27:35.058976  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:27:35.060684  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:35.061381  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:35.071084 11393 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:35.082113 11393 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:35.083081  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:35.086465  5349 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:35.093459  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:35.132696  5349 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 20.
I20260709 12:27:35.144418  4568 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:35.161121  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 108614us
I20260709 12:27:35.159090 11585 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:35.189697 11585 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 20 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:35.190646 11585 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:35.235965  4495 catalog_manager.cc:5697] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 9 to 22. New cstate: current_term: 22 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:35.237709  4495 catalog_manager.cc:5697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 4 to 12, term changed from 18 to 19, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to <none>, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted, no longer has a pending config: VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193), VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195), VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 19 committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:35.244174  4495 catalog_manager.cc:5697] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 14 to 21. New cstate: current_term: 21 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:35.278929  5349 raft_consensus.cc:1240] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:35.282866 11571 consensus_queue.cc:1059] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:35.283979 11571 raft_consensus.cc:3055] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:35.285471 11571 raft_consensus.cc:740] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:35.286576 11571 consensus_queue.cc:260] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:35.289263 11571 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:35.319283 11393 raft_consensus.cc:993] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:35.319835 11393 raft_consensus.cc:1081] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:35.324033  4748 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:27:35.324643  4748 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:35.325001  4748 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:35.346457  4495 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 169) took 1118 ms. Trace:
I20260709 12:27:35.346951  4495 rpcz_store.cc:276] 0709 12:27:34.227594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.227752 (+   158us) service_pool.cc:224] Handling call
0709 12:27:35.346393 (+1118641us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:35.254905 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:35.255417 (+   512us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:35.255449 (+    32us) write_op.cc:435] Acquired schema lock
0709 12:27:35.255465 (+    16us) tablet.cc:662] Decoding operations
0709 12:27:35.255964 (+   499us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:35.256017 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:35.256054 (+    37us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:27:35.256360 (+   306us) tablet.cc:701] Row locks acquired
0709 12:27:35.256378 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:27:35.256508 (+   130us) write_op.cc:270] Start()
0709 12:27:35.256608 (+   100us) write_op.cc:276] Timestamp: P: 1783600055256467 usec, L: 0
0709 12:27:35.256623 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 12:27:35.260380 (+  3757us) log.cc:844] Serialized 1259 byte log entry
0709 12:27:35.266943 (+  6563us) op_driver.cc:464] REPLICATION: finished
0709 12:27:35.267802 (+   859us) write_op.cc:301] APPLY: starting
0709 12:27:35.267890 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:35.269612 (+  1722us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:35.269693 (+    81us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:35.271369 (+  1676us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:35.271543 (+   174us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:27:35.271586 (+    43us) write_op.cc:312] APPLY: finished
0709 12:27:35.271793 (+   207us) log.cc:844] Serialized 41 byte log entry
0709 12:27:35.271937 (+   144us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:35.272026 (+    89us) write_op.cc:454] Released schema lock
0709 12:27:35.272067 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:35.290992 (+ 18925us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b1e28 00007efff1eec581 00007efff1eebd58 00007efff1ef09f1 00007efff1ef09a9 00007efff1ef0939 00007efff1ef0901 00007efff1eefb2d 0000000000388634 0000000000388479
0709 12:27:35.291182 (+   190us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:35.291213 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":76522,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":635,"apply.run_cpu_time_us":4565,"apply.run_wall_time_us":4564,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12522,"num_ops":3,"prepare.queue_time_us":112,"prepare.run_cpu_time_us":4495,"prepare.run_wall_time_us":36262,"replication_time_us":10206,"spinlock_wait_cycles":7100288}]]}
I20260709 12:27:35.346966  4748 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:35.354950  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:35.355727  4602 raft_consensus.cc:3055] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:35.356128  4602 raft_consensus.cc:740] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:35.357123  4602 consensus_queue.cc:260] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:35.358985  4602 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:35.368723  4497 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52818 (request call id 305) took 43977 ms (client timeout 15000 ms). Trace:
I20260709 12:27:35.370041  4748 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:35.376216  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:35.376914  5349 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:35.378913  4602 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
W20260709 12:27:35.369189  4497 rpcz_store.cc:269] 0709 12:26:51.391358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.391533 (+   175us) service_pool.cc:224] Handling call
0709 12:27:35.368629 (+43977096us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:35.068935 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:35.069457 (+   522us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:35.069490 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:27:35.069507 (+    17us) tablet.cc:662] Decoding operations
0709 12:27:35.070163 (+   656us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:35.070242 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:35.070274 (+    32us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:27:35.070689 (+   415us) tablet.cc:701] Row locks acquired
0709 12:27:35.070710 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:27:35.070856 (+   146us) write_op.cc:270] Start()
0709 12:27:35.112861 (+ 42005us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b540016de28. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff48729e1 00007efff487164c 00007efff1f01624 00007efff1eec286 00007efff1eebd58 00007efff1ef09f1 00007efff1ef09a9 00007efff1ef0939 00007efff1ef0901 00007efff1eefb2d 0000000000388634
0709 12:27:35.113133 (+   272us) write_op.cc:276] Timestamp: P: 1783600055070810 usec, L: 0
0709 12:27:35.113164 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:27:35.113932 (+   768us) log.cc:844] Serialized 1690 byte log entry
0709 12:27:35.158580 (+ 44648us) op_driver.cc:464] REPLICATION: finished
0709 12:27:35.173324 (+ 14744us) write_op.cc:301] APPLY: starting
0709 12:27:35.193603 (+ 20279us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:35.195901 (+  2298us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:35.195923 (+    22us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:35.213826 (+ 17903us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:35.214010 (+   184us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:27:35.214056 (+    46us) write_op.cc:312] APPLY: finished
0709 12:27:35.214287 (+   231us) log.cc:844] Serialized 49 byte log entry
0709 12:27:35.216391 (+  2104us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:35.216510 (+   119us) write_op.cc:454] Released schema lock
0709 12:27:35.216574 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:35.223474 (+  6900us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:35.223509 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":2374,"apply.run_cpu_time_us":6305,"apply.run_wall_time_us":47886,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16696,"mutex_wait_us":1873,"num_ops":4,"prepare.queue_time_us":76966,"prepare.run_cpu_time_us":11490,"prepare.run_wall_time_us":45323,"replication_time_us":45258,"spinlock_wait_cycles":9451264,"thread_start_us":65499,"threads_started":1}]]}
I20260709 12:27:35.384763  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:35.385386  4681 raft_consensus.cc:3060] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:35.396474  4681 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:35.397392  5349 raft_consensus.cc:2468] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:35.398649  4719 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:35.400429 11570 raft_consensus.cc:2804] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:35.400936 11570 raft_consensus.cc:697] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 19 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:35.402384  4494 catalog_manager.cc:5697] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 12 to 16, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 16 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:35.404450  4494 catalog_manager.cc:5697] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 17 to 18, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 18 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:35.406250 11570 consensus_queue.cc:237] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:35.432242  4494 catalog_manager.cc:5697] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 20 to 21, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 21 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:35.454268  4494 catalog_manager.cc:5697] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 15 to 16, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 16 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:35.456331  4494 catalog_manager.cc:5697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 14 to 19, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 19 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:35.487238 11568 raft_consensus.cc:993] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:35.487782 11568 raft_consensus.cc:1081] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:35.487100  4494 catalog_manager.cc:5697] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 13 to 14, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 14 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:35.494546  4665 tablet_service.cc:1558] Processing DeleteTablet for tablet e8a74137ba104fe0b465cab6399c4940 with delete_type TABLET_DATA_TOMBSTONED (TS 3a5aeedd11fe49938079fe86cb01232c not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:60064
I20260709 12:27:35.494282  4494 catalog_manager.cc:5697] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 15 to 16, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 16 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:35.497247 11360 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.497373  4748 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:42632
I20260709 12:27:35.497939  4748 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:35.497822 11360 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:35.498307  4748 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:35.502858  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5c3201048f8346009aefca07428828f3" is_pre_election: true
I20260709 12:27:35.505988  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:35.506651  5345 raft_consensus.cc:1813] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Handling vote request from an unknown peer 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:35.507120  5345 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 20: Already voted for candidate 5c3201048f8346009aefca07428828f3 in this term.
I20260709 12:27:35.508165  4644 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:35.508339 11360 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:35.509029 11360 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:35.518916  4748 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:35.496187  4494 catalog_manager.cc:5697] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 14 to 17, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 17 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:35.523342  4494 catalog_manager.cc:5697] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 13 to 15, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 15 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:35.524837  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:35.525465  4753 raft_consensus.cc:1813] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Handling vote request from an unknown peer 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:35.526019  4753 raft_consensus.cc:2410] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:27:35.535187  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:35.535780  5345 raft_consensus.cc:3055] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:35.536079  5345 raft_consensus.cc:740] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:35.536877  5345 consensus_queue.cc:260] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:35.538183  5345 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:35.539726  4602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:35.540346  4602 raft_consensus.cc:3060] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:35.547959  4602 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:35.549245  5345 raft_consensus.cc:2468] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:35.549289  4720 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:35.550060  4748 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:35.552897 11570 raft_consensus.cc:2804] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:35.554575 11570 raft_consensus.cc:697] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 19 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:35.555575 11570 consensus_queue.cc:237] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:35.559165 11360 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:35.559886  4494 catalog_manager.cc:5697] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 14 to 15, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 15 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:35.566572 11594 tablet_replica.cc:333] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:27:35.567770 11594 raft_consensus.cc:2243] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:35.568751 11594 raft_consensus.cc:2272] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:35.580231 11594 ts_tablet_manager.cc:1916] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:27:35.578377  4494 catalog_manager.cc:5697] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 15 to 18, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 18 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:35.596412  5345 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:35.598703 11587 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:35.623040 11594 ts_tablet_manager.cc:1929] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.11
I20260709 12:27:35.623847 11594 log.cc:1199] T e8a74137ba104fe0b465cab6399c4940 P 3a5aeedd11fe49938079fe86cb01232c: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-1-root/wals/e8a74137ba104fe0b465cab6399c4940
I20260709 12:27:35.583844  4494 catalog_manager.cc:5697] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:35.655025  4494 catalog_manager.cc:5697] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:35.661650  4486 catalog_manager.cc:5028] TS 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): tablet e8a74137ba104fe0b465cab6399c4940 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:27:35.690472 11577 raft_consensus.cc:2955] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 20 LEADER]: Committing config change with OpId 18.12: config changed from index 4 to 12, VOTER 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:35.696470  4753 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 12:27:35.703413  4753 pending_rounds.cc:85] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Aborting all ops after (but not including) 12
I20260709 12:27:35.705456  4753 pending_rounds.cc:107] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:27:35.705842  4753 raft_consensus.cc:2889] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:35.734413 11585 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:27:35.755432  4494 catalog_manager.cc:5697] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 15 to 16, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 16 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:35.784492  4494 catalog_manager.cc:5697] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 10 to 11, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 11 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:35.813195  4494 catalog_manager.cc:5697] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 16 to 17, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 17 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:35.889529  5345 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:27:35.891757  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 57569.7us
I20260709 12:27:35.921504  4601 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:35.926954 11571 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:35.937439 11571 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:36.067340  5349 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:36.069260 11571 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:36.083009  4679 raft_consensus.cc:1275] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:36.107160 11571 consensus_queue.cc:1048] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:36.158423  4600 raft_consensus.cc:1275] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:36.166728 11571 consensus_queue.cc:1048] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:36.239203  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:36.507474  4497 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60024 (request call id 170) took 1107 ms. Trace:
I20260709 12:27:36.507922  4497 rpcz_store.cc:276] 0709 12:27:35.445994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.447087 (+  1093us) service_pool.cc:224] Handling call
0709 12:27:36.507228 (+1060141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3265,"reported_tablets":1,"spinlock_wait_cycles":7424}
I20260709 12:27:36.609203  4496 catalog_manager.cc:5697] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 15 to 22, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 22 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.611063  4496 catalog_manager.cc:5697] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 17 to 18, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 18 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.612670  4496 catalog_manager.cc:5697] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:36.619700  4494 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52808 (request call id 279) took 18546 ms (client timeout 15000 ms). Trace:
I20260709 12:27:36.630205  4496 catalog_manager.cc:5697] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:36.632184  4496 catalog_manager.cc:5697] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.633466  4496 catalog_manager.cc:5697] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 13 to 20, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 20 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.645720  4496 catalog_manager.cc:5697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:36.648075  4496 catalog_manager.cc:5697] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 14 to 18, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 18 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
W20260709 12:27:36.620205  4494 rpcz_store.cc:269] 0709 12:27:18.073800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.074127 (+   327us) service_pool.cc:224] Handling call
0709 12:27:36.619603 (+18545476us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:36.079453 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:36.080161 (+   708us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:36.080210 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:27:36.080256 (+    46us) tablet.cc:662] Decoding operations
0709 12:27:36.081736 (+  1480us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:36.081856 (+   120us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:36.081890 (+    34us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:27:36.085204 (+  3314us) tablet.cc:701] Row locks acquired
0709 12:27:36.085263 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:27:36.085454 (+   191us) write_op.cc:270] Start()
0709 12:27:36.085657 (+   203us) write_op.cc:276] Timestamp: P: 1783600056085376 usec, L: 0
0709 12:27:36.085696 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:27:36.086501 (+   805us) log.cc:844] Serialized 6310 byte log entry
0709 12:27:36.142996 (+ 56495us) op_driver.cc:464] REPLICATION: finished
0709 12:27:36.215509 (+ 72513us) write_op.cc:301] APPLY: starting
0709 12:27:36.215636 (+   127us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:36.373322 (+157686us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:36.373456 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:36.485227 (+111771us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:36.485617 (+   390us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 12:27:36.485676 (+    59us) write_op.cc:312] APPLY: finished
0709 12:27:36.486087 (+   411us) log.cc:844] Serialized 148 byte log entry
0709 12:27:36.486284 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:36.486450 (+   166us) write_op.cc:454] Released schema lock
0709 12:27:36.486520 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:36.541810 (+ 55290us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:36.541841 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":354235,"reported_tablets":51,"spinlock_wait_cycles":32000,"child_traces":[["op",{"apply.queue_time_us":72198,"apply.run_cpu_time_us":23010,"apply.run_wall_time_us":326287,"cfile_cache_hit":66,"cfile_cache_hit_bytes":70911,"num_ops":16,"prepare.queue_time_us":77163,"prepare.run_cpu_time_us":5673,"prepare.run_wall_time_us":7494,"replication_time_us":56612,"thread_start_us":106795,"threads_started":2}]]}
I20260709 12:27:36.661775  4496 catalog_manager.cc:5697] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 16 to 18, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 18 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:36.664182  4496 catalog_manager.cc:5697] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 13 to 22, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 22 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.718485  4496 catalog_manager.cc:5697] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 15 to 17, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 17 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.719913  4496 catalog_manager.cc:5697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 15 to 16, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 16 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.721470  4496 catalog_manager.cc:5697] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 17 to 20, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 20 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.722935  4496 catalog_manager.cc:5697] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 17 to 21, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 21 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:36.724437  4496 catalog_manager.cc:5697] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.725560  4496 catalog_manager.cc:5697] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.726800  4496 catalog_manager.cc:5697] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 18 to 19, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 19 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:36.728266  4496 catalog_manager.cc:5697] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.729440  4496 catalog_manager.cc:5697] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 19 to 22, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 22 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.731447  4496 catalog_manager.cc:5697] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 17 to 18, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 18 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.733131  4496 catalog_manager.cc:5697] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:36.734858  4496 catalog_manager.cc:5697] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 12 to 14, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 14 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:36.737142  4496 catalog_manager.cc:5697] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 14 to 16, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 16 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.759188  4496 catalog_manager.cc:5697] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 17 to 18, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 18 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:36.969245  4601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:36.970096  4601 raft_consensus.cc:606] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:36.983945  4601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:36.984589  4601 raft_consensus.cc:606] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:36.990128  4601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:36.990732  4601 raft_consensus.cc:606] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:36.998474  4600 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53908897ce974579947af90acaf0aed7"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:36.999246  4600 raft_consensus.cc:606] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:37.002853  4600 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4"
dest_uuid: "5c3201048f8346009aefca07428828f3"
mode: GRACEFUL
new_leader_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:41556
I20260709 12:27:37.003454  4600 raft_consensus.cc:606] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Received request to transfer leadership to TS 49277a7653524787a715232ab908bbc6
I20260709 12:27:37.003769  4600 raft_consensus.cc:614] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:37.016880  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:37.036156  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:37.045504  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:37.197304 11577 raft_consensus.cc:993] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:27:37.197896 11577 raft_consensus.cc:1081] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:27:37.200341  4681 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51856
I20260709 12:27:37.200942  4681 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:37.201298  4681 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:37.247061  4497 catalog_manager.cc:5697] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 18 to 19, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 19 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:37.249521  4497 catalog_manager.cc:5697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 19 to 20, leader changed from <none> to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 20 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:37.250805  4497 catalog_manager.cc:5697] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 18 to 19, leader changed from dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 19 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:37.225610  4681 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:37.299209  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:37.299913  4600 raft_consensus.cc:3055] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:37.300208  4600 raft_consensus.cc:740] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:37.301030  4600 consensus_queue.cc:260] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:37.302320  4600 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:37.303148 11599 raft_consensus.cc:993] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: : Instructing follower 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:37.305294 11599 raft_consensus.cc:1081] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Signalling peer 49277a7653524787a715232ab908bbc6 to start an election
I20260709 12:27:37.307334  4753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53908897ce974579947af90acaf0aed7"
dest_uuid: "49277a7653524787a715232ab908bbc6"
 from {username='slave'} at 127.0.0.1:43864
I20260709 12:27:37.307860  4753 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:37.308214  4753 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:37.315124  4681 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:37.315416  4752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:37.316201  4752 raft_consensus.cc:3060] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:37.320665  4600 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:27:37.322245  4645 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:37.323237 11603 raft_consensus.cc:2804] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:37.323745 11603 raft_consensus.cc:697] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:37.324702 11603 consensus_queue.cc:237] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:37.332860 11370 raft_consensus.cc:993] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:27:37.333413 11370 raft_consensus.cc:1081] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:27:37.335865  4752 raft_consensus.cc:2468] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:27:37.337213  4676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51856
I20260709 12:27:37.337790  4753 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:37.337806  4676 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:37.339761  4676 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:37.342279 11370 raft_consensus.cc:993] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:27:37.342759 11370 raft_consensus.cc:1081] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:27:37.344853  4753 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:37.346912  4598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:37.347532  4598 raft_consensus.cc:3055] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:37.347805  4598 raft_consensus.cc:740] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:37.348562  4598 consensus_queue.cc:260] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:37.352969  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:37.353579  4679 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:37.363343  4674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51856
I20260709 12:27:37.364012  4674 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:37.364379  4674 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:37.365239  4598 raft_consensus.cc:3060] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:37.352483  4676 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:37.370401  4679 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:37.373814  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:37.371301  4676 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:37.374457  4748 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:37.378204  4720 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:37.375097  4601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:37.380505  4601 raft_consensus.cc:3055] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:37.380241  4674 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:37.380838  4601 raft_consensus.cc:740] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:37.381726  4601 consensus_queue.cc:260] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:37.382233  4674 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:37.382808  4601 raft_consensus.cc:3060] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:37.383145  4600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "5c3201048f8346009aefca07428828f3"
I20260709 12:27:37.397324  4600 raft_consensus.cc:3055] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:37.397692  4600 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:37.398463  4600 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:37.385278 11571 raft_consensus.cc:2804] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:37.399552 11571 raft_consensus.cc:697] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 17 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:37.400457 11571 consensus_queue.cc:237] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:37.407042  4600 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:37.416129  4748 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 20.
I20260709 12:27:37.429724  4598 raft_consensus.cc:2468] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
I20260709 12:27:37.430382  4601 raft_consensus.cc:2468] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 20.
I20260709 12:27:37.431267  4600 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:27:37.438774  4645 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:37.440306 11603 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:37.441063 11603 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:37.442000 11603 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:37.439723  4645 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:37.460282 11586 raft_consensus.cc:2804] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:37.460937 11586 raft_consensus.cc:697] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 20 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:37.461987 11586 consensus_queue.cc:237] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:37.473892  4496 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52808 (request call id 280) took 3700 ms. Trace:
I20260709 12:27:37.480438  4495 catalog_manager.cc:5697] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 17 to 21, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 21 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:37.474323  4496 rpcz_store.cc:276] 0709 12:27:33.773862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.774031 (+   169us) service_pool.cc:224] Handling call
0709 12:27:33.814681 (+ 40650us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b540016dfd4. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee420090 00007effee4bef3a 00007efff4274e9c 00007efff437304e 00007efff28342a4 00007efff2834232 00007efff2834161 00007efff28340dc 00007efff2833392 00007effec7f148c 00007effec7f08c6
0709 12:27:37.473812 (+3659131us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:36.783820 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:36.784403 (+   583us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:36.784434 (+    31us) write_op.cc:435] Acquired schema lock
0709 12:27:36.784451 (+    17us) tablet.cc:662] Decoding operations
0709 12:27:36.788240 (+  3789us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:36.788325 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:36.788351 (+    26us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:27:36.797222 (+  8871us) tablet.cc:701] Row locks acquired
0709 12:27:36.797241 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:27:36.797378 (+   137us) write_op.cc:270] Start()
0709 12:27:36.797545 (+   167us) write_op.cc:276] Timestamp: P: 1783600056797332 usec, L: 0
0709 12:27:36.797586 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:27:36.798301 (+   715us) log.cc:844] Serialized 9390 byte log entry
0709 12:27:36.877550 (+ 79249us) op_driver.cc:464] REPLICATION: finished
0709 12:27:36.877919 (+   369us) write_op.cc:301] APPLY: starting
0709 12:27:36.878007 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:36.975937 (+ 97930us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:36.976041 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:37.200312 (+224271us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:37.200629 (+   317us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=0,mrs_lookups=0
0709 12:27:37.200678 (+    49us) write_op.cc:312] APPLY: finished
0709 12:27:37.201179 (+   501us) log.cc:844] Serialized 212 byte log entry
0709 12:27:37.205735 (+  4556us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:37.205939 (+   204us) write_op.cc:454] Released schema lock
0709 12:27:37.206002 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:37.214268 (+  8266us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:37.214296 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":59646,"reported_tablets":51,"spinlock_wait_cycles":80007552,"child_traces":[["op",{"apply.queue_time_us":94,"apply.run_cpu_time_us":30352,"apply.run_wall_time_us":336304,"cfile_cache_hit":96,"cfile_cache_hit_bytes":100176,"num_ops":24,"prepare.queue_time_us":16434,"prepare.run_cpu_time_us":5939,"prepare.run_wall_time_us":22302,"replication_time_us":79800,"thread_start_us":19757,"threads_started":1}]]}
I20260709 12:27:37.483866  4495 catalog_manager.cc:5697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 21 to 22, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 22 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:37.486075  4495 catalog_manager.cc:5697] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 19 to 20, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 20 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
I20260709 12:27:37.502552  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:37.503185  4748 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:37.489049  4495 catalog_manager.cc:5697] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 12 to 13, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 13 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
W20260709 12:27:37.532339  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 92622.2us
W20260709 12:27:37.569708 11622 log.cc:927] Time spent T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:27:37.599588  4748 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
W20260709 12:27:37.610262  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 87591.4us
W20260709 12:27:37.619387  4488 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3be428c36f9242c6a2b8aa853f2c8b2c with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 49277a7653524787a715232ab908bbc6 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 19 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:37.627439  4495 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52812 (request call id 295) took 1035 ms. Trace:
I20260709 12:27:37.629411  4674 consensus_queue.cc:237] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:37.628192  4495 rpcz_store.cc:276] 0709 12:27:36.591925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.592114 (+   189us) service_pool.cc:224] Handling call
0709 12:27:37.627365 (+1035251us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:37.519985 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:37.520599 (+   614us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:37.520636 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:27:37.520669 (+    33us) tablet.cc:662] Decoding operations
0709 12:27:37.521335 (+   666us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:37.521396 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:37.521431 (+    35us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:27:37.521898 (+   467us) tablet.cc:701] Row locks acquired
0709 12:27:37.521916 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:27:37.522111 (+   195us) write_op.cc:270] Start()
0709 12:27:37.522233 (+   122us) write_op.cc:276] Timestamp: P: 1783600057522060 usec, L: 0
0709 12:27:37.522261 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:27:37.522960 (+   699us) log.cc:844] Serialized 1694 byte log entry
0709 12:27:37.529459 (+  6499us) op_driver.cc:464] REPLICATION: finished
0709 12:27:37.534026 (+  4567us) write_op.cc:301] APPLY: starting
0709 12:27:37.536688 (+  2662us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:37.538888 (+  2200us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:37.538979 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:37.541157 (+  2178us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:37.541321 (+   164us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:27:37.541377 (+    56us) write_op.cc:312] APPLY: finished
0709 12:27:37.551625 (+ 10248us) log.cc:844] Serialized 49 byte log entry
0709 12:27:37.560725 (+  9100us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:37.569075 (+  8350us) write_op.cc:454] Released schema lock
0709 12:27:37.576745 (+  7670us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:37.586496 (+  9751us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:37.586638 (+   142us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16507,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":1994,"apply.run_cpu_time_us":6011,"apply.run_wall_time_us":52421,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16696,"num_ops":4,"prepare.queue_time_us":126,"prepare.run_cpu_time_us":3311,"prepare.run_wall_time_us":7240,"replication_time_us":7039}]]}
W20260709 12:27:37.665880  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 113924us
I20260709 12:27:37.695005  5349 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:37.697185 11603 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:37.700510  4601 raft_consensus.cc:1275] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:37.713900 11619 consensus_queue.cc:1048] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:37.729435 11619 raft_consensus.cc:2955] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index 10 to 12, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:37.737828  4601 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:37.739588 11603 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:37.738862  5349 raft_consensus.cc:2955] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 10 to 12, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:37.756589  4497 catalog_manager.cc:5697] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 19 to 20, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 20 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:37.758545  4497 catalog_manager.cc:5697] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 18 to 19, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 19 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:37.763985  4497 catalog_manager.cc:5697] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 13 to 14, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 14 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:37.777053  4601 raft_consensus.cc:2955] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 10 to 12, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:37.831342  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1b7b0c6c811e455abe898cb4d21acad4 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:27:37.902611  4495 catalog_manager.cc:5697] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 10 to 12, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New cstate: current_term: 18 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:37.938179  4600 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:37.941244 11627 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:37.944676  4753 raft_consensus.cc:1275] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:37.948520  4501 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52818 (request call id 307) took 7539 ms. Trace:
I20260709 12:27:37.949067  4501 rpcz_store.cc:276] 0709 12:27:30.375827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.376103 (+   276us) service_pool.cc:224] Handling call
0709 12:27:37.898989 (+7522886us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8726,"reported_tablets":52}
I20260709 12:27:37.951049  4737 tablet_service.cc:1558] Processing DeleteTablet for tablet 1b7b0c6c811e455abe898cb4d21acad4 with delete_type TABLET_DATA_TOMBSTONED (TS 49277a7653524787a715232ab908bbc6 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:50178
W20260709 12:27:37.951345  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 93956.4us
I20260709 12:27:37.953475 11638 consensus_queue.cc:1048] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:27:37.972836  4601 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:37.976567 11638 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:27:38.016721  4602 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:38.019268 11639 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:38.020134  4676 raft_consensus.cc:1275] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:38.022328 11627 consensus_queue.cc:1048] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:27:38.032895  4498 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52808 (request call id 278) took 45808 ms (client timeout 15000 ms). Trace:
W20260709 12:27:38.033527  4498 rpcz_store.cc:269] 0709 12:26:52.201422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.201708 (+   286us) service_pool.cc:224] Handling call
0709 12:27:38.009343 (+45807635us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2781,"reported_tablets":51}
W20260709 12:27:38.081384  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 78991.6us
I20260709 12:27:38.290100 11645 tablet_replica.cc:333] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:27:38.461679  4499 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 1b7b0c6c811e455abe898cb4d21acad4 with cas_config_opid_index 10: aborting the task: latest config opid_index 12; task opid_index 10
W20260709 12:27:38.469406  4499 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52812 (request call id 294) took 16777 ms (client timeout 15000 ms). Trace:
W20260709 12:27:38.481835  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 128167us
I20260709 12:27:38.473493 11645 raft_consensus.cc:2243] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:38.470045  4499 rpcz_store.cc:269] 0709 12:27:21.699142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.703818 (+  4676us) service_pool.cc:224] Handling call
0709 12:27:38.469238 (+16765420us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":260977,"reported_tablets":51}
I20260709 12:27:38.499429  4496 catalog_manager.cc:5697] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 16 to 17, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 17 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
W20260709 12:27:38.557256  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 61315.9us
I20260709 12:27:38.574185  4665 tablet_service.cc:1558] Processing DeleteTablet for tablet e8a74137ba104fe0b465cab6399c4940 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 12)) from {username='slave'} at 127.0.0.1:60064
I20260709 12:27:38.631763 11645 raft_consensus.cc:2272] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:38.717994  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 141321us
I20260709 12:27:38.724170  4748 raft_consensus.cc:1275] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:27:38.722054  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 147631us
I20260709 12:27:38.784269  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:38.791630  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 61781.7us
I20260709 12:27:38.813366  4750 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:27:38.833654  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 151863us
I20260709 12:27:38.865952 11656 consensus_queue.cc:1048] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:38.875191 11681 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:27:38.877172  4488 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3be428c36f9242c6a2b8aa853f2c8b2c with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 49277a7653524787a715232ab908bbc6 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 19 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:38.886932  4486 catalog_manager.cc:5028] TS 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): tablet e8a74137ba104fe0b465cab6399c4940 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
W20260709 12:27:38.887584  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 57990.4us
I20260709 12:27:38.853163 11645 ts_tablet_manager.cc:1916] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:27:38.913635  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 55974.6us
W20260709 12:27:39.041199  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 58437.8us
W20260709 12:27:39.206421  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 283461us
W20260709 12:27:39.214697 11695 log.cc:927] Time spent T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
W20260709 12:27:39.357690  4571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.014s	sys 0.080s
W20260709 12:27:39.358083  4571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.014s	sys 0.081s
I20260709 12:27:39.358076 11370 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
W20260709 12:27:39.358558  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 633155us
I20260709 12:27:39.358881 11370 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:39.361382 11370 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:39.395393  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 49955.6us
I20260709 12:27:39.421604  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:39.460249  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 120419us
I20260709 12:27:39.483579  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:39.503755  4569 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:39.504669 11679 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:39.524678  4496 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52808 (request call id 282) took 1069 ms. Trace:
I20260709 12:27:39.529142  4499 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52812 (request call id 298) took 1018 ms. Trace:
I20260709 12:27:39.525177  4496 rpcz_store.cc:276] 0709 12:27:38.455520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.455688 (+   168us) service_pool.cc:224] Handling call
0709 12:27:39.524610 (+1068922us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:38.638626 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:38.639187 (+   561us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:38.639227 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:27:38.639245 (+    18us) tablet.cc:662] Decoding operations
0709 12:27:38.639611 (+   366us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:38.639666 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:38.639695 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:27:38.639881 (+   186us) tablet.cc:701] Row locks acquired
0709 12:27:38.639899 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:27:38.640037 (+   138us) write_op.cc:270] Start()
0709 12:27:38.640148 (+   111us) write_op.cc:276] Timestamp: P: 1783600058639987 usec, L: 0
0709 12:27:38.640170 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:27:38.640849 (+   679us) log.cc:844] Serialized 538 byte log entry
0709 12:27:38.860164 (+219315us) op_driver.cc:464] REPLICATION: finished
0709 12:27:38.875440 (+ 15276us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb2efc85 00007effeb2f06a2 00007effee0621db 00007effee05daaf 00007effeb3ac0ac 00007efff42069a9 00007effeb3c869b 00007effeb3c693f 00007effeb3c8c02 00007efff1eecef0
0709 12:27:39.217243 (+341803us) write_op.cc:301] APPLY: starting
0709 12:27:39.217338 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:39.218566 (+  1228us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:39.218592 (+    26us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:39.219386 (+   794us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:39.219611 (+   225us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:27:39.219676 (+    65us) write_op.cc:312] APPLY: finished
0709 12:27:39.219897 (+   221us) log.cc:844] Serialized 25 byte log entry
0709 12:27:39.220811 (+   914us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:39.220903 (+    92us) write_op.cc:454] Released schema lock
0709 12:27:39.220959 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:39.221387 (+   428us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:39.221420 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":320732,"apply.run_cpu_time_us":3437,"apply.run_wall_time_us":4146,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8301,"num_ops":1,"prepare.queue_time_us":117336,"prepare.run_cpu_time_us":2570,"prepare.run_wall_time_us":2571,"replication_time_us":219770,"spinlock_wait_cycles":26861824,"thread_start_us":191442,"threads_started":2}]]}
I20260709 12:27:39.529820  4499 rpcz_store.cc:276] 0709 12:27:38.510998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.514390 (+  3392us) service_pool.cc:224] Handling call
0709 12:27:39.529085 (+1014695us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":235778,"reported_tablets":3}
I20260709 12:27:39.530318 11645 ts_tablet_manager.cc:1929] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.11
I20260709 12:27:39.563319 11645 log.cc:1199] T 1b7b0c6c811e455abe898cb4d21acad4 P 49277a7653524787a715232ab908bbc6: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root/wals/1b7b0c6c811e455abe898cb4d21acad4
I20260709 12:27:39.564922 11645 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50178 (request call id 5) took 1619 ms. Trace:
I20260709 12:27:39.565291 11645 rpcz_store.cc:276] 0709 12:27:37.948368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.950616 (+  2248us) service_pool.cc:224] Handling call
0709 12:27:38.032574 (+ 81958us) ts_tablet_manager.cc:1151] Deleting tablet 1b7b0c6c811e455abe898cb4d21acad4
0709 12:27:38.079033 (+ 46459us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:27:39.369026 (+1289993us) tablet_metadata.cc:665] Metadata flushed
0709 12:27:39.467707 (+ 98681us) tablet_metadata.cc:665] Metadata flushed
0709 12:27:39.564826 (+ 97119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46146,"tablet-delete.queue_time_us":61716,"thread_start_us":27627,"threads_started":1}
I20260709 12:27:39.589886  4488 catalog_manager.cc:5028] TS 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): tablet 1b7b0c6c811e455abe898cb4d21acad4 (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
I20260709 12:27:39.685406  4501 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:52818 (request call id 308) took 1047 ms. Trace:
I20260709 12:27:39.685799  4501 rpcz_store.cc:276] 0709 12:27:38.638073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.638940 (+   867us) service_pool.cc:224] Handling call
0709 12:27:39.685332 (+1046392us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3542,"reported_tablets":30}
I20260709 12:27:39.694314  4749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:50188
I20260709 12:27:39.694933  4749 raft_consensus.cc:606] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:27:39.704850  4749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:50188
I20260709 12:27:39.705510  4749 raft_consensus.cc:606] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:27:39.714484  4749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:50188
I20260709 12:27:39.715091  4749 raft_consensus.cc:606] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Received request to transfer leadership to TS 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:27:39.729175  4749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:50188
I20260709 12:27:39.729813  4749 raft_consensus.cc:606] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:27:39.740568  4749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:50188
I20260709 12:27:39.741144  4749 raft_consensus.cc:606] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:27:39.742946  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:39.744833  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:39.748916  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:39.749534  4598 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:39.768576 11686 raft_consensus.cc:993] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:27:39.769162 11657 raft_consensus.cc:1081] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:27:39.785873  4602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60026
I20260709 12:27:39.786561  4602 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:39.787009  4602 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:39.793634  4749 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:39.794972 11599 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:39.797984  4675 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:39.802310 11599 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:39.803203  4602 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:39.807122  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:39.807789  4675 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:39.814380  4568 consensus_peers.cc:597] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9702ea78030547e5a2f6fffe805075da. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:39.816588  4675 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:39.824128 11649 raft_consensus.cc:2955] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:39.826424  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:39.827033  4753 raft_consensus.cc:3055] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:39.827315  4753 raft_consensus.cc:740] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:39.828048  4753 consensus_queue.cc:260] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:39.829144  4753 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:39.832878  4749 raft_consensus.cc:2955] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:39.834338  4569 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:39.835210 11679 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:39.835695 11679 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:39.836529 11679 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:39.822966  4602 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:39.850721  4753 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 17.
I20260709 12:27:39.903746  4501 catalog_manager.cc:5697] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 12, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 18 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:39.911629 11697 raft_consensus.cc:993] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:27:39.912184 11697 raft_consensus.cc:1081] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:27:39.917752  4602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60026
I20260709 12:27:39.918329  4602 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:39.918695  4602 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:39.934388  4602 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:39.939684 11718 log.cc:927] Time spent T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:27:39.943423  4602 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:39.944985  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:39.945886  4680 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:39.947103  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:39.947860  4753 raft_consensus.cc:3055] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:39.948231  4753 raft_consensus.cc:740] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:39.943589  4499 catalog_manager.cc:5697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 16 to 17, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 17 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:39.950083  4753 consensus_queue.cc:260] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:39.952708  4753 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:39.958799  4681 raft_consensus.cc:2955] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:39.967700  4680 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 23.
I20260709 12:27:39.971469  4753 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 23.
I20260709 12:27:39.973941  4569 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:39.976328 11649 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:39.976835 11649 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 23 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:39.977888 11649 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:40.076113  4681 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:40.113489 11676 raft_consensus.cc:993] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: : Instructing follower 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:27:40.114034 11676 raft_consensus.cc:1081] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Signalling peer 3a5aeedd11fe49938079fe86cb01232c to start an election
I20260709 12:27:40.121212 11686 raft_consensus.cc:993] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:27:40.122563 11686 raft_consensus.cc:1081] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:27:40.125065  4602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60026
I20260709 12:27:40.125665  4602 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:40.126025  4602 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:40.126474  4679 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8"
dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
 from {username='slave'} at 127.0.0.1:51858
I20260709 12:27:40.127053  4679 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:40.127409  4679 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:40.132865  4753 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:40.136466 11641 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:27:40.146232  4644 consensus_peers.cc:597] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2d6e5f40333a47559cd852e4dfa7c86b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:40.149547  4679 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:40.152565  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:40.153115  4749 raft_consensus.cc:3055] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:40.153411  4749 raft_consensus.cc:740] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:40.155874  4679 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 election: Requested vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:40.154346  4749 consensus_queue.cc:260] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:40.157078  4600 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:40.157409  4749 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:40.169580  4602 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:40.175627  4499 catalog_manager.cc:5697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 22 to 23, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 23 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:40.182585  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:40.183236  4680 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:40.184437  4602 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:40.194059  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:40.163750 11680 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:27:40.194952  5342 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:40.202112  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:40.208462  4747 raft_consensus.cc:3055] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:40.208791  4749 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 21.
I20260709 12:27:40.208801  4747 raft_consensus.cc:740] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:40.213667  4747 consensus_queue.cc:260] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:40.217293  4647 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:40.218431  4747 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:40.218890 11680 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:40.219484 11680 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 21 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:40.221338 11680 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:40.232592  5342 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 21.
I20260709 12:27:40.257252  4680 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 23.
I20260709 12:27:40.264207 11705 raft_consensus.cc:993] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:27:40.265096 11705 raft_consensus.cc:1081] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:27:40.267849  4598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60026
I20260709 12:27:40.268446  4598 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:40.268797  4598 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:40.272287  4569 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:40.276494  4598 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:40.276998 11679 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:40.277729 11679 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 23 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:40.278990  4598 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:40.278793 11679 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:40.284397  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:40.285012  4748 raft_consensus.cc:3055] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:40.285311  4748 raft_consensus.cc:740] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:40.284381  4747 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 23.
I20260709 12:27:40.290359  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:40.290992  4680 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:40.299005  4680 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:40.302609  4569 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:40.303635 11649 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:40.304294 11649 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:40.297806  4748 consensus_queue.cc:260] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:40.307498  4748 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:40.307585 11649 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:40.324360  4680 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:40.326941 11648 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:40.350600  4748 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 19.
I20260709 12:27:40.362836 11641 raft_consensus.cc:2955] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:40.370231  4752 raft_consensus.cc:2955] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:40.394989  4602 raft_consensus.cc:2955] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:40.406423  4499 catalog_manager.cc:5697] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 20 to 21, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 21 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:40.413719  4496 catalog_manager.cc:5697] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 22 to 23, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 23 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:40.437991 11729 ts_tablet_manager.cc:933] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
I20260709 12:27:40.469467  4496 catalog_manager.cc:5697] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 10, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 16 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:40.478250  4749 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:40.483784  4501 catalog_manager.cc:5697] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 18 to 19, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 19 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:40.542222  4488 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3be428c36f9242c6a2b8aa853f2c8b2c with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 49277a7653524787a715232ab908bbc6 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 19 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:40.534736 11679 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
W20260709 12:27:40.630802 11487 log.cc:927] Time spent T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:27:40.638783 11729 tablet_copy_client.cc:323] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.193:45077
I20260709 12:27:40.647197 11746 ts_tablet_manager.cc:933] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:40.651775  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 104806us
I20260709 12:27:40.671717  4748 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:27:40.757997 11648 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.110s
I20260709 12:27:40.815897  5349 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:27:40.810474  4680 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 12:27:40.813866  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 124188us
I20260709 12:27:40.841818 11641 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.004s
I20260709 12:27:40.847110  4614 tablet_copy_service.cc:140] P 5c3201048f8346009aefca07428828f3: Received BeginTabletCopySession request for tablet 9702ea78030547e5a2f6fffe805075da from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:53448)
I20260709 12:27:40.847821  4614 tablet_copy_service.cc:161] P 5c3201048f8346009aefca07428828f3: Beginning new tablet copy session on tablet 9702ea78030547e5a2f6fffe805075da from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:53448: session id = dbe960c35c84451faf4e5b5c9233fa39-9702ea78030547e5a2f6fffe805075da
I20260709 12:27:40.866897 11746 tablet_copy_client.cc:323] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.194:42599
I20260709 12:27:40.917501  4614 tablet_copy_source_session.cc:215] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:40.929651  4752 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:40.931185  4748 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:40.953030  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:40.959088  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 147560us
W20260709 12:27:40.959410  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 197061us
I20260709 12:27:40.961498  4680 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 12:27:40.952839  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 260927us
I20260709 12:27:40.974485  4747 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:27:40.977932  4679 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:41.002643  4691 tablet_copy_service.cc:140] P 3a5aeedd11fe49938079fe86cb01232c: Received BeginTabletCopySession request for tablet 2d6e5f40333a47559cd852e4dfa7c86b from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:48198)
I20260709 12:27:41.003276  4691 tablet_copy_service.cc:161] P 3a5aeedd11fe49938079fe86cb01232c: Beginning new tablet copy session on tablet 2d6e5f40333a47559cd852e4dfa7c86b from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:48198: session id = dbe960c35c84451faf4e5b5c9233fa39-2d6e5f40333a47559cd852e4dfa7c86b
I20260709 12:27:40.956538 11648 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:27:40.963542 11740 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:27:41.043135 11729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9702ea78030547e5a2f6fffe805075da. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:41.095953  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 85869us
W20260709 12:27:41.103097  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 110778us
I20260709 12:27:41.143203  4749 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:50188
I20260709 12:27:41.144109  4749 raft_consensus.cc:606] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
W20260709 12:27:41.159168  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 164976us
W20260709 12:27:41.193185  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 157794us
I20260709 12:27:41.129906 11599 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
W20260709 12:27:41.200676  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 164991us
W20260709 12:27:41.214730  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 97188us
I20260709 12:27:41.157774 11763 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:27:41.232610 11636 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:27:41.260782  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 54243.6us
W20260709 12:27:41.279395  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 160735us
I20260709 12:27:41.291734 11740 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:27:41.294866 11767 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:27:41.342396  4691 tablet_copy_source_session.cc:215] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:41.348151 11746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d6e5f40333a47559cd852e4dfa7c86b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:41.356055  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 79121us
W20260709 12:27:41.371739  4550 auto_rebalancer.cc:629] Failed to schedule move for tablet 26f80146cffc4af2ac0af76b3a99c213: Illegal state: Replica 49277a7653524787a715232ab908bbc6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } }
W20260709 12:27:41.408983  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 44779.3us
I20260709 12:27:41.447386 11755 raft_consensus.cc:993] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:27:41.447750  4751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972"
dest_uuid: "49277a7653524787a715232ab908bbc6"
mode: GRACEFUL
new_leader_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:50188
I20260709 12:27:41.448398  4751 raft_consensus.cc:606] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Received request to transfer leadership to TS 5c3201048f8346009aefca07428828f3
I20260709 12:27:41.450842 11686 raft_consensus.cc:1081] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:27:41.452754  4748 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:41.453645  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:27:41.460836  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:41.472667  4601 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60026
I20260709 12:27:41.473624  4601 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:41.474169  4601 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:41.510146  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:41.543663  4601 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:41.553522 11781 log.cc:927] Time spent T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:27:41.554299  4601 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:41.555871  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:41.557191  4680 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:41.564082  4680 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 21.
I20260709 12:27:41.585969  4569 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:41.671257 11649 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:41.682847 11649 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 21 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:41.683993 11649 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:41.693168  4719 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 66407.8us
I20260709 12:27:41.695478  4756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:41.696131  4756 raft_consensus.cc:3055] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:41.696508  4756 raft_consensus.cc:740] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:41.723161 11746 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:50638 (request call id 1039) took 1120 ms. Trace:
I20260709 12:27:41.697294  4756 consensus_queue.cc:260] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:41.681253 11729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:50644 (request call id 994) took 1415 ms. Trace:
W20260709 12:27:41.743462  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 135977us
I20260709 12:27:41.744341  4756 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:41.730521 11746 rpcz_store.cc:276] 0709 12:27:40.602277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.602437 (+   160us) service_pool.cc:224] Handling call
0709 12:27:40.669789 (+ 67352us) ts_tablet_manager.cc:934] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
0709 12:27:41.346769 (+676980us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:27:41.560327 (+213558us) tablet_metadata.cc:665] Metadata flushed
0709 12:27:41.560462 (+   135us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:27:41.722344 (+161882us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:27:41.722671 (+   327us) tablet_replica.cc:206] Creating consensus instance
0709 12:27:41.723029 (+   358us) ts_tablet_manager.cc:989] Replica registered.
0709 12:27:41.723091 (+    62us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":318,"tablet-copy.queue_time_us":44154,"thread_start_us":81175,"threads_started":2}
I20260709 12:27:41.758316 11746 tablet_copy_client.cc:806] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:41.758960 11746 tablet_copy_client.cc:670] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:41.771801  4756 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 21.
I20260709 12:27:41.773527 11666 raft_consensus.cc:993] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: : Instructing follower 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:27:41.774124 11666 raft_consensus.cc:1081] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Signalling peer 5c3201048f8346009aefca07428828f3 to start an election
I20260709 12:27:41.743242 11729 rpcz_store.cc:276] 0709 12:27:40.216575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.216720 (+   145us) service_pool.cc:224] Handling call
0709 12:27:40.453764 (+237044us) ts_tablet_manager.cc:934] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077)
0709 12:27:41.021673 (+567909us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:27:41.349453 (+327780us) tablet_metadata.cc:665] Metadata flushed
0709 12:27:41.354714 (+  5261us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:27:41.541091 (+186377us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:27:41.541521 (+   430us) tablet_replica.cc:206] Creating consensus instance
0709 12:27:41.617248 (+ 75727us) ts_tablet_manager.cc:989] Replica registered.
0709 12:27:41.629960 (+ 12712us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":18792,"tablet-copy.queue_time_us":180637,"thread_start_us":289541,"threads_started":2}
W20260709 12:27:41.782493  4719 consensus_peers.cc:597] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aba6aa625d5a49658718de09523b8a9b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:41.795604 11746 tablet_copy_client.cc:538] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:27:41.791926 11729 tablet_copy_client.cc:806] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:41.797349 11729 tablet_copy_client.cc:670] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:41.798625  4679 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:41.802201  4602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972"
dest_uuid: "5c3201048f8346009aefca07428828f3"
 from {username='slave'} at 127.0.0.1:60026
I20260709 12:27:41.802758  4602 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:41.803138  4602 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:41.825119  4602 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:41.832438  4601 raft_consensus.cc:1240] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 21 LEADER]: Rejecting Update request from peer 49277a7653524787a715232ab908bbc6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:41.843220  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:41.843868  4674 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:41.851112 11729 tablet_copy_client.cc:538] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:27:41.862895  4674 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:41.865092  4602 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:41.875607  4569 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3; no voters: 
I20260709 12:27:41.876530 11772 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:41.877096 11772 raft_consensus.cc:697] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Becoming Leader. State: Replica: 5c3201048f8346009aefca07428828f3, State: Running, Role: LEADER
I20260709 12:27:41.878018 11772 consensus_queue.cc:237] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:41.878472  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:41.879123  4749 raft_consensus.cc:3055] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:41.879434  4749 raft_consensus.cc:740] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:41.880198  4749 consensus_queue.cc:260] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:41.881067  4749 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:41.887671  4501 catalog_manager.cc:5697] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 20 to 21, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 21 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:41.890650  4602 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:41.898967  4749 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c3201048f8346009aefca07428828f3 in term 14.
I20260709 12:27:41.914178 11729 tablet_bootstrap.cc:492] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:27:41.924381 11746 tablet_bootstrap.cc:492] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:27:41.934378 11705 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:27:41.971517 11765 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:27:42.033005 11688 raft_consensus.cc:2955] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Committing config change with OpId 20.11: config changed from index -1 to 11, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:42.038375  4674 raft_consensus.cc:2955] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Committing config change with OpId 20.11: config changed from index -1 to 11, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:42.033842  4602 raft_consensus.cc:2955] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Committing config change with OpId 20.11: config changed from index -1 to 11, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
W20260709 12:27:42.111340  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 43819.6us
I20260709 12:27:42.106246  4496 catalog_manager.cc:5697] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index -1 to 11, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 20 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:42.165658 11729 tablet_bootstrap.cc:492] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:42.166770 11729 tablet_bootstrap.cc:492] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:27:42.169459 11729 ts_tablet_manager.cc:1403] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.256s	user 0.050s	sys 0.028s
I20260709 12:27:42.170440  4756 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:27:42.174291 11649 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:42.203549  4674 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:27:42.208985 11745 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:42.231292 11729 raft_consensus.cc:359] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:42.256778 11729 raft_consensus.cc:740] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:27:42.257680 11729 consensus_queue.cc:260] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:42.270326  5344 raft_consensus.cc:1217] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEARNER]: Deduplicated request from leader. Original: 18.11->[18.12-18.12]   Dedup: 18.12->[]
I20260709 12:27:42.277534 11729 ts_tablet_manager.cc:1434] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.108s	user 0.009s	sys 0.000s
I20260709 12:27:42.278091  4501 catalog_manager.cc:5697] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 reported cstate change: term changed from 13 to 14, leader changed from 49277a7653524787a715232ab908bbc6 (127.3.89.195) to 5c3201048f8346009aefca07428828f3 (127.3.89.193). New cstate: current_term: 14 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:42.283896  4614 tablet_copy_service.cc:342] P 5c3201048f8346009aefca07428828f3: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-9702ea78030547e5a2f6fffe805075da received from {username='slave'} at 127.0.0.1:53448
I20260709 12:27:42.284669  4614 tablet_copy_service.cc:434] P 5c3201048f8346009aefca07428828f3: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-9702ea78030547e5a2f6fffe805075da on tablet 9702ea78030547e5a2f6fffe805075da with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:42.314481 11729 ts_tablet_manager.cc:933] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:42.367573 11746 tablet_bootstrap.cc:492] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:42.370056 11746 tablet_bootstrap.cc:492] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:27:42.370816 11746 ts_tablet_manager.cc:1403] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.447s	user 0.046s	sys 0.006s
I20260709 12:27:42.395438  4748 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:27:42.397423 11734 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:42.401702 11729 tablet_copy_client.cc:323] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.195:41049
I20260709 12:27:42.405349  4766 tablet_copy_service.cc:140] P 49277a7653524787a715232ab908bbc6: Received BeginTabletCopySession request for tablet aba6aa625d5a49658718de09523b8a9b from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:42632)
I20260709 12:27:42.405923  4766 tablet_copy_service.cc:161] P 49277a7653524787a715232ab908bbc6: Beginning new tablet copy session on tablet aba6aa625d5a49658718de09523b8a9b from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:42632: session id = dbe960c35c84451faf4e5b5c9233fa39-aba6aa625d5a49658718de09523b8a9b
I20260709 12:27:42.435201  4766 tablet_copy_source_session.cc:215] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:42.439458 11729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aba6aa625d5a49658718de09523b8a9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:42.447270 11746 raft_consensus.cc:359] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:42.457742 11746 raft_consensus.cc:740] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:27:42.458580 11746 consensus_queue.cc:260] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:42.486052  4674 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:27:42.492830 11734 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:42.514540 11746 ts_tablet_manager.cc:1434] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.143s	user 0.008s	sys 0.000s
I20260709 12:27:42.518321  4691 tablet_copy_service.cc:342] P 3a5aeedd11fe49938079fe86cb01232c: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-2d6e5f40333a47559cd852e4dfa7c86b received from {username='slave'} at 127.0.0.1:48198
I20260709 12:27:42.519621  4691 tablet_copy_service.cc:434] P 3a5aeedd11fe49938079fe86cb01232c: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-2d6e5f40333a47559cd852e4dfa7c86b on tablet 2d6e5f40333a47559cd852e4dfa7c86b with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:42.581851 11729 tablet_copy_client.cc:806] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:42.582495 11729 tablet_copy_client.cc:670] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:42.591931 11729 tablet_copy_client.cc:538] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:27:42.615442  4488 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3be428c36f9242c6a2b8aa853f2c8b2c with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 49277a7653524787a715232ab908bbc6 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 19 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:42.631951 11729 tablet_bootstrap.cc:492] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:27:42.679354  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:42.809878 11729 tablet_bootstrap.cc:492] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=1 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:42.810900 11729 tablet_bootstrap.cc:492] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:27:42.811589 11729 ts_tablet_manager.cc:1403] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.180s	user 0.041s	sys 0.012s
I20260709 12:27:42.818993 11729 raft_consensus.cc:359] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:42.820963  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:27:42.819855 11729 raft_consensus.cc:740] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:27:42.822149 11729 consensus_queue.cc:260] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:42.823112  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:42.827795  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:42.830542  5349 raft_consensus.cc:1217] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.10->[20.11-20.11]   Dedup: 20.11->[]
I20260709 12:27:42.824851 11729 ts_tablet_manager.cc:1434] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:27:42.835623  4766 tablet_copy_service.cc:342] P 49277a7653524787a715232ab908bbc6: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-aba6aa625d5a49658718de09523b8a9b received from {username='slave'} at 127.0.0.1:42632
I20260709 12:27:42.836576  4766 tablet_copy_service.cc:434] P 49277a7653524787a715232ab908bbc6: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-aba6aa625d5a49658718de09523b8a9b on tablet aba6aa625d5a49658718de09523b8a9b with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:42.854063  5349 raft_consensus.cc:1217] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.9->[16.10-16.10]   Dedup: 16.10->[]
I20260709 12:27:42.903963  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:42.913867  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 59072.5us
I20260709 12:27:42.985394  3431 tablet_server.cc:179] TabletServer@127.3.89.193:0 shutting down...
W20260709 12:27:42.988682  4645 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:42.990479 11817 raft_consensus.cc:1064] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:27:42.999563 11817 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:43.026516  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 64143.2us
W20260709 12:27:43.036096  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.036783  5312 consensus_peers.cc:597] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.043179  4645 consensus_peers.cc:597] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.044116  4645 consensus_peers.cc:597] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.079401  4645 consensus_peers.cc:597] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:43.109248  4748 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:27:43.111325  4645 consensus_peers.cc:597] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.112039  4645 consensus_peers.cc:597] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.102871  5312 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.118462  4645 consensus_peers.cc:597] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.121069  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 79572.7us
I20260709 12:27:43.132488 11772 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:27:43.152741  5349 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEARNER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:27:43.190582  5312 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.191344  5312 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:43.192310 11734 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:43.196898  4674 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer 5c3201048f8346009aefca07428828f3: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:43.202754 11826 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:43.206967  4645 consensus_peers.cc:597] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.225981  4720 consensus_peers.cc:597] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.235041  5312 consensus_peers.cc:597] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.251168  5312 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.255028  4645 consensus_peers.cc:597] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.263110  4720 consensus_peers.cc:597] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.263777  4720 consensus_peers.cc:597] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.264297  4720 consensus_peers.cc:597] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.266455  4720 consensus_peers.cc:597] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.269352  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 68295.2us
W20260709 12:27:43.269475  4720 consensus_peers.cc:597] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:43.274164 11816 raft_consensus.cc:2955] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:43.276472  5344 raft_consensus.cc:2955] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:43.282811  4645 consensus_peers.cc:597] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.317799  4720 consensus_peers.cc:597] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:43.330686  4498 catalog_manager.cc:5697] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "5c3201048f8346009aefca07428828f3" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:43.349664  4720 consensus_peers.cc:597] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:43.346580 11755 raft_consensus.cc:1064] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
W20260709 12:27:43.357831  4720 consensus_peers.cc:597] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.365797  4720 consensus_peers.cc:597] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:43.370645 11755 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 9, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:43.377389  5312 consensus_peers.cc:597] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.393496  5312 consensus_peers.cc:597] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:43.403048  4675 raft_consensus.cc:2955] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:43.406198  4720 consensus_peers.cc:597] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:43.464452  4748 raft_consensus.cc:2955] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:43.533455  4645 consensus_peers.cc:597] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.547730  4645 consensus_peers.cc:597] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.570171  4645 consensus_peers.cc:597] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:43.611747  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 136844us
I20260709 12:27:43.647715  5344 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [term 20 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:43.665270  4675 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:43.669981 11832 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
W20260709 12:27:43.746294  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 67072.4us
W20260709 12:27:43.774241  4720 consensus_peers.cc:597] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:43.805270 11855 raft_consensus.cc:1064] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
W20260709 12:27:43.805608 11834 log.cc:927] Time spent T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
W20260709 12:27:43.844614  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 127960us
I20260709 12:27:43.818459 11855 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:43.884764  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 188417us
I20260709 12:27:43.990733 11825 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:44.025673 11825 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.024717 11734 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:44.048903 11825 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.048652 11734 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:44.050977 11734 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.052896 11826 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:44.053474 11826 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.055454 11861 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:44.055655 11826 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.056107 11861 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.056980 11871 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.057591 11871 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.058547 11861 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.060559 11871 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.061228  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.063773  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.071518  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.073846  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:44.045253  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 174233us
W20260709 12:27:44.077858  4720 consensus_peers.cc:597] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:44.091750  4647 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 97689.2us
W20260709 12:27:44.104835  4720 consensus_peers.cc:597] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:44.106196  4720 consensus_peers.cc:597] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:44.129650 11872 raft_consensus.cc:493] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:44.130337 11872 raft_consensus.cc:515] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.132697 11734 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:44.133324 11734 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.135691 11734 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.137499 11734 raft_consensus.cc:493] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:44.138099 11734 raft_consensus.cc:515] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.118872 11861 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:44.142021 11861 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.143676 11734 leader_election.cc:290] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.144251 11861 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.144927 11842 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:44.154723 11872 leader_election.cc:290] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.157327  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.162494  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.164249  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:44.171802  4645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:44.180022  5314 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 85247.6us
I20260709 12:27:44.173415 11872 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.181107 11817 raft_consensus.cc:493] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:44.191391 11817 raft_consensus.cc:515] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.185622 11734 raft_consensus.cc:493] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.199575 11734 raft_consensus.cc:515] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.204201 11734 leader_election.cc:290] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.218096  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.210294 11872 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.248806 11745 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.250748 11745 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:44.253101 11745 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.255002 11860 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.255502 11816 raft_consensus.cc:493] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.255650 11860 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.256057 11816 raft_consensus.cc:515] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.255146 11872 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.259245  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.260907  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.262609  4753 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:44.257995 11816 leader_election.cc:290] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.267808 11860 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:44.270539  4720 consensus_peers.cc:597] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:44.271056  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 156751us
I20260709 12:27:44.282701 11826 raft_consensus.cc:493] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:44.283376 11826 raft_consensus.cc:515] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.287730 11745 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:44.288405 11745 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.285495 11826 leader_election.cc:290] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.289820 11817 leader_election.cc:290] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.290766 11745 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.290848 11734 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.291501 11734 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.294106  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.295254  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.295221 11734 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.296885  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.305399  5349 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:27:44.317998  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 238812us
I20260709 12:27:44.314585 11856 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:27:44.323971 11770 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:44.345234  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:44.343068 11860 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:44.349999 11873 raft_consensus.cc:493] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.350680 11873 raft_consensus.cc:515] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.349011 11860 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.352936 11873 leader_election.cc:290] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.353130 11734 raft_consensus.cc:493] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:44.357542  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.360070  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.353911 11734 raft_consensus.cc:515] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.370939 11734 leader_election.cc:290] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.353636 11860 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.384095  5344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b7b0c6c811e455abe898cb4d21acad4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:44.386334  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d410e14541184b6bbaff38737ce4bc02" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:44.387789  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:44.392840  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:44.394003  5349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:44.395118  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:44.396283  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:44.397912  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc78aa9a3bf48afb52246b50ad414eb" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:44.414134  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.415597  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abf423ac214b42acbe1696a11279567c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.418038  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.422503  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f682ea56c3b4f2bb6e9413f3d98c6fd" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.415954  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.425092  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b27de60f9f44928b0068ab1e562568c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.427294  4568 leader_election.cc:304] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:44.439651  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34a0b6acf7c6497e8ef8ffd63b4127d6" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.446076 11873 raft_consensus.cc:2749] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:44.446163  4568 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:44.451920  4568 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:44.453902 11873 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:44.465724 11865 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:44.471882 11865 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.474433 11865 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.450637 11817 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:44.485900 11817 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.488013 11817 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.434144 11734 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.494376 11734 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.496817 11734 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.454669 11825 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:44.499531 11755 raft_consensus.cc:2955] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Committing config change with OpId 20.12: config changed from index 11 to 12, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:44.503000  4680 raft_consensus.cc:2955] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index 11 to 12, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:44.504814  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e20c8bae3534224bb7a82db42cab1a5" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.507357  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4a575b75e05458186c89873e1e7622b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.511049  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:44.513403  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba6aa625d5a49658718de09523b8a9b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.513646 11828 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:44.514307 11828 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.515321 11883 raft_consensus.cc:493] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.521989  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca7c574e14c4faeac378ee00c0a261b" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.535588  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.537019  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70fb561fb8b446969622ee2193d90917" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.537981  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.538926  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cd71e05324451f8e6dfcf6c2e76829" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:44.520545 11872 raft_consensus.cc:493] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:44.552580  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.553325  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.555840  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:44.562707  3431 tablet_replica.cc:333] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:44.564224  3431 raft_consensus.cc:2243] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Raft consensus shutting down.
W20260709 12:27:44.565975  4645 consensus_peers.cc:597] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:44.530556 11817 raft_consensus.cc:493] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:44.572012  4499 catalog_manager.cc:5697] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 11 to 12, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:44.578967 11888 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.579607 11888 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.591359  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.591670  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.574136 11826 raft_consensus.cc:493] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.597131 11888 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.588991 11745 raft_consensus.cc:493] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:44.574460 11872 raft_consensus.cc:515] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.538305 11883 raft_consensus.cc:515] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.619180  3431 raft_consensus.cc:2272] T 9ced730aeaf240b69d68fd0ee470d68d P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:44.611776 11865 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.612088 11871 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.666505 11865 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.590965 11817 raft_consensus.cc:515] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.617781 11828 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.608079 11826 raft_consensus.cc:515] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:44.728241 11893 log.cc:927] Time spent T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:27:44.730721  5345 raft_consensus.cc:2955] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index 11 to 12, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:44.692682 11871 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.741084 11871 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.742895 11872 leader_election.cc:290] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.743247  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:44.627485 11745 raft_consensus.cc:515] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:44.750438  5342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:44.755362 11883 leader_election.cc:290] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.759753 11865 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:44.776408  4486 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 97624.1us
W20260709 12:27:44.794673  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 44669.8us
I20260709 12:27:44.801138  3431 tablet_replica.cc:333] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:44.814677  3431 raft_consensus.cc:2243] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:44.825472  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.837594 11828 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:27:44.850113 11828 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.850533  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.855129  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea533a99927c40d48b894cf00a75301d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.858403  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.863534  3431 raft_consensus.cc:2272] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:44.872804  4486 catalog_manager.cc:4755] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): ChangeConfig:REMOVE_PEER RPC failed for tablet 9702ea78030547e5a2f6fffe805075da: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:44.873099 11883 raft_consensus.cc:493] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:27:44.915156  3431 tablet_replica.cc:333] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:44.925781 11745 leader_election.cc:290] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.928138 11817 leader_election.cc:290] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:44.930538 11828 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.936997  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.916742 11833 raft_consensus.cc:2955] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index 10 to 11, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:44.962656 11883 raft_consensus.cc:515] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.965240 11883 leader_election.cc:290] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.940579  4569 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:44.949735 11826 leader_election.cc:290] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:44.969290 11842 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:44.872146  4748 raft_consensus.cc:2955] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:44.954927  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.966508  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:44.973095  3431 raft_consensus.cc:2243] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:44.983196  4569 leader_election.cc:304] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:44.989401 11871 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:44.977295 11842 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:44.993489 11842 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:44.999351  4675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:45.012888  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42d36b6bb6e84733876824d95e25967e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.014618 11828 raft_consensus.cc:493] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:27:45.015426  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51b7b2074e114d8f872f149fa8acdbcf" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:45.018026  4568 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 146448us
I20260709 12:27:45.021396  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:45.022249 11817 raft_consensus.cc:2749] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:45.022998  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6f49e7bc416420cb85a6d4319f46c4d" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.021031  5345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dafe783bf87845d6bb64a279ed7d0f75" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:45.015270 11828 raft_consensus.cc:515] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:45.031708 11828 leader_election.cc:290] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:45.044142  3431 raft_consensus.cc:2272] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.049127  3431 tablet_replica.cc:333] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.054049  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:45.054774  3431 raft_consensus.cc:2243] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.055568  3431 raft_consensus.cc:2272] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.055469  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:45.084205  3431 tablet_replica.cc:333] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.088092  3431 raft_consensus.cc:2243] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:45.092293  4569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.096824  3431 raft_consensus.cc:2272] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.102914  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8ee74acbd5c45f0934f6f37d2154bc7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:45.109953  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 85827.6us
W20260709 12:27:45.119032  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 94723.9us
I20260709 12:27:45.130959  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6e604992a304031ad5cbdc32d90edb4" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.132462  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3be428c36f9242c6a2b8aa853f2c8b2c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.132874  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23027d24ef1943bdbea76085a97b055c" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.134727  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.132189  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae7ffb2806e441e787796287d56e8b8a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.137841  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18c2a1115a9a4b4a975dadb63bef992a" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.138774  4750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501b9483d1b74b73876563dd85e610d0" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.139093  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4163edf182b549dfb724d2e5b5b1ac0e" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.139865  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485fe0cee4b248f4802d3c15dc97a877" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.140022  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53908897ce974579947af90acaf0aed7" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.141300  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12796fc41f234615a607ba222506e893" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:45.149904  4569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 51155.8us
I20260709 12:27:45.151840  3431 tablet_replica.cc:333] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.152835  3431 raft_consensus.cc:2243] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.156293  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d675fbac0f844dc9309a662ba1c96f9" candidate_uuid: "5c3201048f8346009aefca07428828f3" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.174799  3431 raft_consensus.cc:2272] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.190279  5346 raft_consensus.cc:2955] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:45.193590  4571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:41049, user_credentials={real_user=slave}} blocked reactor thread for 57477.9us
I20260709 12:27:45.198127  3431 tablet_replica.cc:333] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.199663  3431 raft_consensus.cc:2243] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:27:45.196872  4753 consensus_queue.cc:237] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:45.201177  4571 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.209852  4499 catalog_manager.cc:5697] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 10 to 11, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:45.220628  4571 raft_consensus.cc:2686] T 2d6e5f40333a47559cd852e4dfa7c86b P 5c3201048f8346009aefca07428828f3: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:27:45.221515  5345 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:45.223029 11844 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:45.225953  4571 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.226738 11828 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:45.230751  3431 raft_consensus.cc:2272] T 8f05b2c44da743e49df87266876b3604 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.230715  4571 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.207356  4568 leader_election.cc:304] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:45.232885  4571 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:45.233559 11817 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:45.234352  4673 raft_consensus.cc:1275] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:45.234369 11895 raft_consensus.cc:2749] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:45.237296  3431 tablet_replica.cc:333] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.238288  3431 raft_consensus.cc:2243] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:27:45.241459 11844 raft_consensus.cc:2955] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index 12 to 13, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:45.245756 11863 consensus_queue.cc:1048] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:45.239185  3431 raft_consensus.cc:2272] T e85386ee6a30465188e201d5ee6ae972 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.247462  5345 raft_consensus.cc:2955] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:45.249521  3431 tablet_replica.cc:333] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.250515  3431 raft_consensus.cc:2243] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:27:45.252584  4674 raft_consensus.cc:2955] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:45.258666  4568 leader_election.cc:304] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:45.259351  4568 raft_consensus.cc:2686] T a8ee74acbd5c45f0934f6f37d2154bc7 P 5c3201048f8346009aefca07428828f3: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:27:45.265929 11828 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:45.266775  3431 raft_consensus.cc:2272] T e8a74137ba104fe0b465cab6399c4940 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.275270  4571 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.276073 11865 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:45.277446  4571 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.278209 11865 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:45.279323  4571 leader_election.cc:304] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:45.280872  4488 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet aba6aa625d5a49658718de09523b8a9b with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:27:45.281708 11865 raft_consensus.cc:2749] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:27:45.291769  4488 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet aba6aa625d5a49658718de09523b8a9b with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER failed with leader 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 12 but the committed config has opid_index of 13
I20260709 12:27:45.292471  3431 tablet_replica.cc:333] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.293905  4571 leader_election.cc:304] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:45.294116  4673 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:45.294677 11865 raft_consensus.cc:2749] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:45.295847  4571 leader_election.cc:304] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.293421  3431 raft_consensus.cc:2243] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.296857 11865 raft_consensus.cc:2749] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:45.301537  5345 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:45.303591 11867 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:45.306988  4753 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:45.311744  4571 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.311442 11867 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:45.312979 11856 raft_consensus.cc:2955] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:45.315407 11865 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:45.316711  4571 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:45.317458 11865 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:45.320354  4571 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.321558 11865 raft_consensus.cc:2749] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:45.323113  4499 catalog_manager.cc:5697] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 12 to 13, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New cstate: current_term: 20 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:45.326308  5345 raft_consensus.cc:2955] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:45.332057  4720 leader_election.cc:336] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:45.332564  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:45.333293 11842 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:45.301517  3431 raft_consensus.cc:2272] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.336706  3431 tablet_replica.cc:333] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.337498  3431 raft_consensus.cc:2243] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.338744  4753 raft_consensus.cc:2955] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:45.347354  4571 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.353768  3431 raft_consensus.cc:2272] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.355430  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2d6e5f40333a47559cd852e4dfa7c86b with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:27:45.356385 11828 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:45.363271  4571 leader_election.cc:304] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:45.363775  4499 catalog_manager.cc:5697] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 11 to 12, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New cstate: current_term: 16 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:45.393795 11828 raft_consensus.cc:2749] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:45.395202  4571 leader_election.cc:304] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:27:45.395852  4571 raft_consensus.cc:2686] T b7cd71e05324451f8e6dfcf6c2e76829 P 5c3201048f8346009aefca07428828f3: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:27:45.396409  3431 tablet_replica.cc:333] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.396724  4571 leader_election.cc:304] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:27:45.397396  4571 raft_consensus.cc:2686] T 42d36b6bb6e84733876824d95e25967e P 5c3201048f8346009aefca07428828f3: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:27:45.397511  3431 raft_consensus.cc:2243] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:27:45.398582  4571 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.422372 11828 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:45.428076  3431 raft_consensus.cc:2272] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.430734  4571 leader_election.cc:304] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:45.432006  4571 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.438036  4571 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:45.436327 11895 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:45.443275  4571 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.435712 11828 raft_consensus.cc:2749] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:45.446275  4571 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.442781 11883 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:45.446060 11734 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:45.452116  4571 raft_consensus.cc:2686] T ae7ffb2806e441e787796287d56e8b8a P 5c3201048f8346009aefca07428828f3: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:27:45.453784  4571 leader_election.cc:304] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.461267  4571 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.462642  4571 leader_election.cc:304] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.462134  3431 tablet_replica.cc:333] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.465530  4571 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.465867  3431 raft_consensus.cc:2243] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.462749 11828 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:45.464625 11817 raft_consensus.cc:2749] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:45.470541  4571 raft_consensus.cc:2686] T 485fe0cee4b248f4802d3c15dc97a877 P 5c3201048f8346009aefca07428828f3: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:27:45.461860 11883 raft_consensus.cc:2749] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:45.471553  4571 leader_election.cc:304] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:27:45.486685  4571 raft_consensus.cc:2686] T 53908897ce974579947af90acaf0aed7 P 5c3201048f8346009aefca07428828f3: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:27:45.487668  4571 leader_election.cc:304] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:45.489207  3431 raft_consensus.cc:2272] T b6e604992a304031ad5cbdc32d90edb4 P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.489229  4571 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c3201048f8346009aefca07428828f3; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:45.489930 11883 raft_consensus.cc:2749] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:45.493710 11895 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:45.508409  3431 tablet_replica.cc:333] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.509297  3431 raft_consensus.cc:2243] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.520529  3431 raft_consensus.cc:2272] T 4163edf182b549dfb724d2e5b5b1ac0e P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:45.522372  4645 consensus_peers.cc:597] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:45.528781  3431 tablet_replica.cc:333] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.529750  3431 raft_consensus.cc:2243] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.530409  3431 raft_consensus.cc:2272] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 5c3201048f8346009aefca07428828f3 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.541626  3431 tablet_replica.cc:333] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
W20260709 12:27:45.542474  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:45.542573  3431 raft_consensus.cc:2243] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.543610  3431 raft_consensus.cc:2272] T 1b7b0c6c811e455abe898cb4d21acad4 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.546243  3431 tablet_replica.cc:333] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.547101  3431 raft_consensus.cc:2243] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.549665  3431 raft_consensus.cc:2272] T 51b7b2074e114d8f872f149fa8acdbcf P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.552567  3431 tablet_replica.cc:333] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.567200  3431 raft_consensus.cc:2243] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.567937  3431 raft_consensus.cc:2272] T ea533a99927c40d48b894cf00a75301d P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.570823  3431 tablet_replica.cc:333] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.571678  3431 raft_consensus.cc:2243] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:27:45.572780  3431 raft_consensus.cc:2272] T 231dbfb1b78d491eb8685070c2431d82 P 5c3201048f8346009aefca07428828f3 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.581300  3431 tablet_replica.cc:333] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.591303  3431 raft_consensus.cc:2243] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.591995  3431 raft_consensus.cc:2272] T 8fc78aa9a3bf48afb52246b50ad414eb P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.594895  3431 tablet_replica.cc:333] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.595755  3431 raft_consensus.cc:2243] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:27:45.596928  3431 raft_consensus.cc:2272] T f173b0d5264e4eff8b1704becc555780 P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.599962  3431 tablet_replica.cc:333] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.600787  3431 raft_consensus.cc:2243] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.601413  3431 raft_consensus.cc:2272] T 7b27de60f9f44928b0068ab1e562568c P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.613219  3431 tablet_replica.cc:333] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.614142  3431 raft_consensus.cc:2243] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:27:45.615257  3431 raft_consensus.cc:2272] T 23fb4c89626845d8bac3ba87a5346995 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.618074  3431 tablet_replica.cc:333] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.618961  3431 raft_consensus.cc:2243] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.619598  3431 raft_consensus.cc:2272] T c6f49e7bc416420cb85a6d4319f46c4d P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.626547 11833 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:45.627187 11833 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:45.631752  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:45.629452 11833 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:45.638487  4645 leader_election.cc:336] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:45.639024  4645 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:45.639806  3431 tablet_replica.cc:333] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.640767  3431 raft_consensus.cc:2243] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.641359  3431 raft_consensus.cc:2272] T 4d675fbac0f844dc9309a662ba1c96f9 P 5c3201048f8346009aefca07428828f3 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.642266 11833 raft_consensus.cc:2749] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:45.644178  3431 tablet_replica.cc:333] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.645088  3431 raft_consensus.cc:2243] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.645823  3431 raft_consensus.cc:2272] T 9ca7c574e14c4faeac378ee00c0a261b P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.648965  3431 tablet_replica.cc:333] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.657928  3431 raft_consensus.cc:2243] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 21 LEADER]: Raft consensus shutting down.
W20260709 12:27:45.658545 11828 consensus_queue.cc:1175] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "49277a7653524787a715232ab908bbc6" responder_term: 21 status { last_received { term: 21 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 21 index: 12 } }
I20260709 12:27:45.659422  3431 raft_consensus.cc:2272] T 09d362006ea24f48a30fa120bbdf8860 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.662276  3431 tablet_replica.cc:333] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.663172  3431 raft_consensus.cc:2243] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:27:45.664240  3431 raft_consensus.cc:2272] T 1d3124fc8af34f378ebd9cff21ee0e6b P 5c3201048f8346009aefca07428828f3 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.677918  3431 tablet_replica.cc:333] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.678861  3431 raft_consensus.cc:2243] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.679590  3431 raft_consensus.cc:2272] T 70fb561fb8b446969622ee2193d90917 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.682358  3431 tablet_replica.cc:333] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.683169  3431 raft_consensus.cc:2243] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.683785  3431 raft_consensus.cc:2272] T 7cc90257afed46488d4574c06b22a1a4 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.686570  3431 tablet_replica.cc:333] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.687394  3431 raft_consensus.cc:2243] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.688272  3431 raft_consensus.cc:2272] T dafe783bf87845d6bb64a279ed7d0f75 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.733876  3431 tablet_replica.cc:333] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.734758  3431 raft_consensus.cc:2243] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:45.735517  3431 raft_consensus.cc:2272] T aba6aa625d5a49658718de09523b8a9b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.752832  3431 tablet_replica.cc:333] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.759464  3431 raft_consensus.cc:2243] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 23 LEADER]: Raft consensus shutting down.
W20260709 12:27:45.767390  4720 consensus_peers.cc:597] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:45.825482  3431 raft_consensus.cc:2272] T 26f80146cffc4af2ac0af76b3a99c213 P 5c3201048f8346009aefca07428828f3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:45.832619  3431 tablet_replica.cc:333] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:45.835857  3431 raft_consensus.cc:2243] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:45.839263  4645 consensus_peers.cc:597] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:45.859951 11839 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:45.861307 11839 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:45.896010  4720 leader_election.cc:336] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:45.898433 11839 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:45.901731  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:45.904194  4674 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:45.908543  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:45.913910 11839 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:45.914398 11839 raft_consensus.cc:493] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:45.914830 11839 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:45.929466  4486 catalog_manager.cc:4755] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet aba6aa625d5a49658718de09523b8a9b: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 12:27:45.931962 11755 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:45.932727 11755 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:45.935726 11755 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:45.945680  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:45.951545  4645 consensus_peers.cc:597] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:45.968094  3431 raft_consensus.cc:2272] T 23027d24ef1943bdbea76085a97b055c P 5c3201048f8346009aefca07428828f3 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:45.995190  5312 consensus_peers.cc:597] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.005645  3431 tablet_replica.cc:333] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.034885 11869 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.036177 11869 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:46.010987  5312 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.040464 11869 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:46.044371  4720 leader_election.cc:336] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:46.048243  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:46.049278 11842 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:46.050863  4720 consensus_peers.cc:597] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.054503  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:46.042191  3431 raft_consensus.cc:2243] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:46.060485  3431 raft_consensus.cc:2272] T 13ebb8f343c743a1a28db116a4881004 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:46.073047  4720 leader_election.cc:336] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:46.073666  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:46.075228 11842 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:46.075724  4645 consensus_peers.cc:597] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.080245  3431 tablet_replica.cc:333] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.042614 11839 raft_consensus.cc:515] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:46.082760 11839 leader_election.cc:290] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:46.092120  3431 raft_consensus.cc:2243] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:46.093451  3431 raft_consensus.cc:2272] T 12796fc41f234615a607ba222506e893 P 5c3201048f8346009aefca07428828f3 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:46.099174  3431 tablet_replica.cc:333] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.100455  3431 raft_consensus.cc:2243] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:46.101910  3431 raft_consensus.cc:2272] T d410e14541184b6bbaff38737ce4bc02 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:46.103307  4645 consensus_peers.cc:597] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.106045  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ced730aeaf240b69d68fd0ee470d68d" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:46.106765  4674 raft_consensus.cc:3060] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:46.120831  4674 raft_consensus.cc:2468] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
W20260709 12:27:46.121899  4645 consensus_peers.cc:597] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.141974  4720 leader_election.cc:304] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:46.143175 11755 raft_consensus.cc:2804] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:46.143800 11755 raft_consensus.cc:697] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:27:46.150986  5312 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.151608  3431 tablet_replica.cc:333] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.145074 11755 consensus_queue.cc:237] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:46.153156  3431 raft_consensus.cc:2243] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:46.153806  4720 consensus_peers.cc:597] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.154204  3431 raft_consensus.cc:2272] T abf423ac214b42acbe1696a11279567c P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:46.164413  5312 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.166289  3431 tablet_replica.cc:333] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.167608  3431 raft_consensus.cc:2243] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:46.169513  3431 raft_consensus.cc:2272] T 3e20c8bae3534224bb7a82db42cab1a5 P 5c3201048f8346009aefca07428828f3 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:46.173209  3431 tablet_replica.cc:333] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.182425  3431 raft_consensus.cc:2243] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:46.183776  3431 raft_consensus.cc:2272] T b4a575b75e05458186c89873e1e7622b P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:46.189741  4720 leader_election.cc:336] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:46.204437  4720 consensus_peers.cc:597] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:46.205303  4720 consensus_peers.cc:597] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.225876  3431 tablet_replica.cc:333] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.226972  3431 raft_consensus.cc:2243] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:46.235810  4720 consensus_peers.cc:597] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.240250  4496 catalog_manager.cc:5697] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 19 to 20, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 20 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:46.243774  4720 consensus_peers.cc:597] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:46.253427  5312 consensus_peers.cc:597] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.250489  3431 raft_consensus.cc:2272] T 3be428c36f9242c6a2b8aa853f2c8b2c P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:46.265852  4645 consensus_peers.cc:597] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.266655  3431 tablet_replica.cc:333] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.268391  3431 raft_consensus.cc:2243] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:46.269285  3431 raft_consensus.cc:2272] T f54c73ae37134549b12188dddc4fa794 P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:46.288286  3431 tablet_replica.cc:333] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.289702  3431 raft_consensus.cc:2243] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:46.294278  3431 raft_consensus.cc:2272] T 501b9483d1b74b73876563dd85e610d0 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:46.295907  5312 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.298698  3431 tablet_replica.cc:333] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3: stopping tablet replica
W20260709 12:27:46.308768  4720 consensus_peers.cc:597] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.320516  3431 raft_consensus.cc:2243] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:46.321609  3431 raft_consensus.cc:2272] T 18c2a1115a9a4b4a975dadb63bef992a P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:46.357388  4720 consensus_peers.cc:597] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:46.361539  5312 consensus_peers.cc:597] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:46.365604  4645 consensus_peers.cc:597] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.374241  3431 tablet_replica.cc:333] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
W20260709 12:27:46.379863  4720 consensus_peers.cc:597] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:46.380230  5312 consensus_peers.cc:597] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.381155  3431 raft_consensus.cc:2243] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 21 LEADER]: Raft consensus shutting down.
W20260709 12:27:46.383886  4720 consensus_peers.cc:597] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.389714  3431 raft_consensus.cc:2272] T 8d9a7637095341feabd700d53579d5d9 P 5c3201048f8346009aefca07428828f3 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:46.395313  3431 tablet_replica.cc:333] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.398566  3431 raft_consensus.cc:2243] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:27:46.402842  3431 raft_consensus.cc:2272] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 5c3201048f8346009aefca07428828f3 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:46.409973  3431 tablet_replica.cc:333] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.421489  3431 raft_consensus.cc:2243] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:27:46.423025  3431 raft_consensus.cc:2272] T ee319a06cc4642fba7ae226abe36e352 P 5c3201048f8346009aefca07428828f3 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:46.425751  4645 consensus_peers.cc:597] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.426586  3431 tablet_replica.cc:333] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.427688  3431 raft_consensus.cc:2243] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:46.428694  3431 raft_consensus.cc:2272] T 86297ef907dd4e3590805d3cf15efaca P 5c3201048f8346009aefca07428828f3 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:46.432204  3431 tablet_replica.cc:333] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.442209  3431 raft_consensus.cc:2243] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:46.443467  3431 raft_consensus.cc:2272] T 96cc733643644a9ca6f1b2613e987020 P 5c3201048f8346009aefca07428828f3 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:46.447252  3431 tablet_replica.cc:333] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.448084  3431 raft_consensus.cc:2243] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:46.449160  3431 raft_consensus.cc:2272] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:46.458866  3431 tablet_replica.cc:333] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3: stopping tablet replica
I20260709 12:27:46.459638  3431 raft_consensus.cc:2243] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:27:46.465130  3431 raft_consensus.cc:2272] T 9702ea78030547e5a2f6fffe805075da P 5c3201048f8346009aefca07428828f3 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:46.487581  4720 consensus_peers.cc:597] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:46.510082  4720 consensus_peers.cc:597] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.519251 11755 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.520256 11755 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:46.524199 11880 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.524240  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:46.525202 11880 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:46.526405  5346 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:46.526971 11755 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:46.529539  4720 leader_election.cc:336] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:46.531032  4719 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:46.532272  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:46.532630 11880 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:46.533016  4748 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 20.
I20260709 12:27:46.536657  5314 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:46.538831 11882 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:46.539310 11882 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.539772 11882 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:46.545843 11839 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:46.546391 11839 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.546860 11839 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:46.554800  5312 leader_election.cc:336] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:46.575927  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 46421.6us
I20260709 12:27:46.599892 11867 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.600572 11867 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:46.603713 11867 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:46.610608  4645 consensus_peers.cc:597] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.611748  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:46.612506  4748 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:27:46.613886  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:46.616182 11856 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:46.616611 11856 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.617030 11856 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:46.688391 11839 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:46.691836  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:46.721774 11839 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:46.736073  4720 leader_election.cc:336] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:46.754212 11904 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.754940 11904 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:46.757833 11904 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:46.758318  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:46.759653  4488 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3be428c36f9242c6a2b8aa853f2c8b2c with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 49277a7653524787a715232ab908bbc6 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 19 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:46.771173 11856 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:46.735777 11882 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:46.785769  4486 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 48219.5us
I20260709 12:27:46.786531 11904 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.787140 11904 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:46.788751 11867 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.790634  5346 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 21: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:46.789352 11867 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:46.792455  4719 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:46.793603 11755 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:46.793732 11867 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:46.794341 11867 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.794632  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:46.794905 11867 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:46.795382  4748 raft_consensus.cc:2393] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 21: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:46.796995 11867 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:46.798316  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:46.800256  4720 consensus_peers.cc:597] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:46.798913  4748 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
W20260709 12:27:46.803409  5312 leader_election.cc:336] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:46.803705  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:46.805892  4680 raft_consensus.cc:1275] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:46.807406  5312 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:46.808676 11885 raft_consensus.cc:2749] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:46.805838 11882 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:46.811358 11905 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:46.812098 11905 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.818778 11904 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:46.819344  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:46.828680 11856 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:46.829685  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f05b2c44da743e49df87266876b3604" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:46.830279  4748 raft_consensus.cc:3060] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:46.812659 11905 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:46.837693 11885 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.838312 11885 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:46.840889 11885 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:46.845085  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:46.845774  4680 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:46.850483  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:46.851239 11885 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:46.851630 11885 raft_consensus.cc:493] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.851999 11885 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:46.854790  4748 raft_consensus.cc:2468] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:46.856099  4647 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:46.857252 11907 raft_consensus.cc:2804] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:46.857976 11907 raft_consensus.cc:697] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:46.859192 11907 consensus_queue.cc:237] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:46.860558 11885 raft_consensus.cc:515] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:46.873473 11885 leader_election.cc:290] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:46.874682  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:27:46.875056  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 84486.4us
I20260709 12:27:46.875370  4680 raft_consensus.cc:3060] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:46.877938  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa1ebbcb7f4a7e9bcf064d4198f48e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:46.878674  5346 raft_consensus.cc:2393] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 18: Already voted for candidate dbe960c35c84451faf4e5b5c9233fa39 in this term.
I20260709 12:27:46.881217 11905 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:46.884388  4680 raft_consensus.cc:2468] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:46.884480  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85386ee6a30465188e201d5ee6ae972" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:46.890450  5312 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:46.890609  4753 raft_consensus.cc:3060] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:46.891353 11885 raft_consensus.cc:2804] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:46.891919 11885 raft_consensus.cc:697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:46.893781 11885 consensus_queue.cc:237] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:46.899979  4753 raft_consensus.cc:2468] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:46.901520  4647 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:46.904264 11907 raft_consensus.cc:2804] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:46.904820 11907 raft_consensus.cc:697] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:46.905906 11907 consensus_queue.cc:237] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:46.912928  5312 leader_election.cc:336] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:46.922895 11867 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.923602 11867 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:46.927531  5312 leader_election.cc:336] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:46.928063 11909 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.928675 11909 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:46.929032 11867 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:46.930963  4645 leader_election.cc:336] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:46.931941 11909 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:46.932155  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:46.932331  4645 consensus_peers.cc:597] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:46.953816  4720 consensus_peers.cc:597] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:46.916777 11905 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:46.960328  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:46.978415 11842 consensus_queue.cc:1048] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:46.995146 11755 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.995888 11842 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:46.996754 11842 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:46.999250 11842 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:47.010883 11842 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.000945 11839 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.005944 11885 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.008692 11755 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.011977 11839 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.016486 11905 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.027127 11905 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.029388 11905 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:47.029906 11839 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:47.031740  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:47.032356  4748 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 23.
I20260709 12:27:47.033897  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:47.027127 11885 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:47.035315 11908 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:47.035745 11908 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.036087 11908 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:47.039937  5312 leader_election.cc:336] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:47.020320  4720 leader_election.cc:336] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.011499 11842 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.059355 11842 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:47.060407 11755 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:47.060956  4496 catalog_manager.cc:5697] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 17 to 18, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 18 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:47.063910  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:47.064682  4681 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 17.
W20260709 12:27:47.074163  4645 leader_election.cc:336] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.074684  4645 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.075414 11905 raft_consensus.cc:2749] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:47.077020  4645 leader_election.cc:336] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:47.077899  4645 leader_election.cc:336] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.078291  4645 leader_election.cc:304] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:47.079034 11905 raft_consensus.cc:2749] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:47.080116  4645 leader_election.cc:336] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:47.080732  4720 leader_election.cc:336] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.086658  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.072474  4501 catalog_manager.cc:5697] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 14 to 15, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 15 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:47.088948  4720 leader_election.cc:336] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.090344 11885 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:47.089826 11839 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:47.090956 11839 raft_consensus.cc:493] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.091217  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:47.091913  4681 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
I20260709 12:27:47.093174  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.091409 11839 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:47.095428  4501 catalog_manager.cc:5697] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 14 to 15, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 15 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:47.089746  4720 leader_election.cc:336] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.096859 11885 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:47.097280 11885 raft_consensus.cc:493] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.097675 11885 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:47.090006 11908 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.104960  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:47.105641  4748 raft_consensus.cc:3060] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:47.112133 11908 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:47.119230  4748 raft_consensus.cc:2468] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 24.
I20260709 12:27:47.120829 11905 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.121699 11905 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.123975 11905 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:47.125247  4647 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:47.126763 11907 raft_consensus.cc:2804] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:47.126807  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:47.127405 11907 raft_consensus.cc:697] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 24 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:47.127616  4753 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 13.
I20260709 12:27:47.131181  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f80146cffc4af2ac0af76b3a99c213" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:47.128428 11907 consensus_queue.cc:237] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.134099  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:47.134675  4681 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 13.
I20260709 12:27:47.129302  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:47.136736  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:47.137320  4681 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:47.139074 11905 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:47.139520 11905 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.139873 11905 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:47.151166 11839 raft_consensus.cc:515] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.154781 11885 raft_consensus.cc:515] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:47.176046  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.172075 11842 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:27:47.178391  4645 leader_election.cc:336] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.179690  4645 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.178978 11842 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.182730 11842 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:47.183260 11842 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:47.189838 11839 leader_election.cc:290] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:47.198294  5312 leader_election.cc:336] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:47.201987  4645 leader_election.cc:336] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.201265 11908 raft_consensus.cc:2749] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:47.206259  4645 leader_election.cc:336] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.207484  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.207571  4645 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 5c3201048f8346009aefca07428828f3, dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:47.208871  4645 leader_election.cc:336] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.209859 11908 raft_consensus.cc:2749] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:47.215320  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f173b0d5264e4eff8b1704becc555780" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:47.216019  4674 raft_consensus.cc:3060] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:47.221705 11839 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:47.222251 11839 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.222646 11839 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:47.226949  4486 catalog_manager.cc:4755] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): ChangeConfig:REMOVE_PEER RPC failed for tablet 9702ea78030547e5a2f6fffe805075da: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.227866 11905 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.230746 11905 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:47.232326  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:47.232136 11839 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.233603  4748 raft_consensus.cc:2393] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3a5aeedd11fe49938079fe86cb01232c in current term 14: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:27:47.234835 11839 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:47.236183  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.236439 11885 leader_election.cc:290] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:47.237601 11755 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:47.241004 11755 raft_consensus.cc:493] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.241400 11755 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:47.241922  4645 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:27:47.245613  4645 leader_election.cc:336] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.247337 11908 raft_consensus.cc:2749] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:47.261011  4674 raft_consensus.cc:2468] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 18.
I20260709 12:27:47.263700  5312 leader_election.cc:304] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.264892 11885 raft_consensus.cc:2804] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:47.265630 11885 raft_consensus.cc:697] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:47.266697 11885 consensus_queue.cc:237] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:47.253201 11755 raft_consensus.cc:515] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.292237  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:47.292907  4681 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 21.
I20260709 12:27:47.290233  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d3124fc8af34f378ebd9cff21ee0e6b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:47.294205  4673 raft_consensus.cc:3060] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:47.295555  4748 raft_consensus.cc:1275] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:27:47.297399 11907 consensus_queue.cc:1048] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:27:47.303608  4720 leader_election.cc:336] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.310361  4679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
W20260709 12:27:47.311059  4645 leader_election.cc:336] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.311205  4679 raft_consensus.cc:2393] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 14: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
W20260709 12:27:47.311964  4645 leader_election.cc:336] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:47.317106  4645 leader_election.cc:336] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.317531  4645 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.318224 11907 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:27:47.318635  4645 consensus_peers.cc:597] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:47.319416  4673 raft_consensus.cc:2468] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:47.321941 11755 leader_election.cc:290] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:47.309118  4720 leader_election.cc:336] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:27:47.324232  4720 leader_election.cc:336] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:47.329867  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.330236  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fb4c89626845d8bac3ba87a5346995" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:47.331969  4673 raft_consensus.cc:3060] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:47.332739  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.334873 11913 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:47.336112  4720 leader_election.cc:304] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.333334 11839 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:47.339712  4720 connection.cc:769] client connection to 127.3.89.193:45077 send error: Network error: sendmsg error: Connection reset by peer (error 104)
I20260709 12:27:47.338980 11913 raft_consensus.cc:2804] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:47.343108 11913 raft_consensus.cc:697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:47.343890 11913 consensus_queue.cc:237] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.350821 11839 raft_consensus.cc:493] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.366225 11839 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:47.386765  4673 raft_consensus.cc:2468] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 21.
I20260709 12:27:47.389546 11885 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.390244 11885 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:47.394582  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:47.393101 11839 raft_consensus.cc:515] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.402813 11885 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:47.417064  4674 raft_consensus.cc:1275] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:27:47.419656 11839 leader_election.cc:290] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:47.421389 11880 consensus_queue.cc:1048] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:47.426481  4673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:47.440830 11867 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.441617 11867 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:47.453994  4720 leader_election.cc:336] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:27:47.488593  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 43703.6us
W20260709 12:27:47.507629  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 50228.1us
I20260709 12:27:47.521159  4720 leader_election.cc:304] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.546563 11867 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:47.575094  4645 consensus_peers.cc:597] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:47.575157  5312 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:27:47.583266 11880 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:47.584838 11931 raft_consensus.cc:2804] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:47.609661  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09d362006ea24f48a30fa120bbdf8860" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:47.610374  4680 raft_consensus.cc:3060] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:47.611352  5312 leader_election.cc:336] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:27:47.612295  5312 consensus_peers.cc:597] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:47.622967  4751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:47.614298 11931 raft_consensus.cc:697] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 21 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
W20260709 12:27:47.636245  4645 leader_election.cc:336] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:27:47.633621 11931 consensus_queue.cc:237] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.636679  4645 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.645704  4753 raft_consensus.cc:1275] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:47.655844  4501 catalog_manager.cc:5697] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 23 to 24, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 24 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:47.671573  4680 raft_consensus.cc:2468] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 22.
W20260709 12:27:47.675239  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 134553us
I20260709 12:27:47.678623 11934 consensus_queue.cc:1048] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:47.684409  4674 raft_consensus.cc:1275] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:27:47.688613  4499 catalog_manager.cc:5697] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 17 to 18, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 18 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:47.690434  4748 raft_consensus.cc:1275] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:27:47.704967 11933 consensus_queue.cc:1048] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:47.706987 11842 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.707566 11842 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.710181 11842 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:47.711982 11908 consensus_queue.cc:1048] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:47.712828 11934 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.713406 11934 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.715860 11934 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:47.718752  4748 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:47.720177  4748 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 21.
I20260709 12:27:47.670004 11938 raft_consensus.cc:2749] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:47.742780  5312 consensus_peers.cc:597] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:47.730487 11928 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.743732 11928 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.735169  4647 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:47.751232 11908 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:47.751807 11908 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.752290 11908 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:47.754232 11928 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:47.759160  4720 leader_election.cc:304] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
W20260709 12:27:47.764981  4720 leader_election.cc:336] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:27:47.768514 11842 raft_consensus.cc:2804] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:47.783763 11842 raft_consensus.cc:697] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:47.807127 11842 consensus_queue.cc:237] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.813962 11908 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.816078 11908 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:47.818933  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:47.819713  4674 raft_consensus.cc:2393] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 49277a7653524787a715232ab908bbc6 in current term 22: Already voted for candidate 3a5aeedd11fe49938079fe86cb01232c in this term.
I20260709 12:27:47.821828  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
W20260709 12:27:47.821976  4720 leader_election.cc:336] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:27:47.822552  4681 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 23.
I20260709 12:27:47.824090 11918 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.824627 11918 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:47.829391  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9a7637095341feabd700d53579d5d9" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:47.830106  4749 raft_consensus.cc:3060] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:47.830345 11918 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:27:47.834383  4720 leader_election.cc:336] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:27:47.781286  4501 catalog_manager.cc:5697] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 17 to 18, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 18 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:47.834566  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:27:47.834981  4720 consensus_peers.cc:597] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:47.835258  5346 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 18.
I20260709 12:27:47.859683  4681 raft_consensus.cc:1275] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:27:47.860658  4645 leader_election.cc:336] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:27:47.866124  4645 leader_election.cc:336] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:27:47.866704 11934 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
W20260709 12:27:47.866966  4645 consensus_peers.cc:597] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:47.867331 11934 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.856452  4496 catalog_manager.cc:5697] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 20 to 21, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 21 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:47.875514 11934 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:47.877833  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:27:47.882603  4645 leader_election.cc:336] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:27:47.883054  4645 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.884231 11934 raft_consensus.cc:2749] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:47.887269 11931 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.887878 11931 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.891352 11931 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:47.899046  4749 raft_consensus.cc:2468] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 22.
I20260709 12:27:47.901336  4680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:47.903381  4674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c" is_pre_election: true
I20260709 12:27:47.903999  4674 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 19.
I20260709 12:27:47.904781  4647 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.908547 11938 raft_consensus.cc:2804] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:47.909027 11938 raft_consensus.cc:697] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 22 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:47.909937 11938 consensus_queue.cc:237] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.912459  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.913272 11931 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:47.914615  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.925161 11931 consensus_queue.cc:1048] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:47.918710 11918 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:47.929935 11918 raft_consensus.cc:493] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.930401 11918 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:47.923939  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:47.944471 11842 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:47.947819 11842 raft_consensus.cc:493] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.948212 11842 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:47.957635 11908 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:47.958298 11908 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:47.966391  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:47.967074  4749 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:47.967151  4501 catalog_manager.cc:5697] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 21 to 22, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 22 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:47.961756 11842 raft_consensus.cc:515] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
W20260709 12:27:47.972111  4645 leader_election.cc:336] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:27:47.973783 11842 leader_election.cc:290] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:47.974213  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:27:47.974813  5346 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 18.
I20260709 12:27:47.979091 11918 raft_consensus.cc:515] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:47.979652  4644 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:47.980168 11908 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:47.983415  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee319a06cc4642fba7ae226abe36e352" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:47.990370  4681 raft_consensus.cc:3060] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:47.983518  4720 leader_election.cc:336] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:27:47.991295  4720 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:27:48.001955 11755 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:48.003508  4720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111) [suppressed 387 similar messages]
I20260709 12:27:48.003948 11934 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:48.004184  4720 leader_election.cc:336] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:27:48.004421 11934 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5c3201048f8346009aefca07428828f3)
I20260709 12:27:48.004841 11934 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:48.025954  4501 catalog_manager.cc:5697] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 21 to 22, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 22 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:48.027779 11934 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:48.034922  4681 raft_consensus.cc:2468] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 20.
I20260709 12:27:48.037029 11918 leader_election.cc:290] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 24 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:48.038337  4720 leader_election.cc:336] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:27:48.054653  4486 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 68856.1us
I20260709 12:27:48.056356  4720 leader_election.cc:304] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:48.057874 11918 raft_consensus.cc:2804] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:48.058396 11918 raft_consensus.cc:697] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:48.059262 11918 consensus_queue.cc:237] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:48.061867  4681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4e6edc86e64b8ca5e81a1f1c312757" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "3a5aeedd11fe49938079fe86cb01232c"
I20260709 12:27:48.062443  4681 raft_consensus.cc:3060] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:48.071789  4720 leader_election.cc:336] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:27:48.075018 11934 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:27:48.076090  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:27:48.076771  5346 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:48.077813  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9702ea78030547e5a2f6fffe805075da" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:48.078562  4749 raft_consensus.cc:3060] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:48.080750  4681 raft_consensus.cc:2468] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 24.
I20260709 12:27:48.082197  4720 leader_election.cc:304] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 5c3201048f8346009aefca07428828f3
W20260709 12:27:48.094195  4645 leader_election.cc:336] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:27:48.101773 11918 raft_consensus.cc:2804] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:48.104380 11918 raft_consensus.cc:697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 24 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:27:48.107415 11918 consensus_queue.cc:237] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:48.114995  5346 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:27:48.097342  4501 catalog_manager.cc:5697] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 19 to 20, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 20 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:48.134876  4680 raft_consensus.cc:1275] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:27:48.146452 11951 log.cc:927] Time spent T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:27:48.149638 11928 consensus_queue.cc:1048] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:48.149235  4749 raft_consensus.cc:2468] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 19.
I20260709 12:27:48.154326  4647 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 5c3201048f8346009aefca07428828f3
I20260709 12:27:48.155741 11940 raft_consensus.cc:2804] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:48.156276 11940 raft_consensus.cc:697] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:48.157219 11940 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:48.200347  4645 consensus_peers.cc:597] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:48.233935  4720 consensus_peers.cc:597] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:48.234581  4720 consensus_peers.cc:597] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:48.246031  4501 catalog_manager.cc:5697] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 18 to 19, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 19 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:48.253778  4496 catalog_manager.cc:5697] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 23 to 24, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 24 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:48.272675  4645 consensus_peers.cc:597] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:48.314144  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:48.344594  4679 raft_consensus.cc:1275] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:48.352402 11928 consensus_queue.cc:1048] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:48.368474  4720 consensus_peers.cc:597] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:48.378367  4749 raft_consensus.cc:1275] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:48.380861 11938 consensus_queue.cc:1048] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:27:48.416841  4645 consensus_peers.cc:597] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:48.427325  3431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 4626)
W20260709 12:27:48.530593  4645 consensus_peers.cc:597] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:48.547724  4681 raft_consensus.cc:1275] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:27:48.548709  4720 consensus_peers.cc:597] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:48.560506  4720 consensus_peers.cc:597] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:48.561769  5312 consensus_peers.cc:597] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:48.567619 11842 consensus_queue.cc:1048] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:48.567644  4645 consensus_peers.cc:597] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:48.592185  4720 consensus_peers.cc:597] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:48.638628  4674 raft_consensus.cc:1275] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 24 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:48.644711  4749 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:27:48.647930 11940 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:27:48.660357  5312 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:48.660939 11918 consensus_queue.cc:1048] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:27:48.707127  5312 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:48.750809  4720 consensus_peers.cc:597] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:48.760914  4720 consensus_peers.cc:597] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:48.785321  4645 consensus_peers.cc:597] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:48.786265  4645 consensus_peers.cc:597] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:48.788396  5345 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:27:48.817405  4720 consensus_peers.cc:597] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:48.795827 11934 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
W20260709 12:27:48.835554  4645 consensus_peers.cc:597] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:48.847715  4720 consensus_peers.cc:597] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:48.852761 11939 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.853447 11939 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:48.855916 11939 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:48.857120  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:48.857774  4749 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 21.
I20260709 12:27:48.860185  5314 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:27:48.864035  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 69586.8us
I20260709 12:27:48.861068 11939 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:48.884078  4720 consensus_peers.cc:597] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:48.878422 11939 raft_consensus.cc:493] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.914462 11939 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:48.915298  4645 consensus_peers.cc:597] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:48.952500  4720 consensus_peers.cc:597] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:48.954816  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 49737.1us
W20260709 12:27:48.964432  5312 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:48.965176  5312 leader_election.cc:336] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:27:48.971691  4720 consensus_peers.cc:597] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:48.961333 11939 raft_consensus.cc:515] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:48.979177  5312 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:48.981913  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a74137ba104fe0b465cab6399c4940" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:48.982609  4749 raft_consensus.cc:3060] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:48.990062  4749 raft_consensus.cc:2468] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 22.
I20260709 12:27:48.991529  5314 leader_election.cc:304] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:27:48.992493 11969 raft_consensus.cc:2804] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:48.993109 11969 raft_consensus.cc:697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:27:48.996248 11969 consensus_queue.cc:237] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:48.998438  4720 consensus_peers.cc:597] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:48.998560  5312 consensus_peers.cc:597] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:49.001772 11939 leader_election.cc:290] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:49.008562  5312 leader_election.cc:336] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:27:49.051142  5312 consensus_peers.cc:597] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:49.053848  4496 catalog_manager.cc:5697] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 20 to 22, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 22 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:49.060683  4645 consensus_peers.cc:597] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:49.061510  4645 consensus_peers.cc:597] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:49.097391  5312 consensus_peers.cc:597] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:49.112257  4720 consensus_peers.cc:597] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:49.133828 11934 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.134567 11934 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:49.139616  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:27:49.140343  4749 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 14.
I20260709 12:27:49.143218  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:49.149222 11940 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:49.149737 11940 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.150127 11940 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:49.157203  4720 consensus_peers.cc:597] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:49.171765  4645 leader_election.cc:336] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:27:49.178352 11934 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:49.197966  4645 consensus_peers.cc:597] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:49.254954  4720 consensus_peers.cc:597] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:49.305637 11940 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:49.341500  4680 consensus_queue.cc:237] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:49.353010  5345 raft_consensus.cc:1275] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:27:49.359637 11909 consensus_queue.cc:1048] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:49.369668 11909 raft_consensus.cc:2955] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Committing config change with OpId 19.15: config changed from index 13 to 15, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:49.380753  5344 raft_consensus.cc:2955] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 13 to 15, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:49.386281 11940 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Requested vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:49.387605  4749 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231dbfb1b78d491eb8685070c2431d82" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:27:49.406126  4749 raft_consensus.cc:3060] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:49.402092  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 58196.8us
I20260709 12:27:49.413416  4496 catalog_manager.cc:5697] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 13 to 15, VOTER 49277a7653524787a715232ab908bbc6 (127.3.89.195) evicted. New cstate: current_term: 19 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:49.429759  3431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 4626)
I20260709 12:27:49.448112  4737 tablet_service.cc:1558] Processing DeleteTablet for tablet 9702ea78030547e5a2f6fffe805075da with delete_type TABLET_DATA_TOMBSTONED (TS 49277a7653524787a715232ab908bbc6 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:50178
I20260709 12:27:49.466790 11975 tablet_replica.cc:333] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:27:49.467962 11975 raft_consensus.cc:2243] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:49.474248 11975 raft_consensus.cc:2272] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:49.483402  4749 raft_consensus.cc:2468] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5aeedd11fe49938079fe86cb01232c in term 15.
I20260709 12:27:49.484876  4647 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6; no voters: 
I20260709 12:27:49.489486  4749 raft_consensus.cc:1275] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:49.491302 11975 ts_tablet_manager.cc:1916] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:27:49.491914 11938 raft_consensus.cc:2804] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:49.492654 11938 raft_consensus.cc:697] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Becoming Leader. State: Replica: 3a5aeedd11fe49938079fe86cb01232c, State: Running, Role: LEADER
I20260709 12:27:49.493741 11938 consensus_queue.cc:237] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:27:49.497494  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9702ea78030547e5a2f6fffe805075da with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
I20260709 12:27:49.497965 11933 consensus_queue.cc:1048] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:27:49.501097  4645 consensus_peers.cc:597] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:49.501811  4645 consensus_peers.cc:597] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:49.502369  4645 leader_election.cc:336] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:27:49.529845  5312 consensus_peers.cc:597] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:49.541111  4496 catalog_manager.cc:5697] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: term changed from 13 to 15, leader changed from 5c3201048f8346009aefca07428828f3 (127.3.89.193) to 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194). New cstate: current_term: 15 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:49.554575  4720 consensus_peers.cc:597] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:49.598773 11975 ts_tablet_manager.cc:1929] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.14
I20260709 12:27:49.601058 11975 log.cc:1199] T 9702ea78030547e5a2f6fffe805075da P 49277a7653524787a715232ab908bbc6: Deleting WAL directory at /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599794910850-3431-0/minicluster-data/ts-2-root/wals/9702ea78030547e5a2f6fffe805075da
I20260709 12:27:49.606937  4488 catalog_manager.cc:5028] TS 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): tablet 9702ea78030547e5a2f6fffe805075da (table test-workload [id=6849b42b5cf7481bb881dcc0034d2e81]) successfully deleted
W20260709 12:27:49.802233  4720 consensus_peers.cc:597] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:49.820150  5312 consensus_peers.cc:597] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:49.869716  4645 consensus_peers.cc:597] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:49.895267  4645 consensus_peers.cc:597] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:49.984969  4645 consensus_peers.cc:597] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:50.071964  4749 raft_consensus.cc:1275] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:50.074010 11938 consensus_queue.cc:1048] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:27:50.241240  5312 consensus_peers.cc:597] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:50.305003  4645 consensus_peers.cc:597] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:50.340087  4720 consensus_peers.cc:597] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:50.343700  4486 catalog_manager.cc:4755] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet aba6aa625d5a49658718de09523b8a9b: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:27:50.430321  3431 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 4626)
W20260709 12:27:50.533460  4720 consensus_peers.cc:597] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:50.629452  4645 consensus_peers.cc:597] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:50.633488  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:50.652659  4720 consensus_peers.cc:597] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:50.722501  4645 consensus_peers.cc:597] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:50.841878  4674 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
W20260709 12:27:50.871466  4645 consensus_peers.cc:597] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:50.874866  4749 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:27:50.876793  4644 consensus_peers.cc:597] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 86297ef907dd4e3590805d3cf15efaca. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:50.876914 11940 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:27:50.883556  4645 consensus_peers.cc:597] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:50.886183  4720 consensus_peers.cc:597] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:50.909698 11938 raft_consensus.cc:2955] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index -1 to 11, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:50.911280  4748 raft_consensus.cc:2955] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index -1 to 11, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
W20260709 12:27:50.919853  4645 consensus_peers.cc:597] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:50.947098  4496 catalog_manager.cc:5697] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 11, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 15 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:50.961549  4748 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
W20260709 12:27:50.985459  4719 consensus_peers.cc:597] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f54c73ae37134549b12188dddc4fa794. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:50.990520  4674 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:27:50.992575  4720 consensus_peers.cc:597] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:50.995656 11977 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:27:50.999240  4720 consensus_peers.cc:597] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:50.999680  4720 consensus_peers.cc:597] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:51.023175 11931 raft_consensus.cc:2955] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:51.035641  4674 raft_consensus.cc:2955] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
W20260709 12:27:51.044289  5312 consensus_peers.cc:597] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:51.048889  4681 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:51.061264  4496 catalog_manager.cc:5697] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index -1 to 12, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 18 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:51.085508  4720 consensus_peers.cc:597] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:51.135948  4644 consensus_peers.cc:597] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 13ebb8f343c743a1a28db116a4881004. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:51.137560  4749 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:51.139169 11985 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:27:51.143648  4645 consensus_peers.cc:597] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:51.166007  4720 consensus_peers.cc:597] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:51.178850 11940 raft_consensus.cc:2955] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:51.180943  4753 raft_consensus.cc:2955] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
W20260709 12:27:51.197528  4720 consensus_peers.cc:597] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:51.208168  4749 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:51.211246  4496 catalog_manager.cc:5697] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index -1 to 12, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 19 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:51.251345  4719 consensus_peers.cc:597] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 -> Peer dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067): Couldn't send request to peer dbe960c35c84451faf4e5b5c9233fa39. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 485fe0cee4b248f4802d3c15dc97a877. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:51.269227  4681 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:27:51.278595  5312 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:51.281894 11931 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:27:51.285759  4720 consensus_peers.cc:597] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:51.292480  4645 consensus_peers.cc:597] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:51.293820  5312 consensus_peers.cc:597] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:51.299459  4720 consensus_peers.cc:597] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:51.306260 12003 ts_tablet_manager.cc:933] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:27:51.309962  4645 consensus_peers.cc:597] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:51.320653  4720 consensus_peers.cc:597] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:51.322175  4720 consensus_peers.cc:597] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:51.343441  4645 consensus_peers.cc:597] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:51.353978  5312 consensus_peers.cc:597] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:51.365028 12003 tablet_copy_client.cc:323] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.194:42599
I20260709 12:27:51.383424  4691 tablet_copy_service.cc:140] P 3a5aeedd11fe49938079fe86cb01232c: Received BeginTabletCopySession request for tablet 86297ef907dd4e3590805d3cf15efaca from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:48198)
I20260709 12:27:51.384021  4691 tablet_copy_service.cc:161] P 3a5aeedd11fe49938079fe86cb01232c: Beginning new tablet copy session on tablet 86297ef907dd4e3590805d3cf15efaca from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:48198: session id = dbe960c35c84451faf4e5b5c9233fa39-86297ef907dd4e3590805d3cf15efaca
W20260709 12:27:51.394650  5312 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:51.395161  5312 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:51.396468  4691 tablet_copy_source_session.cc:215] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:51.403575 12003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86297ef907dd4e3590805d3cf15efaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:51.406042 11994 raft_consensus.cc:2955] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 21 LEADER]: Committing config change with OpId 21.12: config changed from index -1 to 12, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:51.412688  4674 raft_consensus.cc:2955] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Committing config change with OpId 21.12: config changed from index -1 to 12, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:51.428876 12003 tablet_copy_client.cc:806] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
W20260709 12:27:51.430851  3431 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 4626)
I20260709 12:27:51.448729 12003 tablet_copy_client.cc:670] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
W20260709 12:27:51.443902  5312 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:51.461417  4496 catalog_manager.cc:5697] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index -1 to 12, NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) added. New cstate: current_term: 21 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } }
I20260709 12:27:51.467859 12003 tablet_copy_client.cc:538] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:27:51.496121  4720 consensus_peers.cc:597] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:51.496732 12003 tablet_bootstrap.cc:492] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:27:51.511430 12008 ts_tablet_manager.cc:933] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:27:51.519279  4720 consensus_peers.cc:597] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:51.538717 12008 tablet_copy_client.cc:323] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.195:41049
I20260709 12:27:51.541209  4766 tablet_copy_service.cc:140] P 49277a7653524787a715232ab908bbc6: Received BeginTabletCopySession request for tablet f54c73ae37134549b12188dddc4fa794 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:42632)
I20260709 12:27:51.541988  4766 tablet_copy_service.cc:161] P 49277a7653524787a715232ab908bbc6: Beginning new tablet copy session on tablet f54c73ae37134549b12188dddc4fa794 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:42632: session id = dbe960c35c84451faf4e5b5c9233fa39-f54c73ae37134549b12188dddc4fa794
I20260709 12:27:51.552455  4766 tablet_copy_source_session.cc:215] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:27:51.554214  4720 consensus_peers.cc:597] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:51.593744  4720 consensus_peers.cc:597] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:51.594817 12011 ts_tablet_manager.cc:933] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:27:51.607793 12008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54c73ae37134549b12188dddc4fa794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:51.614028 12011 tablet_copy_client.cc:323] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.194:42599
W20260709 12:27:51.614421  4645 consensus_peers.cc:597] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:51.615922  4691 tablet_copy_service.cc:140] P 3a5aeedd11fe49938079fe86cb01232c: Received BeginTabletCopySession request for tablet 13ebb8f343c743a1a28db116a4881004 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:48198)
I20260709 12:27:51.616532  4691 tablet_copy_service.cc:161] P 3a5aeedd11fe49938079fe86cb01232c: Beginning new tablet copy session on tablet 13ebb8f343c743a1a28db116a4881004 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:48198: session id = dbe960c35c84451faf4e5b5c9233fa39-13ebb8f343c743a1a28db116a4881004
I20260709 12:27:51.625988  4691 tablet_copy_source_session.cc:215] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:51.633466 12011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ebb8f343c743a1a28db116a4881004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:51.673062 12003 tablet_bootstrap.cc:492] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=1 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:51.688643 12008 tablet_copy_client.cc:806] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:51.689373 12008 tablet_copy_client.cc:670] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:51.691440 12011 tablet_copy_client.cc:806] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:51.692016 12011 tablet_copy_client.cc:670] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:51.735586 12011 tablet_copy_client.cc:538] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:27:51.745541  5312 consensus_peers.cc:597] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:51.751075  4720 consensus_peers.cc:597] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:51.760013  4645 consensus_peers.cc:597] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:51.777714 12008 tablet_copy_client.cc:538] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:27:51.780659 12011 tablet_bootstrap.cc:492] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
W20260709 12:27:51.794589  4645 consensus_peers.cc:597] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:51.818044 12003 tablet_bootstrap.cc:492] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:27:51.818936 12003 ts_tablet_manager.cc:1403] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.323s	user 0.042s	sys 0.009s
I20260709 12:27:51.833671 12013 ts_tablet_manager.cc:933] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39: Initiating tablet copy from peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:27:51.850023 12013 tablet_copy_client.cc:323] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.195:41049
I20260709 12:27:51.853425  4766 tablet_copy_service.cc:140] P 49277a7653524787a715232ab908bbc6: Received BeginTabletCopySession request for tablet 485fe0cee4b248f4802d3c15dc97a877 from peer dbe960c35c84451faf4e5b5c9233fa39 ({username='slave'} at 127.0.0.1:42632)
I20260709 12:27:51.854074  4766 tablet_copy_service.cc:161] P 49277a7653524787a715232ab908bbc6: Beginning new tablet copy session on tablet 485fe0cee4b248f4802d3c15dc97a877 from peer dbe960c35c84451faf4e5b5c9233fa39 at {username='slave'} at 127.0.0.1:42632: session id = dbe960c35c84451faf4e5b5c9233fa39-485fe0cee4b248f4802d3c15dc97a877
I20260709 12:27:51.861196 12008 tablet_bootstrap.cc:492] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:27:51.871063 12003 raft_consensus.cc:359] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:51.877893 12003 raft_consensus.cc:740] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
W20260709 12:27:51.876315  4645 consensus_peers.cc:597] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:51.879828  4645 consensus_peers.cc:597] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:51.883462  4766 tablet_copy_source_session.cc:215] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:51.900713 12013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 485fe0cee4b248f4802d3c15dc97a877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:51.878737 12003 consensus_queue.cc:260] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:51.904561 12003 ts_tablet_manager.cc:1434] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.085s	user 0.003s	sys 0.005s
W20260709 12:27:51.906720  5312 consensus_peers.cc:597] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:51.910996  4691 tablet_copy_service.cc:342] P 3a5aeedd11fe49938079fe86cb01232c: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-86297ef907dd4e3590805d3cf15efaca received from {username='slave'} at 127.0.0.1:48198
I20260709 12:27:51.911515  4691 tablet_copy_service.cc:434] P 3a5aeedd11fe49938079fe86cb01232c: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-86297ef907dd4e3590805d3cf15efaca on tablet 86297ef907dd4e3590805d3cf15efaca with peer dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:51.935945  4720 consensus_peers.cc:597] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:52.001741 12013 tablet_copy_client.cc:806] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:52.002707 12013 tablet_copy_client.cc:670] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:52.025271 12013 tablet_copy_client.cc:538] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:27:52.085836 12013 tablet_bootstrap.cc:492] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap starting.
I20260709 12:27:52.086512 12011 tablet_bootstrap.cc:492] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:52.098450 12011 tablet_bootstrap.cc:492] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:27:52.099308 12011 ts_tablet_manager.cc:1403] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.319s	user 0.041s	sys 0.015s
I20260709 12:27:52.100980 12008 tablet_bootstrap.cc:492] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=3 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:52.106016 12008 tablet_bootstrap.cc:492] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:27:52.106940 12008 ts_tablet_manager.cc:1403] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.274s	user 0.036s	sys 0.014s
I20260709 12:27:52.104090 12011 raft_consensus.cc:359] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:52.108505 12011 raft_consensus.cc:740] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:27:52.109253 12011 consensus_queue.cc:260] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:52.114406 12011 ts_tablet_manager.cc:1434] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 12:27:52.147426  4691 tablet_copy_service.cc:342] P 3a5aeedd11fe49938079fe86cb01232c: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-13ebb8f343c743a1a28db116a4881004 received from {username='slave'} at 127.0.0.1:48198
I20260709 12:27:52.147893  4691 tablet_copy_service.cc:434] P 3a5aeedd11fe49938079fe86cb01232c: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-13ebb8f343c743a1a28db116a4881004 on tablet 13ebb8f343c743a1a28db116a4881004 with peer dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:27:52.158211 12008 raft_consensus.cc:359] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:52.166538 12008 raft_consensus.cc:740] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:27:52.172305 12008 consensus_queue.cc:260] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:52.180317  5345 raft_consensus.cc:1217] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.10->[15.11-15.11]   Dedup: 15.11->[]
W20260709 12:27:52.182667  4645 consensus_peers.cc:597] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:52.230306 12008 ts_tablet_manager.cc:1434] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.123s	user 0.007s	sys 0.002s
I20260709 12:27:52.232532  4766 tablet_copy_service.cc:342] P 49277a7653524787a715232ab908bbc6: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-f54c73ae37134549b12188dddc4fa794 received from {username='slave'} at 127.0.0.1:42632
I20260709 12:27:52.233089  4766 tablet_copy_service.cc:434] P 49277a7653524787a715232ab908bbc6: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-f54c73ae37134549b12188dddc4fa794 on tablet f54c73ae37134549b12188dddc4fa794 with peer dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:52.347187  4720 consensus_peers.cc:597] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:27:52.358441  5344 raft_consensus.cc:1217] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEARNER]: Deduplicated request from leader. Original: 18.11->[18.12-18.12]   Dedup: 18.12->[]
W20260709 12:27:52.448871  3431 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 4626)
I20260709 12:27:52.501163 12013 tablet_bootstrap.cc:492] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:52.508124 12013 tablet_bootstrap.cc:492] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39: Bootstrap complete.
I20260709 12:27:52.508948 12013 ts_tablet_manager.cc:1403] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39: Time spent bootstrapping tablet: real 0.423s	user 0.043s	sys 0.005s
I20260709 12:27:52.516736 12013 raft_consensus.cc:359] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:52.520429 12013 raft_consensus.cc:740] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Initialized, Role: LEARNER
I20260709 12:27:52.524001 12013 consensus_queue.cc:260] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } }
I20260709 12:27:52.537504 12013 ts_tablet_manager.cc:1434] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:27:52.544143  4766 tablet_copy_service.cc:342] P 49277a7653524787a715232ab908bbc6: Request end of tablet copy session dbe960c35c84451faf4e5b5c9233fa39-485fe0cee4b248f4802d3c15dc97a877 received from {username='slave'} at 127.0.0.1:42632
I20260709 12:27:52.544589  4766 tablet_copy_service.cc:434] P 49277a7653524787a715232ab908bbc6: ending tablet copy session dbe960c35c84451faf4e5b5c9233fa39-485fe0cee4b248f4802d3c15dc97a877 on tablet 485fe0cee4b248f4802d3c15dc97a877 with peer dbe960c35c84451faf4e5b5c9233fa39
W20260709 12:27:52.555320  4645 consensus_peers.cc:597] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:52.585557  5344 raft_consensus.cc:1217] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 LEARNER]: Deduplicated request from leader. Original: 19.11->[19.12-19.12]   Dedup: 19.12->[]
W20260709 12:27:52.616580  4645 consensus_peers.cc:597] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:52.682204  5312 consensus_peers.cc:597] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:52.696856  4720 consensus_peers.cc:597] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:52.713624 11940 raft_consensus.cc:1064] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:27:52.723510  5344 raft_consensus.cc:1217] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 LEARNER]: Deduplicated request from leader. Original: 21.11->[21.12-21.12]   Dedup: 21.12->[]
I20260709 12:27:52.717104 11940 consensus_queue.cc:237] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:52.741990  5312 consensus_peers.cc:597] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:52.764824  5344 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [term 15 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:52.766631 12006 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:52.776046  4749 raft_consensus.cc:1275] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:52.779063 11985 consensus_queue.cc:1048] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:52.807792 12006 raft_consensus.cc:2955] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:52.811023  4753 raft_consensus.cc:2955] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:52.849035  4645 consensus_peers.cc:597] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:52.852535  4498 catalog_manager.cc:5697] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 reported cstate change: config changed from index 11 to 12, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:52.863554  5344 raft_consensus.cc:2955] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:52.873715  4645 consensus_peers.cc:597] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:52.961674 11931 raft_consensus.cc:1064] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:27:52.975953 12007 raft_consensus.cc:1064] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:27:52.976621 11931 consensus_queue.cc:237] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:52.978906 12007 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:27:53.003834  4720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111) [suppressed 494 similar messages]
W20260709 12:27:53.006808  4720 consensus_peers.cc:597] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:53.022145  4753 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:53.024518 12017 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:53.029910  5345 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 LEARNER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:53.043324 12007 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:53.045729  5345 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:53.047297  4681 raft_consensus.cc:1275] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:27:53.048102  4645 consensus_peers.cc:597] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.049644 11931 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:53.062198  4720 consensus_peers.cc:597] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.062855 11931 consensus_queue.cc:1048] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:27:53.064152 12026 log.cc:927] Time spent T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:53.104199 11931 raft_consensus.cc:2955] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:53.106104  5346 raft_consensus.cc:2955] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:53.121176  4681 raft_consensus.cc:2955] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:53.124125  4496 catalog_manager.cc:5697] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:53.135568 12017 raft_consensus.cc:2955] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:53.151733  5344 raft_consensus.cc:2955] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:53.153585 11994 raft_consensus.cc:1064] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6: attempting to promote NON_VOTER dbe960c35c84451faf4e5b5c9233fa39 to VOTER
I20260709 12:27:53.160813 11994 consensus_queue.cc:237] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 10, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:53.180198  5346 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 LEARNER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:27:53.185130  4720 consensus_peers.cc:597] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:53.185245  4674 raft_consensus.cc:1275] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:27:53.186234  4645 consensus_peers.cc:597] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:53.186424 11994 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:53.190706 11994 consensus_queue.cc:1048] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:53.208781  4501 catalog_manager.cc:5697] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:53.232386  4749 raft_consensus.cc:2955] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } attrs { replace: true } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:53.281553  4720 consensus_peers.cc:597] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:27:53.348090  4674 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:27:53.377018  5346 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:27:53.363119  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:53.367583 11994 raft_consensus.cc:2955] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 21 LEADER]: Committing config change with OpId 21.13: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:53.378724 12017 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:53.386529 12007 raft_consensus.cc:2955] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:53.390311  5346 raft_consensus.cc:2955] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:53.391743  4753 raft_consensus.cc:1275] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Refusing update from remote peer 3a5aeedd11fe49938079fe86cb01232c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:27:53.393349  4645 consensus_peers.cc:597] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:27:53.394158 11985 consensus_queue.cc:1048] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:53.418018  4486 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 13ebb8f343c743a1a28db116a4881004 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:27:53.420637  4681 raft_consensus.cc:2955] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:53.431057  5346 raft_consensus.cc:2955] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
I20260709 12:27:53.432451  4496 catalog_manager.cc:5697] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: config changed from index 13 to 14, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New cstate: current_term: 19 leader_uuid: "3a5aeedd11fe49938079fe86cb01232c" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:53.449376  3431 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 4626)
I20260709 12:27:53.405019  4749 raft_consensus.cc:2955] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 5c3201048f8346009aefca07428828f3 (127.3.89.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:53.466845  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 46064.4us
I20260709 12:27:53.466288  4496 catalog_manager.cc:5697] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c reported cstate change: config changed from index 12 to 13, dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:53.486289  4720 consensus_peers.cc:597] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:53.499594  4486 catalog_manager.cc:4755] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 13ebb8f343c743a1a28db116a4881004: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:27:53.520509  4720 consensus_peers.cc:597] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.584618  5312 consensus_peers.cc:597] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.589371  4645 consensus_peers.cc:597] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.594133  5312 consensus_peers.cc:597] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.607393  4720 consensus_peers.cc:597] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.673040  4645 consensus_peers.cc:597] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.696588  4645 consensus_peers.cc:597] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.700181  5312 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.722478  4720 consensus_peers.cc:597] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.750116  4720 consensus_peers.cc:597] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:53.764523  5312 consensus_peers.cc:597] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.813393  4720 consensus_peers.cc:597] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.861151  4645 consensus_peers.cc:597] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.866488  4720 consensus_peers.cc:597] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.871371  5312 consensus_peers.cc:597] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.888943  5312 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.962630  4645 consensus_peers.cc:597] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.967262  5312 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:53.985585  5312 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:54.065814  4720 consensus_peers.cc:597] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:54.073513  4645 consensus_peers.cc:597] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:54.074445  4720 consensus_peers.cc:597] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:54.127676  4645 consensus_peers.cc:597] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:54.134155  4720 consensus_peers.cc:597] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:54.165776  4720 consensus_peers.cc:597] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:54.170531  5312 consensus_peers.cc:597] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:54.171429  4645 consensus_peers.cc:597] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:54.192247  4720 consensus_peers.cc:597] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:54.203547  4720 consensus_peers.cc:597] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:54.203962  4645 consensus_peers.cc:597] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:54.449873  3431 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 4626)
W20260709 12:27:54.511209  4486 catalog_manager.cc:4755] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2d6e5f40333a47559cd852e4dfa7c86b: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:27:54.539104  4720 consensus_peers.cc:597] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:54.636382  4645 consensus_peers.cc:597] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:54.759119  4645 consensus_peers.cc:597] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:54.983490  4488 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 3be428c36f9242c6a2b8aa853f2c8b2c with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 49277a7653524787a715232ab908bbc6 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 19 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } } }
W20260709 12:27:54.993639  4645 consensus_peers.cc:597] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:55.085315  4720 consensus_peers.cc:597] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:55.162266  5312 consensus_peers.cc:597] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:55.198524  5312 consensus_peers.cc:597] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:55.214529  4720 consensus_peers.cc:597] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:55.450501  3431 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 4626)
W20260709 12:27:55.494133  4720 consensus_peers.cc:597] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:55.515244  4720 consensus_peers.cc:597] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:55.750491  4720 consensus_peers.cc:597] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:55.836826  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:55.843279  4720 consensus_peers.cc:597] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:55.845209  4486 catalog_manager.cc:4755] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 13ebb8f343c743a1a28db116a4881004: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:27:55.907290  4720 consensus_peers.cc:597] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.058552  5312 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.059947  5312 consensus_peers.cc:597] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.090502  5312 consensus_peers.cc:597] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.102169  4720 consensus_peers.cc:597] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.113649  4720 consensus_peers.cc:597] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:56.125145  4645 consensus_peers.cc:597] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:56.126878  4645 consensus_peers.cc:597] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:56.154385  4645 consensus_peers.cc:597] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:56.182416  4720 consensus_peers.cc:597] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.297931  4645 consensus_peers.cc:597] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.313078  4720 consensus_peers.cc:597] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.375628  4720 consensus_peers.cc:597] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:56.380256  5312 consensus_peers.cc:597] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.385779  4720 consensus_peers.cc:597] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.388600  5312 consensus_peers.cc:597] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.448115  4720 consensus_peers.cc:597] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:56.451248  3431 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 4626)
W20260709 12:27:56.513013  5312 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.522665  4720 consensus_peers.cc:597] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.527738  4720 consensus_peers.cc:597] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.571522  4645 consensus_peers.cc:597] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.573892  5312 consensus_peers.cc:597] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:56.618851  4702 debug-util.cc:398] Leaking SignalData structure 0x7b0800338d00 after lost signal to thread 3434
W20260709 12:27:56.621074  4702 debug-util.cc:398] Leaking SignalData structure 0x7b08002e4200 after lost signal to thread 4536
W20260709 12:27:56.629797  4702 debug-util.cc:398] Leaking SignalData structure 0x7b080016b480 after lost signal to thread 4706
W20260709 12:27:56.631688  4702 debug-util.cc:398] Leaking SignalData structure 0x7b080016b0e0 after lost signal to thread 4780
W20260709 12:27:56.638592  4702 debug-util.cc:398] Leaking SignalData structure 0x7b08003b7ca0 after lost signal to thread 5373
W20260709 12:27:56.702152  4720 consensus_peers.cc:597] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.708596  4720 consensus_peers.cc:597] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.752208  5312 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.778048  4720 consensus_peers.cc:597] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.834659  4645 consensus_peers.cc:597] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.945688  4645 consensus_peers.cc:597] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:56.969784  5312 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:57.040606  4645 consensus_peers.cc:597] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:57.264719  4645 consensus_peers.cc:597] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:57.332119  4645 consensus_peers.cc:597] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:57.355872  4720 consensus_peers.cc:597] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:57.358785  4720 consensus_peers.cc:597] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:57.451830  3431 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 4626)
W20260709 12:27:57.549225  4645 consensus_peers.cc:597] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:57.551610  4645 consensus_peers.cc:597] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:57.726953  5312 consensus_peers.cc:597] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:57.740033  4645 consensus_peers.cc:597] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:57.750654  5312 consensus_peers.cc:597] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:57.752544  4645 consensus_peers.cc:597] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:57.788995  4720 consensus_peers.cc:597] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:57.984784  4720 consensus_peers.cc:597] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:58.004667  4645 consensus_peers.cc:597] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:27:58.012717  4486 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 44338.7us
W20260709 12:27:58.020071  4486 catalog_manager.cc:4755] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 13ebb8f343c743a1a28db116a4881004: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:27:58.038376  4720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111) [suppressed 467 similar messages]
W20260709 12:27:58.199749  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:58.266274  4645 consensus_peers.cc:597] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:58.312182  4720 consensus_peers.cc:597] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:58.385856  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 73714.9us
W20260709 12:27:58.389860  4720 consensus_peers.cc:597] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:58.432379  4645 consensus_peers.cc:597] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:27:58.441414  4720 consensus_peers.cc:597] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:58.452358  3431 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 4626)
W20260709 12:27:58.464254  4720 consensus_peers.cc:597] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:58.479041  4645 consensus_peers.cc:597] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:58.534200  4720 consensus_peers.cc:597] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:58.566351  5312 consensus_peers.cc:597] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:58.580744  4720 consensus_peers.cc:597] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:58.622762  5312 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:58.648083  4645 consensus_peers.cc:597] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:58.656713  5312 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:58.666258  4645 consensus_peers.cc:597] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:58.678469  4720 consensus_peers.cc:597] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:58.800405  5312 consensus_peers.cc:597] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:58.815549  4720 consensus_peers.cc:597] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:58.832293  5312 consensus_peers.cc:597] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:58.859604  4720 consensus_peers.cc:597] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:58.954869  4720 consensus_peers.cc:597] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:27:58.991312  4720 consensus_peers.cc:597] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.001228  4720 consensus_peers.cc:597] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.002554  5312 consensus_peers.cc:597] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.145691  5312 consensus_peers.cc:597] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:59.146361  4720 consensus_peers.cc:597] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:27:59.169782  4720 consensus_peers.cc:597] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.171197  4720 consensus_peers.cc:597] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.189033  4720 consensus_peers.cc:597] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.197502  4645 consensus_peers.cc:597] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.243294  5312 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.300004  4645 consensus_peers.cc:597] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.308458  4645 consensus_peers.cc:597] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.452931  3431 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 4626)
W20260709 12:27:59.614434  5312 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.837646  4645 consensus_peers.cc:597] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.848443  4720 consensus_peers.cc:597] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.919355  4645 consensus_peers.cc:597] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:59.965835  4645 consensus_peers.cc:597] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:00.045466  4720 consensus_peers.cc:597] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:00.112710  5312 consensus_peers.cc:597] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:00.170585  5312 consensus_peers.cc:597] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:00.211814  4645 consensus_peers.cc:597] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:00.212658  4645 consensus_peers.cc:597] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:00.304378  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 55633.6us
W20260709 12:28:00.371803  4720 consensus_peers.cc:597] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:00.399959  4645 consensus_peers.cc:597] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:00.406165  4645 consensus_peers.cc:597] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:00.426575  4645 consensus_peers.cc:597] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:00.453516  3431 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 4626)
W20260709 12:28:00.481799  4720 consensus_peers.cc:597] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:00.571609  4645 consensus_peers.cc:597] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:00.573776  4720 consensus_peers.cc:597] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:00.674398  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:00.725812  4720 consensus_peers.cc:597] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:00.906037  4720 consensus_peers.cc:597] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:00.935863  4645 consensus_peers.cc:597] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:00.944005  4720 consensus_peers.cc:597] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:00.960737  4645 consensus_peers.cc:597] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:01.020866  5312 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.022158  4720 consensus_peers.cc:597] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.081228  5312 consensus_peers.cc:597] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.117714  4645 consensus_peers.cc:597] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.149072  4720 consensus_peers.cc:597] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.216394  5312 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.273140  4720 consensus_peers.cc:597] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:01.279462  4645 consensus_peers.cc:597] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:01.443723  4720 consensus_peers.cc:597] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.530222  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 62093.5us
W20260709 12:28:01.563723  4720 consensus_peers.cc:597] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.564524  4720 consensus_peers.cc:597] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.565083  4720 consensus_peers.cc:597] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:01.565583  4720 consensus_peers.cc:597] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.658370  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 89692.1us
W20260709 12:28:01.676618  5312 consensus_peers.cc:597] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.681619  5312 consensus_peers.cc:597] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:01.684022  4645 consensus_peers.cc:597] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.711282  5312 consensus_peers.cc:597] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.713446  4720 consensus_peers.cc:597] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.750042  4720 consensus_peers.cc:597] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:01.762809  4720 consensus_peers.cc:597] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.835687  5312 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.853380  4720 consensus_peers.cc:597] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:01.882643  5312 consensus_peers.cc:597] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:02.000149  4645 consensus_peers.cc:597] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:02.021140  4645 consensus_peers.cc:597] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:02.043241  3431 tablet_server.cc:196] TabletServer@127.3.89.193:0 shutdown complete.
W20260709 12:28:02.049506  5312 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:02.280098  4645 consensus_peers.cc:597] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:02.296329  4486 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 78484.6us
W20260709 12:28:02.311054  4645 consensus_peers.cc:597] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:02.311162  4486 catalog_manager.cc:4755] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 13ebb8f343c743a1a28db116a4881004: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:02.323598  4645 consensus_peers.cc:597] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:02.406492  3431 tablet_server.cc:179] TabletServer@127.3.89.194:0 shutting down...
W20260709 12:28:02.409364  4720 consensus_peers.cc:597] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:02.505298  5312 consensus_peers.cc:597] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.538741  5312 consensus_peers.cc:597] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.543413  5312 consensus_peers.cc:597] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.545157  4720 consensus_peers.cc:597] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.550848  4720 consensus_peers.cc:597] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.588584  4645 consensus_peers.cc:597] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:02.594908  4720 consensus_peers.cc:597] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.596076  5312 consensus_peers.cc:597] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:02.598368  4720 consensus_peers.cc:597] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.611347  4720 consensus_peers.cc:597] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.628232  5312 consensus_peers.cc:597] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.635893  4720 consensus_peers.cc:597] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.648267  4720 consensus_peers.cc:597] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.648329  5312 consensus_peers.cc:597] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.648994  4720 consensus_peers.cc:597] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.653826  5312 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.659428  4720 consensus_peers.cc:597] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.666833  5312 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.710569  4720 consensus_peers.cc:597] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:02.713287  4720 consensus_peers.cc:597] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.717624  4720 consensus_peers.cc:597] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.731057  5312 consensus_peers.cc:597] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:02.739682  4720 consensus_peers.cc:597] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.741707  4644 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:39067, user_credentials={real_user=slave}} blocked reactor thread for 87655.6us
W20260709 12:28:02.742050  5312 consensus_peers.cc:597] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.778156  4720 consensus_peers.cc:597] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.780851  4720 consensus_peers.cc:597] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.781718  4720 consensus_peers.cc:597] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.782313  4720 consensus_peers.cc:597] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.805619  5312 consensus_peers.cc:597] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.818621  4720 consensus_peers.cc:597] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:02.833772  4645 consensus_peers.cc:597] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:02.862123  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:02.867072  3431 tablet_replica.cc:333] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:02.868379  3431 raft_consensus.cc:2243] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:02.869514  3431 raft_consensus.cc:2272] T 9ced730aeaf240b69d68fd0ee470d68d P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:02.889020  3431 tablet_replica.cc:333] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:02.890076  3431 raft_consensus.cc:2243] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Raft consensus shutting down.
W20260709 12:28:02.890969  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:02.891518  3431 raft_consensus.cc:2272] T ae7ffb2806e441e787796287d56e8b8a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:02.894361  4645 consensus_peers.cc:597] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:02.904490  4720 consensus_peers.cc:597] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:02.921523  3431 tablet_replica.cc:333] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:02.923133  3431 raft_consensus.cc:2243] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:28:02.924422  3431 raft_consensus.cc:2272] T 28f87749718d445ba1615ff33028c4d8 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:02.926258  4720 consensus_peers.cc:597] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:02.928097  4720 consensus_peers.cc:597] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.932230  4645 consensus_peers.cc:597] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:02.940296  4645 consensus_peers.cc:597] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:02.970532  5312 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:02.983930  3431 tablet_replica.cc:333] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:02.985049  3431 raft_consensus.cc:2243] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:02.986212  3431 raft_consensus.cc:2272] T b7cd71e05324451f8e6dfcf6c2e76829 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:02.989465  3431 tablet_replica.cc:333] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.021000  3431 raft_consensus.cc:2243] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.023896  3431 raft_consensus.cc:2272] T 485fe0cee4b248f4802d3c15dc97a877 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.033161  3431 tablet_replica.cc:333] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.034183  3431 raft_consensus.cc:2243] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.035444  3431 raft_consensus.cc:2272] T 2d6e5f40333a47559cd852e4dfa7c86b P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:03.040052  5312 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 512 similar messages]
I20260709 12:28:03.049619  3431 tablet_replica.cc:333] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.050678  3431 raft_consensus.cc:2243] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.051388  3431 raft_consensus.cc:2272] T a8ee74acbd5c45f0934f6f37d2154bc7 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.054520  3431 tablet_replica.cc:333] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.055435  3431 raft_consensus.cc:2243] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.056689  3431 raft_consensus.cc:2272] T 8f05b2c44da743e49df87266876b3604 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.094024  3431 tablet_replica.cc:333] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.095274  3431 raft_consensus.cc:2243] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.096478  3431 raft_consensus.cc:2272] T e85386ee6a30465188e201d5ee6ae972 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.102041  3431 tablet_replica.cc:333] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.102948  3431 raft_consensus.cc:2243] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.103750  3431 raft_consensus.cc:2272] T 3cbe5bef0f514b01ac552ff54d4d13be P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:03.172286  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 51672.6us
I20260709 12:28:03.175504  3431 tablet_replica.cc:333] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.176523  3431 raft_consensus.cc:2243] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.177217  3431 raft_consensus.cc:2272] T 42d36b6bb6e84733876824d95e25967e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.207423  3431 tablet_replica.cc:333] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.210270  3431 raft_consensus.cc:2243] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.220947  3431 raft_consensus.cc:2272] T 53908897ce974579947af90acaf0aed7 P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.237596  3431 tablet_replica.cc:333] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.238425  3431 raft_consensus.cc:2243] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.239142  3431 raft_consensus.cc:2272] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.242648  3431 tablet_replica.cc:333] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.243592  3431 raft_consensus.cc:2243] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.258001  3431 raft_consensus.cc:2272] T b6e604992a304031ad5cbdc32d90edb4 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.260775  3431 tablet_replica.cc:333] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.261637  3431 raft_consensus.cc:2243] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.262308  3431 raft_consensus.cc:2272] T 4163edf182b549dfb724d2e5b5b1ac0e P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.265167  3431 tablet_replica.cc:333] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.266028  3431 raft_consensus.cc:2243] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.267120  3431 raft_consensus.cc:2272] T 1b7b0c6c811e455abe898cb4d21acad4 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.289845  3431 tablet_replica.cc:333] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.290817  3431 raft_consensus.cc:2243] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.292062  3431 raft_consensus.cc:2272] T 51b7b2074e114d8f872f149fa8acdbcf P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.295265  3431 tablet_replica.cc:333] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.296106  3431 raft_consensus.cc:2243] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.296759  3431 raft_consensus.cc:2272] T ea533a99927c40d48b894cf00a75301d P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:03.242764  4645 consensus_peers.cc:597] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:03.298257  4645 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 55559.3us
W20260709 12:28:03.304941  4645 consensus_peers.cc:597] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:28:03.308179  3431 tablet_replica.cc:333] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.310686  3431 raft_consensus.cc:2243] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.312345  3431 raft_consensus.cc:2272] T d410e14541184b6bbaff38737ce4bc02 P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.315332  3431 tablet_replica.cc:333] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.316284  3431 raft_consensus.cc:2243] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.317605  3431 raft_consensus.cc:2272] T abf423ac214b42acbe1696a11279567c P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.333436  3431 tablet_replica.cc:333] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.334347  3431 raft_consensus.cc:2243] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.335745  3431 raft_consensus.cc:2272] T 501b9483d1b74b73876563dd85e610d0 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.338897  3431 tablet_replica.cc:333] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.339809  3431 raft_consensus.cc:2243] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.341132  3431 raft_consensus.cc:2272] T 559316f720c74aa1b50405e2ed791f6a P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.356966  3431 tablet_replica.cc:333] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.374305  3431 raft_consensus.cc:2243] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.375527  3431 raft_consensus.cc:2272] T b8e62ff81eb34853811314530131292b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.378502  3431 tablet_replica.cc:333] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.379313  3431 raft_consensus.cc:2243] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.385413  3431 raft_consensus.cc:2272] T 8fc78aa9a3bf48afb52246b50ad414eb P 3a5aeedd11fe49938079fe86cb01232c [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.388240  3431 tablet_replica.cc:333] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.389140  3431 raft_consensus.cc:2243] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:03.397763 12067 raft_consensus.cc:670] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 12:28:03.404749  3431 raft_consensus.cc:2272] T f173b0d5264e4eff8b1704becc555780 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.427217  3431 tablet_replica.cc:333] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.428119  3431 raft_consensus.cc:2243] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.428879  3431 raft_consensus.cc:2272] T b4a575b75e05458186c89873e1e7622b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.433211 12067 raft_consensus.cc:493] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:28:03.433930 12067 raft_consensus.cc:515] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:03.438287  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cc90257afed46488d4574c06b22a1a4" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:28:03.441527 12067 leader_election.cc:290] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:28:03.431816  3431 tablet_replica.cc:333] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.443058  3431 raft_consensus.cc:2243] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.444249  3431 raft_consensus.cc:2272] T 3be428c36f9242c6a2b8aa853f2c8b2c P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.447310  3431 tablet_replica.cc:333] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.448143  3431 raft_consensus.cc:2243] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 10 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.449294  3431 raft_consensus.cc:2272] T c3d4ce22fd7947598cc1d1c7732285da P 3a5aeedd11fe49938079fe86cb01232c [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.452625  3431 tablet_replica.cc:333] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.453542  3431 raft_consensus.cc:2243] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.470532 12067 raft_consensus.cc:493] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:28:03.471305 12067 raft_consensus.cc:515] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:03.478046  3431 raft_consensus.cc:2272] T 1fdfc68cfdd1411cad1b4e974e65067a P 3a5aeedd11fe49938079fe86cb01232c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.481112  3431 tablet_replica.cc:333] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.481992  3431 raft_consensus.cc:2243] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.482610  3431 raft_consensus.cc:2272] T 09d362006ea24f48a30fa120bbdf8860 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.482383  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:28:03.485281  3431 tablet_replica.cc:333] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.486116  3431 raft_consensus.cc:2243] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.486447  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab695b6fa8e145c7926a540c3ba5b84e" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:28:03.486976  3431 raft_consensus.cc:2272] T 1d3124fc8af34f378ebd9cff21ee0e6b P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.488435  4644 leader_election.cc:304] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:28:03.489552 12082 raft_consensus.cc:2749] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:03.490865 12067 leader_election.cc:290] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:28:03.493840 12067 raft_consensus.cc:493] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:28:03.495118 12082 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:28:03.495707 12082 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:03.500834  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:28:03.507958  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c73ae37134549b12188dddc4fa794" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:28:03.509289  3431 tablet_replica.cc:333] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.510252  3431 raft_consensus.cc:2243] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.510897  3431 raft_consensus.cc:2272] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.512097  4644 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:28:03.513108 12078 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:03.516695 12082 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:28:03.494436 12067 raft_consensus.cc:515] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:03.513782  3431 tablet_replica.cc:333] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
W20260709 12:28:03.521289  5312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45077, user_credentials={real_user=slave}} blocked reactor thread for 51889.8us
I20260709 12:28:03.521425  3431 raft_consensus.cc:2243] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.522996  3431 raft_consensus.cc:2272] T 13ebb8f343c743a1a28db116a4881004 P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.527776  3431 tablet_replica.cc:333] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.528990  3431 raft_consensus.cc:2243] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:03.517686  4720 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:42599, user_credentials={real_user=slave}} blocked reactor thread for 50160.8us
W20260709 12:28:03.521948  5312 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:03.530786  4720 consensus_peers.cc:597] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:03.531795  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:03.534543  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96cc733643644a9ca6f1b2613e987020" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:28:03.535048 12067 leader_election.cc:290] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:28:03.535485  4645 leader_election.cc:336] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:03.539283  4645 leader_election.cc:304] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
W20260709 12:28:03.539979  4645 raft_consensus.cc:2686] T 7cc90257afed46488d4574c06b22a1a4 P 3a5aeedd11fe49938079fe86cb01232c: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:28:03.540602  4645 consensus_peers.cc:597] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:03.544482  3431 raft_consensus.cc:2272] T dafe783bf87845d6bb64a279ed7d0f75 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.551980 12067 raft_consensus.cc:493] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader dbe960c35c84451faf4e5b5c9233fa39)
I20260709 12:28:03.552569 12067 raft_consensus.cc:515] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:03.555400  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
W20260709 12:28:03.557920  4720 consensus_peers.cc:597] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:03.563732 12067 leader_election.cc:290] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:28:03.566009  5346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718b286f0ba24a90af47c49ec0031b11" candidate_uuid: "3a5aeedd11fe49938079fe86cb01232c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:28:03.570266  4644 leader_election.cc:304] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:28:03.572355 12082 raft_consensus.cc:2749] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:03.574069  3431 tablet_replica.cc:333] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.574965  3431 raft_consensus.cc:2243] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.576252  3431 raft_consensus.cc:2272] T aba6aa625d5a49658718de09523b8a9b P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.580062  3431 tablet_replica.cc:333] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
W20260709 12:28:03.576424  4645 leader_election.cc:336] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:03.580971  3431 raft_consensus.cc:2243] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 24 LEADER]: Raft consensus shutting down.
W20260709 12:28:03.581503  4645 leader_election.cc:336] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:03.582022  4645 leader_election.cc:304] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a5aeedd11fe49938079fe86cb01232c; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:28:03.582234  3431 raft_consensus.cc:2272] T 26f80146cffc4af2ac0af76b3a99c213 P 3a5aeedd11fe49938079fe86cb01232c [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:03.578794  4720 consensus_peers.cc:597] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:03.586938 12082 raft_consensus.cc:2749] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:03.590272  3431 tablet_replica.cc:333] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.591921  3431 raft_consensus.cc:2243] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.593196  3431 raft_consensus.cc:2272] T 7b27de60f9f44928b0068ab1e562568c P 3a5aeedd11fe49938079fe86cb01232c [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.596417  3431 tablet_replica.cc:333] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.597302  3431 raft_consensus.cc:2243] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.598513  3431 raft_consensus.cc:2272] T 4d675fbac0f844dc9309a662ba1c96f9 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.603097  3431 tablet_replica.cc:333] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.603960  3431 raft_consensus.cc:2243] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Raft consensus shutting down.
W20260709 12:28:03.606178  5312 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:03.607244  3431 raft_consensus.cc:2272] T 86297ef907dd4e3590805d3cf15efaca P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:03.607256  4720 consensus_peers.cc:597] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:03.610370  3431 tablet_replica.cc:333] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.611373  3431 raft_consensus.cc:2243] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.612323  3431 raft_consensus.cc:2272] T ee319a06cc4642fba7ae226abe36e352 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.615293  3431 tablet_replica.cc:333] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.616164  3431 raft_consensus.cc:2243] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.616838  3431 raft_consensus.cc:2272] T 96cc733643644a9ca6f1b2613e987020 P 3a5aeedd11fe49938079fe86cb01232c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.619788  3431 tablet_replica.cc:333] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.620558  3431 raft_consensus.cc:2243] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.621409  3431 raft_consensus.cc:2272] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 3a5aeedd11fe49938079fe86cb01232c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.624867  3431 tablet_replica.cc:333] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.625726  3431 raft_consensus.cc:2243] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.626976  3431 raft_consensus.cc:2272] T 8d9a7637095341feabd700d53579d5d9 P 3a5aeedd11fe49938079fe86cb01232c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.631976  3431 tablet_replica.cc:333] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.634853  3431 raft_consensus.cc:2243] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.640946  3431 raft_consensus.cc:2272] T 9702ea78030547e5a2f6fffe805075da P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:03.641037  5312 consensus_peers.cc:597] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:03.644268  3431 tablet_replica.cc:333] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.645174  3431 raft_consensus.cc:2243] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.646092  3431 raft_consensus.cc:2272] T f54c73ae37134549b12188dddc4fa794 P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.651630  3431 tablet_replica.cc:333] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.652751  3431 raft_consensus.cc:2243] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.653818  3431 raft_consensus.cc:2272] T ab695b6fa8e145c7926a540c3ba5b84e P 3a5aeedd11fe49938079fe86cb01232c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.656710  3431 tablet_replica.cc:333] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.657531  3431 raft_consensus.cc:2243] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:28:03.658855  3431 raft_consensus.cc:2272] T 231dbfb1b78d491eb8685070c2431d82 P 3a5aeedd11fe49938079fe86cb01232c [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.661707  3431 tablet_replica.cc:333] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.662461  3431 raft_consensus.cc:2243] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.663208  3431 raft_consensus.cc:2272] T 718b286f0ba24a90af47c49ec0031b11 P 3a5aeedd11fe49938079fe86cb01232c [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.665977  3431 tablet_replica.cc:333] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.666775  3431 raft_consensus.cc:2243] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.667542  3431 raft_consensus.cc:2272] T 23fb4c89626845d8bac3ba87a5346995 P 3a5aeedd11fe49938079fe86cb01232c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:03.670341  3431 tablet_replica.cc:333] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c: stopping tablet replica
I20260709 12:28:03.671080  3431 raft_consensus.cc:2243] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:03.671835  3431 raft_consensus.cc:2272] T 18c2a1115a9a4b4a975dadb63bef992a P 3a5aeedd11fe49938079fe86cb01232c [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:03.725217  4720 consensus_peers.cc:597] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:03.843590  4720 consensus_peers.cc:597] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:03.882083  5312 consensus_peers.cc:597] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:04.005309  4720 consensus_peers.cc:597] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:04.019712  4720 consensus_peers.cc:597] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:04.050668  5312 consensus_peers.cc:597] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:04.066782  5312 consensus_peers.cc:597] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:04.083118 11994 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.083750 11994 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:28:04.085968  4720 consensus_peers.cc:597] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:04.089510 11994 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:28:04.090035  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:28:04.107136  4720 leader_election.cc:336] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.107632  4720 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:28:04.111142 11994 raft_consensus.cc:2749] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:04.120843  4720 consensus_peers.cc:597] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:04.177837  4720 consensus_peers.cc:597] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:04.224226  5312 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:04.235720  4720 consensus_peers.cc:597] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:04.252521  4720 consensus_peers.cc:597] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:04.257758 11994 raft_consensus.cc:493] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.258435 11994 raft_consensus.cc:515] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:04.260730 11994 leader_election.cc:290] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:28:04.272588  4720 leader_election.cc:336] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:04.274432  4720 leader_election.cc:336] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.274850  4720 leader_election.cc:304] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:04.275593 11994 raft_consensus.cc:2749] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:04.286633 11994 raft_consensus.cc:493] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.287182 11994 raft_consensus.cc:515] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:04.289207 11994 leader_election.cc:290] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:28:04.294823  4720 leader_election.cc:336] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:04.295542  4720 leader_election.cc:336] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.295924  4720 leader_election.cc:304] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:04.296641 11994 raft_consensus.cc:2749] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:04.306701  4720 consensus_peers.cc:597] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:04.322359 11994 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.322830 11994 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.324611 11994 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:28:04.325765  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
W20260709 12:28:04.330590  4720 leader_election.cc:336] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.331041  4720 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:28:04.331681 11994 raft_consensus.cc:2749] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:04.360903 12074 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.361515 12074 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.378912  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:28:04.379626  4753 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 21.
I20260709 12:28:04.380924  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:28:04.381759 12081 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:04.382140 12081 raft_consensus.cc:493] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.382484 12081 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:04.382404  5312 leader_election.cc:336] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.384116 12074 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:28:04.495294 11994 raft_consensus.cc:493] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.495718 11994 raft_consensus.cc:515] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:04.497342 11994 leader_election.cc:290] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:28:04.506554 12073 raft_consensus.cc:493] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.507097 12073 raft_consensus.cc:515] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:04.509142 12073 leader_election.cc:290] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:28:04.538614 12073 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.551412 11994 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.551965 11994 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.552287 12081 raft_consensus.cc:515] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.554306 12081 leader_election.cc:290] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:28:04.549330 12073 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.555318  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:28:04.556656 12073 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:28:04.558104  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:28:04.558735  5340 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 10.
I20260709 12:28:04.560266  4719 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:28:04.560575 12085 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.561208 12073 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:28:04.561146 12085 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.561697 12073 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.562165 12073 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:28:04.563288 12085 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:28:04.563652 11994 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:28:04.571739 12073 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.575474 12088 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.576172 12088 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.578380 12088 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:28:04.579355 12089 raft_consensus.cc:493] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.580107 12089 raft_consensus.cc:515] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:04.582762 12089 leader_election.cc:290] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:28:04.592080  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39"
I20260709 12:28:04.592829  5340 raft_consensus.cc:3060] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:28:04.592296 12073 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:28:04.599107  5312 consensus_peers.cc:597] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:04.600841  5312 leader_election.cc:336] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
W20260709 12:28:04.601351  5312 leader_election.cc:336] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
W20260709 12:28:04.601895  5312 leader_election.cc:336] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.606959  4753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28f87749718d445ba1615ff33028c4d8" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:28:04.607649  4753 raft_consensus.cc:3060] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:04.608775  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d4ce22fd7947598cc1d1c7732285da" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:28:04.609633  4747 raft_consensus.cc:2393] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dbe960c35c84451faf4e5b5c9233fa39 in current term 11: Already voted for candidate 49277a7653524787a715232ab908bbc6 in this term.
I20260709 12:28:04.614218  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:28:04.614850  4747 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 16.
I20260709 12:28:04.605428  5340 raft_consensus.cc:2468] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49277a7653524787a715232ab908bbc6 in term 11.
I20260709 12:28:04.616813  4719 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:28:04.617677 12073 raft_consensus.cc:2804] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:28:04.618286 12073 raft_consensus.cc:697] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Becoming Leader. State: Replica: 49277a7653524787a715232ab908bbc6, State: Running, Role: LEADER
I20260709 12:28:04.618599  4753 raft_consensus.cc:2468] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 22.
I20260709 12:28:04.619290 12073 consensus_queue.cc:237] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.630160  5314 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
W20260709 12:28:04.632794  4720 leader_election.cc:336] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:04.633626  4720 leader_election.cc:336] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:04.634184  4720 consensus_peers.cc:597] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:04.635110 12085 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:28:04.638154  5314 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:28:04.638973 12088 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:04.639366 12088 raft_consensus.cc:493] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.639748 12088 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:04.653973 12085 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.654740 12085 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.657292 12085 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:28:04.660212  5314 leader_election.cc:304] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:28:04.668900 12073 raft_consensus.cc:493] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.669607 12073 raft_consensus.cc:515] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:04.664770 12090 raft_consensus.cc:2804] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:04.673192 12090 raft_consensus.cc:697] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:28:04.674232 12090 consensus_queue.cc:237] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.681012 12073 leader_election.cc:290] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:28:04.679152  4501 catalog_manager.cc:5697] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 reported cstate change: term changed from 10 to 11, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to 49277a7653524787a715232ab908bbc6 (127.3.89.195). New cstate: current_term: 11 leader_uuid: "49277a7653524787a715232ab908bbc6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:04.682960  4720 leader_election.cc:336] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.683531  4720 leader_election.cc:304] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:28:04.684425  5312 leader_election.cc:336] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:04.686753  5312 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:04.687544  5312 leader_election.cc:336] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.687996  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:04.688844 12074 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:04.685263 12088 raft_consensus.cc:515] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.685015 12073 raft_consensus.cc:2749] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:04.691852 12088 leader_election.cc:290] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:28:04.696434  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d6e5f40333a47559cd852e4dfa7c86b" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:28:04.697114  4747 raft_consensus.cc:3060] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:04.697815  4720 leader_election.cc:336] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.693552 12074 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.698241  4720 leader_election.cc:304] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:04.698524 12074 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.699000 12073 raft_consensus.cc:2749] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:04.700605 12074 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:28:04.706815 11994 raft_consensus.cc:493] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.707362 11994 raft_consensus.cc:515] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:04.709252 11994 leader_election.cc:290] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:28:04.711635  5312 leader_election.cc:336] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
W20260709 12:28:04.719861  5312 leader_election.cc:336] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.720130 12087 raft_consensus.cc:493] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:28:04.720712  5312 leader_election.cc:336] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:04.720752 12087 raft_consensus.cc:515] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:04.721357  5312 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:04.723253 12087 leader_election.cc:290] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:28:04.723798 12090 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:04.727123  4496 catalog_manager.cc:5697] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 21 to 22, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 22 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:04.731477  4720 leader_election.cc:336] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:04.736402  4747 raft_consensus.cc:2468] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 17.
W20260709 12:28:04.744529  4720 leader_election.cc:336] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.750378  4720 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:28:04.750651  5314 leader_election.cc:304] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
W20260709 12:28:04.751295  4720 leader_election.cc:336] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.751770 12090 raft_consensus.cc:2804] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:04.752262 12090 raft_consensus.cc:697] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:28:04.752753 12088 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.753332 12088 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.753276 12090 consensus_queue.cc:237] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.755964 12088 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:28:04.751080 12087 raft_consensus.cc:2749] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:04.752310  4720 leader_election.cc:336] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.762972  4720 leader_election.cc:304] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:04.764386 12087 raft_consensus.cc:2749] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:04.765420  4720 leader_election.cc:336] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.767612  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:28:04.768395  4747 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 22.
I20260709 12:28:04.769882  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
W20260709 12:28:04.772431  5312 leader_election.cc:336] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.773725 12088 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:04.774237 12088 raft_consensus.cc:493] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:28:04.775588  4720 consensus_peers.cc:597] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:28:04.774693 12088 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:04.777019  4720 leader_election.cc:336] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:04.780376  4720 consensus_peers.cc:597] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:04.781473 12087 raft_consensus.cc:493] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.786974 12087 raft_consensus.cc:515] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:04.788977 12087 leader_election.cc:290] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:28:04.791211  4720 leader_election.cc:336] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.791695  4720 leader_election.cc:304] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:04.795162 12087 raft_consensus.cc:2749] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:28:04.796741  4720 leader_election.cc:336] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.798995 12088 raft_consensus.cc:515] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:28:04.810617  4720 leader_election.cc:336] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:04.812497  5312 consensus_peers.cc:597] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:04.818157  4496 catalog_manager.cc:5697] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 16 to 17, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 17 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:04.825042  4720 leader_election.cc:304] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:04.827126 12073 raft_consensus.cc:2749] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:28:04.827298  4720 leader_election.cc:336] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:04.828627  4720 leader_election.cc:336] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.829008  4720 leader_election.cc:304] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:04.829758 12073 raft_consensus.cc:2749] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:04.833184  4720 leader_election.cc:336] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.835671  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559316f720c74aa1b50405e2ed791f6a" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:28:04.847004  4747 raft_consensus.cc:3060] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:04.836723 12088 leader_election.cc:290] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:28:04.838904  5312 leader_election.cc:336] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.850601 12073 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:28:04.857393  4720 leader_election.cc:336] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:04.857525 12073 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.858098  4720 leader_election.cc:304] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:04.858934 12087 raft_consensus.cc:2749] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:04.862313  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:28:04.864427 12073 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
W20260709 12:28:04.865144  5312 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:04.868269  4747 raft_consensus.cc:2468] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 23.
I20260709 12:28:04.869596  5314 leader_election.cc:304] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:28:04.870894 12088 raft_consensus.cc:2804] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:04.871562 12088 raft_consensus.cc:697] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 23 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:28:04.872678 12088 consensus_queue.cc:237] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:28:04.869710  4720 consensus_peers.cc:597] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:04.883365  4720 leader_election.cc:336] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.884840  4720 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:28:04.886641 12073 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:04.888121  4720 leader_election.cc:336] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:04.894323 12090 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.894887 12090 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.897038 12090 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:28:04.905089  4496 catalog_manager.cc:5697] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 22 to 23, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 23 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:04.906778  5312 leader_election.cc:336] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
W20260709 12:28:04.907466  5312 leader_election.cc:336] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:04.907824  5312 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:04.908758 12074 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:04.935824 12073 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.936444 12073 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.938941 12073 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196:39067)
I20260709 12:28:04.941073  5340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "49277a7653524787a715232ab908bbc6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "dbe960c35c84451faf4e5b5c9233fa39" is_pre_election: true
I20260709 12:28:04.955399 12088 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.955938 12088 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:28:04.957593  4720 consensus_peers.cc:597] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:04.958209 12088 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:28:04.960958  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:28:04.961591  4747 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 19.
I20260709 12:28:04.966252  5314 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 
I20260709 12:28:04.968070 12088 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:04.968452 12088 raft_consensus.cc:493] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
W20260709 12:28:04.971186  5312 leader_election.cc:336] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:04.974129 12088 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:04.980448 12073 raft_consensus.cc:493] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:04.981259 12073 raft_consensus.cc:515] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:04.983834 12073 leader_election.cc:290] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:28:04.992060  5312 consensus_peers.cc:597] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:04.994760 12088 raft_consensus.cc:515] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:04.997850  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13ebb8f343c743a1a28db116a4881004" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6"
I20260709 12:28:04.998399  4747 raft_consensus.cc:3060] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:05.006784  4747 raft_consensus.cc:2468] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 20.
W20260709 12:28:04.995693  4720 leader_election.cc:336] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:05.007650  4720 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, dbe960c35c84451faf4e5b5c9233fa39
I20260709 12:28:05.008076 12088 leader_election.cc:290] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 election: Requested vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:28:05.008216  5312 leader_election.cc:336] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:05.010596 12087 raft_consensus.cc:2749] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:05.014937  5314 leader_election.cc:304] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c
I20260709 12:28:05.016048 12088 raft_consensus.cc:2804] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:05.016610 12088 raft_consensus.cc:697] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 LEADER]: Becoming Leader. State: Replica: dbe960c35c84451faf4e5b5c9233fa39, State: Running, Role: LEADER
I20260709 12:28:05.017720 12088 consensus_queue.cc:237] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:28:05.029397  4720 consensus_peers.cc:597] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:05.038614  5312 consensus_peers.cc:597] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:05.046396  4496 catalog_manager.cc:5697] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 reported cstate change: term changed from 19 to 20, leader changed from 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194) to dbe960c35c84451faf4e5b5c9233fa39 (127.3.89.196). New cstate: current_term: 20 leader_uuid: "dbe960c35c84451faf4e5b5c9233fa39" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:05.055932 12087 raft_consensus.cc:493] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:05.056664 12087 raft_consensus.cc:515] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:05.064528 12087 leader_election.cc:290] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:28:05.080652 12074 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:05.081260 12074 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:05.087209 12074 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:28:05.088632  4747 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86297ef907dd4e3590805d3cf15efaca" candidate_uuid: "dbe960c35c84451faf4e5b5c9233fa39" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "49277a7653524787a715232ab908bbc6" is_pre_election: true
I20260709 12:28:05.089284  4747 raft_consensus.cc:2468] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe960c35c84451faf4e5b5c9233fa39 in term 15.
W20260709 12:28:05.090518  4720 leader_election.cc:336] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:05.098731  5312 leader_election.cc:336] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:05.099556  4720 leader_election.cc:336] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:05.101097  4720 leader_election.cc:304] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:05.106070 12087 raft_consensus.cc:2749] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:05.105111  5312 leader_election.cc:336] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:05.110146  5312 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6, dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:05.111274 12088 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:05.118932  5340 raft_consensus.cc:1275] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Refusing update from remote peer 49277a7653524787a715232ab908bbc6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:28:05.126395 12087 consensus_queue.cc:1048] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:05.131763  3431 tablet_server.cc:196] TabletServer@127.3.89.194:0 shutdown complete.
I20260709 12:28:05.136011 12087 raft_consensus.cc:493] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:05.136772 12087 raft_consensus.cc:515] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } }
I20260709 12:28:05.141652 12087 leader_election.cc:290] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:28:05.146039  4747 raft_consensus.cc:1275] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:28:05.150336 12074 consensus_queue.cc:1048] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:28:05.157810  5312 consensus_peers.cc:597] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:05.159154  5312 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.159576  5312 consensus_peers.cc:597] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.164215  5312 consensus_peers.cc:597] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.211464  4749 raft_consensus.cc:1275] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:28:05.214319 12088 consensus_queue.cc:1048] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:28:05.224363  5312 consensus_peers.cc:597] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.228027  4720 leader_election.cc:336] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:05.267568  4720 consensus_peers.cc:597] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.271241  4720 consensus_peers.cc:597] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.270119  5312 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.272424  4720 leader_election.cc:336] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:05.272989  4720 leader_election.cc:304] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:05.281369 12073 raft_consensus.cc:2749] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:28:05.283818  4720 consensus_peers.cc:597] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.286860  4720 consensus_peers.cc:597] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.310397  4720 consensus_peers.cc:597] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.312018  4720 consensus_peers.cc:597] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.313244  4720 leader_election.cc:336] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
W20260709 12:28:05.314457  4720 consensus_peers.cc:597] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.315574  4720 consensus_peers.cc:597] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:05.320988  4753 raft_consensus.cc:1275] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Refusing update from remote peer dbe960c35c84451faf4e5b5c9233fa39: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:05.322811 12074 consensus_queue.cc:1048] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:28:05.339839  5312 consensus_peers.cc:597] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.352941  5312 consensus_peers.cc:597] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.353706  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.387813  4720 consensus_peers.cc:597] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.389014  4720 consensus_peers.cc:597] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.389559  4720 consensus_peers.cc:597] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.390811  4720 consensus_peers.cc:597] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.393463  4720 leader_election.cc:336] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:05.388447  5312 consensus_peers.cc:597] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.396934  4533 debug-util.cc:398] Leaking SignalData structure 0x7b080000b620 after lost signal to thread 3434
W20260709 12:28:05.397820  5312 consensus_peers.cc:597] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.398150  4533 debug-util.cc:398] Leaking SignalData structure 0x7b080037e840 after lost signal to thread 4536
I20260709 12:28:05.398741  4720 leader_election.cc:304] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49277a7653524787a715232ab908bbc6; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
W20260709 12:28:05.399670  4533 debug-util.cc:398] Leaking SignalData structure 0x7b0800174a80 after lost signal to thread 4780
W20260709 12:28:05.401000  4533 debug-util.cc:398] Leaking SignalData structure 0x7b080018c620 after lost signal to thread 5373
I20260709 12:28:05.399719 12087 raft_consensus.cc:2749] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:05.466699  5312 consensus_peers.cc:597] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:05.476761  4720 consensus_peers.cc:597] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.494433  4720 consensus_peers.cc:597] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:05.497308  3431 tablet_server.cc:179] TabletServer@127.3.89.195:0 shutting down...
W20260709 12:28:05.501428  4720 consensus_peers.cc:597] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:05.538429  4720 consensus_peers.cc:597] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:05.540468  5314 consensus_peers.cc:597] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.559244  5314 consensus_peers.cc:597] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.566872  5312 consensus_peers.cc:597] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.592592  5314 consensus_peers.cc:597] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.595723  5314 consensus_peers.cc:597] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.606135  5314 consensus_peers.cc:597] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.616153  5314 consensus_peers.cc:597] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.633206  5314 consensus_peers.cc:597] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.645184  5314 consensus_peers.cc:597] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.650520  4720 consensus_peers.cc:597] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:05.693704  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:28:05.690662  5314 consensus_peers.cc:597] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.702236  5314 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.706075  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:28:05.707052  3431 tablet_replica.cc:333] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.709944  3431 raft_consensus.cc:2243] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Raft consensus shutting down.
W20260709 12:28:05.711737  4720 consensus_peers.cc:597] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:05.712173  3431 raft_consensus.cc:2272] T 9ced730aeaf240b69d68fd0ee470d68d P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.715059  3431 tablet_replica.cc:333] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.715924  3431 raft_consensus.cc:2243] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.716629  3431 raft_consensus.cc:2272] T ae7ffb2806e441e787796287d56e8b8a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.735482  3431 tablet_replica.cc:333] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
W20260709 12:28:05.736191  5314 consensus_peers.cc:597] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.736340  3431 raft_consensus.cc:2243] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.737182  3431 raft_consensus.cc:2272] T 8f05b2c44da743e49df87266876b3604 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.739877  3431 tablet_replica.cc:333] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.740722  3431 raft_consensus.cc:2243] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.741273  3431 raft_consensus.cc:2272] T 28f87749718d445ba1615ff33028c4d8 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.746431  3431 tablet_replica.cc:333] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.747328  3431 raft_consensus.cc:2243] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Raft consensus shutting down.
W20260709 12:28:05.747706  5314 consensus_peers.cc:597] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.748608  3431 raft_consensus.cc:2272] T b7cd71e05324451f8e6dfcf6c2e76829 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.751502  3431 tablet_replica.cc:333] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.752664  3431 raft_consensus.cc:2243] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.753836  3431 raft_consensus.cc:2272] T 2d6e5f40333a47559cd852e4dfa7c86b P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.758231  3431 tablet_replica.cc:333] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.759071  3431 raft_consensus.cc:2243] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.760583  3431 raft_consensus.cc:2272] T 485fe0cee4b248f4802d3c15dc97a877 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.764811  3431 tablet_replica.cc:333] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.765618  3431 raft_consensus.cc:2243] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.769877  3431 raft_consensus.cc:2272] T e85386ee6a30465188e201d5ee6ae972 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.772987  3431 tablet_replica.cc:333] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.779979  3431 raft_consensus.cc:2243] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.781184  3431 raft_consensus.cc:2272] T e8a74137ba104fe0b465cab6399c4940 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.784116  3431 tablet_replica.cc:333] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.784932  3431 raft_consensus.cc:2243] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.798341  3431 raft_consensus.cc:2272] T 3cbe5bef0f514b01ac552ff54d4d13be P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.801131  3431 tablet_replica.cc:333] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.802003  3431 raft_consensus.cc:2243] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.803124  3431 raft_consensus.cc:2272] T 42d36b6bb6e84733876824d95e25967e P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:05.803618  4720 consensus_peers.cc:597] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:05.804229  4720 consensus_peers.cc:597] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:28:05.805836  3431 tablet_replica.cc:333] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.806742  3431 raft_consensus.cc:2243] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.807806  3431 raft_consensus.cc:2272] T 53908897ce974579947af90acaf0aed7 P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.816860  3431 tablet_replica.cc:333] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.817879  3431 raft_consensus.cc:2243] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.822373  3431 raft_consensus.cc:2272] T b6e604992a304031ad5cbdc32d90edb4 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.825107  3431 tablet_replica.cc:333] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.825865  3431 raft_consensus.cc:2243] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.831537  3431 raft_consensus.cc:2272] T 4163edf182b549dfb724d2e5b5b1ac0e P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.834527  3431 tablet_replica.cc:333] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.835167  3431 raft_consensus.cc:2243] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.835713  3431 raft_consensus.cc:2272] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P 49277a7653524787a715232ab908bbc6 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.837920  3431 tablet_replica.cc:333] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.838759  3431 raft_consensus.cc:2243] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.839408  3431 raft_consensus.cc:2272] T 51b7b2074e114d8f872f149fa8acdbcf P 49277a7653524787a715232ab908bbc6 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.842823  3431 tablet_replica.cc:333] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.843670  3431 raft_consensus.cc:2243] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.844326  3431 raft_consensus.cc:2272] T 7b27de60f9f44928b0068ab1e562568c P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.847023  3431 tablet_replica.cc:333] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.847841  3431 raft_consensus.cc:2243] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.849282  3431 raft_consensus.cc:2272] T 23fb4c89626845d8bac3ba87a5346995 P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:05.850979  5314 consensus_peers.cc:597] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.852809  3431 tablet_replica.cc:333] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.855516  3431 raft_consensus.cc:2243] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.856202  3431 raft_consensus.cc:2272] T 718b286f0ba24a90af47c49ec0031b11 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.858952  3431 tablet_replica.cc:333] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.859798  3431 raft_consensus.cc:2243] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.860602  3431 raft_consensus.cc:2272] T 501b9483d1b74b73876563dd85e610d0 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.863358  3431 tablet_replica.cc:333] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.864198  3431 raft_consensus.cc:2243] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.864755  3431 raft_consensus.cc:2272] T 559316f720c74aa1b50405e2ed791f6a P 49277a7653524787a715232ab908bbc6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.871049  3431 tablet_replica.cc:333] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.871964  3431 raft_consensus.cc:2243] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.872747  3431 raft_consensus.cc:2272] T b8e62ff81eb34853811314530131292b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.875394  3431 tablet_replica.cc:333] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.876205  3431 raft_consensus.cc:2243] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.876996  3431 raft_consensus.cc:2272] T 1fdfc68cfdd1411cad1b4e974e65067a P 49277a7653524787a715232ab908bbc6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.879887  3431 tablet_replica.cc:333] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.880645  3431 raft_consensus.cc:2243] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.881755  3431 raft_consensus.cc:2272] T 9ca7c574e14c4faeac378ee00c0a261b P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.884217  3431 tablet_replica.cc:333] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.885083  3431 raft_consensus.cc:2243] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.886253  3431 raft_consensus.cc:2272] T 09d362006ea24f48a30fa120bbdf8860 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.889061  3431 tablet_replica.cc:333] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.889988  3431 raft_consensus.cc:2243] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.891439  3431 raft_consensus.cc:2272] T 1d3124fc8af34f378ebd9cff21ee0e6b P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.894074  3431 tablet_replica.cc:333] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.894908  3431 raft_consensus.cc:2243] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.895795  3431 raft_consensus.cc:2272] T 70fb561fb8b446969622ee2193d90917 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.898481  3431 tablet_replica.cc:333] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.899204  3431 raft_consensus.cc:2243] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.900223  3431 raft_consensus.cc:2272] T 7cc90257afed46488d4574c06b22a1a4 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.902824  3431 tablet_replica.cc:333] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.903697  3431 raft_consensus.cc:2243] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.904968  3431 raft_consensus.cc:2272] T aba6aa625d5a49658718de09523b8a9b P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.907560  3431 tablet_replica.cc:333] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.908365  3431 raft_consensus.cc:2243] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.908960  3431 raft_consensus.cc:2272] T 26f80146cffc4af2ac0af76b3a99c213 P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.911854  3431 tablet_replica.cc:333] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.912612  3431 raft_consensus.cc:2243] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.913265  3431 raft_consensus.cc:2272] T c6f49e7bc416420cb85a6d4319f46c4d P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.915771  3431 tablet_replica.cc:333] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.916610  3431 raft_consensus.cc:2243] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.917229  3431 raft_consensus.cc:2272] T 4d675fbac0f844dc9309a662ba1c96f9 P 49277a7653524787a715232ab908bbc6 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.919838  3431 tablet_replica.cc:333] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.920605  3431 raft_consensus.cc:2243] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 19 LEADER]: Raft consensus shutting down.
W20260709 12:28:05.921594  5314 consensus_peers.cc:597] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.922087  3431 raft_consensus.cc:2272] T 3be428c36f9242c6a2b8aa853f2c8b2c P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.924624  3431 tablet_replica.cc:333] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.925463  3431 raft_consensus.cc:2243] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 11 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.928960  3431 raft_consensus.cc:2272] T c3d4ce22fd7947598cc1d1c7732285da P 49277a7653524787a715232ab908bbc6 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.937057  3431 tablet_replica.cc:333] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.938328  3431 raft_consensus.cc:2243] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.939569  3431 raft_consensus.cc:2272] T b4a575b75e05458186c89873e1e7622b P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.942323  3431 tablet_replica.cc:333] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.943135  3431 raft_consensus.cc:2243] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.943785  3431 raft_consensus.cc:2272] T 12796fc41f234615a607ba222506e893 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.946465  3431 tablet_replica.cc:333] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.947263  3431 raft_consensus.cc:2243] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.950836  3431 raft_consensus.cc:2272] T abf423ac214b42acbe1696a11279567c P 49277a7653524787a715232ab908bbc6 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.955000  3431 tablet_replica.cc:333] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.955740  3431 raft_consensus.cc:2243] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.956573  3431 raft_consensus.cc:2272] T 3e20c8bae3534224bb7a82db42cab1a5 P 49277a7653524787a715232ab908bbc6 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.959316  3431 tablet_replica.cc:333] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.960170  3431 raft_consensus.cc:2243] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 12 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.961517  3431 raft_consensus.cc:2272] T 23027d24ef1943bdbea76085a97b055c P 49277a7653524787a715232ab908bbc6 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.964286  3431 tablet_replica.cc:333] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
W20260709 12:28:05.964927  5312 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:28:05.965126  3431 raft_consensus.cc:2243] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.966236  3431 raft_consensus.cc:2272] T 13ebb8f343c743a1a28db116a4881004 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.969380  3431 tablet_replica.cc:333] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.970207  3431 raft_consensus.cc:2243] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Raft consensus shutting down.
W20260709 12:28:05.970706  5314 consensus_peers.cc:597] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.971220  3431 raft_consensus.cc:2272] T 18c2a1115a9a4b4a975dadb63bef992a P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.974076  3431 tablet_replica.cc:333] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.974869  3431 raft_consensus.cc:2243] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.975448  3431 raft_consensus.cc:2272] T 231dbfb1b78d491eb8685070c2431d82 P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.978181  3431 tablet_replica.cc:333] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.978984  3431 raft_consensus.cc:2243] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.980032  3431 raft_consensus.cc:2272] T 8e4e6edc86e64b8ca5e81a1f1c312757 P 49277a7653524787a715232ab908bbc6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.982985  3431 tablet_replica.cc:333] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.984011  3431 raft_consensus.cc:2243] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.984704  3431 raft_consensus.cc:2272] T 8d9a7637095341feabd700d53579d5d9 P 49277a7653524787a715232ab908bbc6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.989053  3431 tablet_replica.cc:333] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.989795  3431 raft_consensus.cc:2243] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:05.990988  3431 raft_consensus.cc:2272] T ee319a06cc4642fba7ae226abe36e352 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.993959  3431 tablet_replica.cc:333] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.994876  3431 raft_consensus.cc:2243] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:05.996016  3431 raft_consensus.cc:2272] T 86297ef907dd4e3590805d3cf15efaca P 49277a7653524787a715232ab908bbc6 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:05.998802  3431 tablet_replica.cc:333] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:05.999575  3431 raft_consensus.cc:2243] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:06.000799  3431 raft_consensus.cc:2272] T 96cc733643644a9ca6f1b2613e987020 P 49277a7653524787a715232ab908bbc6 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:06.003732  3431 tablet_replica.cc:333] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:06.004487  3431 raft_consensus.cc:2243] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:28:06.005640  3431 raft_consensus.cc:2272] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:06.008270  3431 tablet_replica.cc:333] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:06.009027  3431 raft_consensus.cc:2243] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:06.009857  3431 raft_consensus.cc:2272] T ab695b6fa8e145c7926a540c3ba5b84e P 49277a7653524787a715232ab908bbc6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:06.013556  3431 tablet_replica.cc:333] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6: stopping tablet replica
I20260709 12:28:06.014321  3431 raft_consensus.cc:2243] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:28:06.015969  3431 raft_consensus.cc:2272] T f54c73ae37134549b12188dddc4fa794 P 49277a7653524787a715232ab908bbc6 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:06.025434  5312 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:06.236099  5312 consensus_peers.cc:597] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:06.312379  5312 consensus_peers.cc:597] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:28:06.343479 12119 raft_consensus.cc:493] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:06.344882 12119 raft_consensus.cc:515] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:06.361433 12119 leader_election.cc:290] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:28:06.369889  5312 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:06.370402  5312 leader_election.cc:336] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:06.371151  5312 leader_election.cc:336] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:06.371592  5312 leader_election.cc:304] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:06.373921 12110 raft_consensus.cc:2749] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:06.474889  5312 consensus_peers.cc:597] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:06.558096  5312 consensus_peers.cc:597] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:06.791710 12112 raft_consensus.cc:493] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:06.792385 12112 raft_consensus.cc:515] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:06.794691 12112 leader_election.cc:290] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
W20260709 12:28:06.803377  5312 leader_election.cc:336] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
W20260709 12:28:06.805879  5312 leader_election.cc:336] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:06.806317  5312 leader_election.cc:304] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:06.809075 12112 raft_consensus.cc:2749] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:06.836522 12112 raft_consensus.cc:493] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:28:06.837378 12112 raft_consensus.cc:515] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:28:06.854750  5314 leader_election.cc:336] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111)
I20260709 12:28:06.855733 12110 raft_consensus.cc:493] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:06.856482 12110 raft_consensus.cc:515] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:06.859616 12110 leader_election.cc:290] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:28:06.861177  5312 leader_election.cc:336] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:06.861698  5312 leader_election.cc:304] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:28:06.865190 12112 leader_election.cc:290] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:28:06.865752 12112 raft_consensus.cc:2749] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:06.870805  5314 leader_election.cc:336] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111)
W20260709 12:28:06.880189  5312 leader_election.cc:336] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:06.907150  5312 leader_election.cc:336] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:06.907625  5312 leader_election.cc:304] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:28:06.909621 12112 raft_consensus.cc:2749] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:28:06.916067  5312 leader_election.cc:336] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:06.925588 12110 raft_consensus.cc:493] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3a5aeedd11fe49938079fe86cb01232c)
I20260709 12:28:06.926285 12110 raft_consensus.cc:515] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:06.928514 12110 leader_election.cc:290] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599)
I20260709 12:28:06.929435 12110 raft_consensus.cc:493] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:28:06.930150 12110 raft_consensus.cc:515] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:28:06.938547  5314 leader_election.cc:336] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111)
I20260709 12:28:06.942585 12110 leader_election.cc:290] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:28:06.961112  5312 leader_election.cc:336] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
W20260709 12:28:06.961665  5312 leader_election.cc:336] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:06.962030  5312 leader_election.cc:304] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:28:06.964495 12110 raft_consensus.cc:2749] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:06.965732  5312 consensus_peers.cc:597] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:28:06.967147  5312 leader_election.cc:336] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:06.967528  5312 leader_election.cc:304] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 5c3201048f8346009aefca07428828f3
I20260709 12:28:06.969664 12110 raft_consensus.cc:2749] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:06.985479 12110 raft_consensus.cc:493] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:28:06.986250 12110 raft_consensus.cc:515] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
W20260709 12:28:06.993690  5314 leader_election.cc:336] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111)
I20260709 12:28:06.994418 12110 leader_election.cc:290] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
I20260709 12:28:06.995370 12112 raft_consensus.cc:493] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:28:06.996096 12112 raft_consensus.cc:515] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:06.998287 12112 leader_election.cc:290] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:28:07.003118  5314 leader_election.cc:336] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111)
W20260709 12:28:07.023552  5312 leader_election.cc:336] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:07.024031  5312 leader_election.cc:304] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:28:07.024750 12110 raft_consensus.cc:2749] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:07.025445  5312 leader_election.cc:336] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:07.025906  5312 leader_election.cc:304] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:28:07.035399 12110 raft_consensus.cc:2749] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:07.038127  5312 leader_election.cc:336] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:07.126562 12110 raft_consensus.cc:493] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:28:07.127357 12110 raft_consensus.cc:515] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:07.129838 12110 leader_election.cc:290] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:28:07.141057  5314 leader_election.cc:336] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111)
W20260709 12:28:07.142815  5312 leader_election.cc:336] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:07.143255  5312 leader_election.cc:304] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:28:07.145105 12112 raft_consensus.cc:2749] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:07.149886  5312 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:28:07.175637 12110 raft_consensus.cc:493] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:28:07.176651 12110 raft_consensus.cc:515] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:07.179224 12110 leader_election.cc:290] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:28:07.187772  5314 leader_election.cc:336] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111)
W20260709 12:28:07.192744  5312 leader_election.cc:336] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:07.193176  5312 leader_election.cc:304] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 49277a7653524787a715232ab908bbc6, 5c3201048f8346009aefca07428828f3
I20260709 12:28:07.194065 12110 raft_consensus.cc:2749] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:28:07.229852  5312 consensus_peers.cc:597] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:07.279124 12110 raft_consensus.cc:493] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:28:07.280099 12110 raft_consensus.cc:515] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } attrs { replace: true } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:07.283593 12110 leader_election.cc:290] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077), 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:28:07.301234  5314 leader_election.cc:336] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111)
W20260709 12:28:07.302379  5312 leader_election.cc:336] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
I20260709 12:28:07.302803  5312 leader_election.cc:304] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:28:07.303571 12110 raft_consensus.cc:2749] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:07.304526  5312 leader_election.cc:336] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:07.335215 12120 raft_consensus.cc:493] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 49277a7653524787a715232ab908bbc6)
I20260709 12:28:07.336072 12120 raft_consensus.cc:515] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: false } }
I20260709 12:28:07.342825 12120 leader_election.cc:290] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599), 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049)
W20260709 12:28:07.343686  5312 leader_election.cc:336] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111)
W20260709 12:28:07.344240  5314 leader_election.cc:336] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111)
I20260709 12:28:07.344655  5314 leader_election.cc:304] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe960c35c84451faf4e5b5c9233fa39; no voters: 3a5aeedd11fe49938079fe86cb01232c, 49277a7653524787a715232ab908bbc6
I20260709 12:28:07.345494 12120 raft_consensus.cc:2749] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:28:07.435508  5312 consensus_peers.cc:597] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:07.460999  5312 consensus_peers.cc:597] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:07.585933  5312 consensus_peers.cc:597] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:07.608409  5312 consensus_peers.cc:597] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:07.674818  5312 consensus_peers.cc:597] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:07.715583  5312 consensus_peers.cc:597] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:07.716281  5312 consensus_peers.cc:597] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:07.790086  5312 consensus_peers.cc:597] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:07.807636  5312 consensus_peers.cc:597] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:07.833654  5314 consensus_peers.cc:597] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:07.846076  5312 consensus_peers.cc:597] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:07.864849  5312 consensus_peers.cc:597] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:07.880417  5312 consensus_peers.cc:597] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:07.894148  5312 consensus_peers.cc:597] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:07.935973  5314 consensus_peers.cc:597] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:07.987538  5312 consensus_peers.cc:597] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:07.993386  5312 consensus_peers.cc:597] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:08.000686  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 3a5aeedd11fe49938079fe86cb01232c (127.3.89.194:42599): Couldn't send request to peer 3a5aeedd11fe49938079fe86cb01232c. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:42599: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:08.043445  3431 tablet_server.cc:196] TabletServer@127.3.89.195:0 shutdown complete.
W20260709 12:28:08.045655  5312 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111) [suppressed 635 similar messages]
W20260709 12:28:08.049657  5312 consensus_peers.cc:597] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): Couldn't send request to peer 5c3201048f8346009aefca07428828f3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:28:08.077590  5314 consensus_peers.cc:597] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:08.118368  5314 consensus_peers.cc:597] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:08.215744  5314 consensus_peers.cc:597] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:08.231769  3431 tablet_server.cc:179] TabletServer@127.3.89.196:0 shutting down...
W20260709 12:28:08.253337  5314 consensus_peers.cc:597] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:08.258667  5314 consensus_peers.cc:597] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:08.270978  5314 consensus_peers.cc:597] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:08.293648  5314 consensus_peers.cc:597] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:08.302462  5314 consensus_peers.cc:597] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:08.306159  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:08.307210  3431 tablet_replica.cc:333] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.308796  3431 raft_consensus.cc:2243] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:08.309834  3431 raft_consensus.cc:2272] T 485fe0cee4b248f4802d3c15dc97a877 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.313195  3431 tablet_replica.cc:333] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.314342  3431 raft_consensus.cc:2243] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.316385  3431 pending_rounds.cc:70] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39: Trying to abort 1 pending ops.
I20260709 12:28:08.316663  3431 pending_rounds.cc:77] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39: Aborting op as it isn't in flight: id { term: 20 index: 15 } timestamp: 7305625948245934080 op_type: NO_OP noop_request { }
I20260709 12:28:08.317186  3431 raft_consensus.cc:2889] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:08.317622  3431 raft_consensus.cc:2272] T 13ebb8f343c743a1a28db116a4881004 P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.321221  3431 tablet_replica.cc:333] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.322203  3431 raft_consensus.cc:2243] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:08.323227  3431 raft_consensus.cc:2272] T 86297ef907dd4e3590805d3cf15efaca P dbe960c35c84451faf4e5b5c9233fa39 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.326584  3431 tablet_replica.cc:333] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.327509  3431 raft_consensus.cc:2243] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:08.328967  3431 raft_consensus.cc:2272] T 9702ea78030547e5a2f6fffe805075da P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.332238  3431 tablet_replica.cc:333] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.333029  3431 raft_consensus.cc:2243] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:08.334117  3431 raft_consensus.cc:2272] T f54c73ae37134549b12188dddc4fa794 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.338095  3431 tablet_replica.cc:333] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.339107  3431 raft_consensus.cc:2243] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:08.340117  3431 raft_consensus.cc:2272] T 1b7b0c6c811e455abe898cb4d21acad4 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.343444  3431 tablet_replica.cc:333] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
W20260709 12:28:08.344125  5314 consensus_peers.cc:597] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:08.344419  3431 raft_consensus.cc:2243] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:08.345261  3431 raft_consensus.cc:2272] T 4f682ea56c3b4f2bb6e9413f3d98c6fd P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.350142  3431 tablet_replica.cc:333] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.351287  3431 raft_consensus.cc:2243] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.352775  3431 raft_consensus.cc:2272] T b8e62ff81eb34853811314530131292b P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.356194  3431 tablet_replica.cc:333] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
W20260709 12:28:08.356930  5314 consensus_peers.cc:597] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 -> Peer 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): Couldn't send request to peer 49277a7653524787a715232ab908bbc6. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:08.357146  3431 raft_consensus.cc:2243] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:08.358029  3431 raft_consensus.cc:2272] T 8fc78aa9a3bf48afb52246b50ad414eb P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.360672  3431 tablet_replica.cc:333] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.361284  3431 raft_consensus.cc:2243] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.362471  3431 raft_consensus.cc:2272] T f173b0d5264e4eff8b1704becc555780 P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.365249  3431 tablet_replica.cc:333] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.366042  3431 raft_consensus.cc:2243] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:08.367348  3431 raft_consensus.cc:2272] T aba6aa625d5a49658718de09523b8a9b P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.371100  3431 tablet_replica.cc:333] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.371985  3431 raft_consensus.cc:2243] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.373281  3431 raft_consensus.cc:2272] T 12796fc41f234615a607ba222506e893 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.376168  3431 tablet_replica.cc:333] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.376821  3431 raft_consensus.cc:2243] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.377908  3431 raft_consensus.cc:2272] T 70fb561fb8b446969622ee2193d90917 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.380312  3431 tablet_replica.cc:333] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.380952  3431 raft_consensus.cc:2243] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.382158  3431 raft_consensus.cc:2272] T 23fa1ebbcb7f4a7e9bcf064d4198f48e P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.384755  3431 tablet_replica.cc:333] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.385308  3431 raft_consensus.cc:2243] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.386430  3431 raft_consensus.cc:2272] T d410e14541184b6bbaff38737ce4bc02 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.388514  3431 tablet_replica.cc:333] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.389101  3431 raft_consensus.cc:2243] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.390148  3431 raft_consensus.cc:2272] T 3e20c8bae3534224bb7a82db42cab1a5 P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.392429  3431 tablet_replica.cc:333] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.392994  3431 raft_consensus.cc:2243] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.394173  3431 raft_consensus.cc:2272] T 28f87749718d445ba1615ff33028c4d8 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.396239  3431 tablet_replica.cc:333] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.396858  3431 raft_consensus.cc:2243] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.397891  3431 raft_consensus.cc:2272] T 34a0b6acf7c6497e8ef8ffd63b4127d6 P dbe960c35c84451faf4e5b5c9233fa39 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.400297  3431 tablet_replica.cc:333] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.400967  3431 raft_consensus.cc:2243] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.401994  3431 raft_consensus.cc:2272] T 1fdfc68cfdd1411cad1b4e974e65067a P dbe960c35c84451faf4e5b5c9233fa39 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.404088  3431 tablet_replica.cc:333] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.404805  3431 raft_consensus.cc:2243] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.406309  3431 raft_consensus.cc:2272] T dafe783bf87845d6bb64a279ed7d0f75 P dbe960c35c84451faf4e5b5c9233fa39 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.408931  3431 tablet_replica.cc:333] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.409637  3431 raft_consensus.cc:2243] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.411056  3431 raft_consensus.cc:2272] T 2d6e5f40333a47559cd852e4dfa7c86b P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.414811  3431 tablet_replica.cc:333] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.415650  3431 raft_consensus.cc:2243] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.417059  3431 raft_consensus.cc:2272] T 718b286f0ba24a90af47c49ec0031b11 P dbe960c35c84451faf4e5b5c9233fa39 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.419809  3431 tablet_replica.cc:333] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.420490  3431 raft_consensus.cc:2243] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.421597  3431 raft_consensus.cc:2272] T ab695b6fa8e145c7926a540c3ba5b84e P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.423852  3431 tablet_replica.cc:333] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.424407  3431 raft_consensus.cc:2243] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:08.424948  3431 raft_consensus.cc:2272] T 23027d24ef1943bdbea76085a97b055c P dbe960c35c84451faf4e5b5c9233fa39 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.427587  3431 tablet_replica.cc:333] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.428257  3431 raft_consensus.cc:2243] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.429196  3431 raft_consensus.cc:2272] T 3cbe5bef0f514b01ac552ff54d4d13be P dbe960c35c84451faf4e5b5c9233fa39 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.432381  3431 tablet_replica.cc:333] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.433024  3431 raft_consensus.cc:2243] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:08.433662  3431 raft_consensus.cc:2272] T 9ca7c574e14c4faeac378ee00c0a261b P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.436081  3431 tablet_replica.cc:333] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.436681  3431 raft_consensus.cc:2243] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.437793  3431 raft_consensus.cc:2272] T c6f49e7bc416420cb85a6d4319f46c4d P dbe960c35c84451faf4e5b5c9233fa39 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.440196  3431 tablet_replica.cc:333] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.440809  3431 raft_consensus.cc:2243] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.442122  3431 raft_consensus.cc:2272] T 559316f720c74aa1b50405e2ed791f6a P dbe960c35c84451faf4e5b5c9233fa39 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.444629  3431 tablet_replica.cc:333] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.445241  3431 raft_consensus.cc:2243] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.446326  3431 raft_consensus.cc:2272] T e8a74137ba104fe0b465cab6399c4940 P dbe960c35c84451faf4e5b5c9233fa39 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.449050  3431 tablet_replica.cc:333] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.449760  3431 raft_consensus.cc:2243] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:08.450476  3431 raft_consensus.cc:2272] T 3be428c36f9242c6a2b8aa853f2c8b2c P dbe960c35c84451faf4e5b5c9233fa39 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.453343  3431 tablet_replica.cc:333] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.453998  3431 raft_consensus.cc:2243] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:08.454758  3431 raft_consensus.cc:2272] T c3d4ce22fd7947598cc1d1c7732285da P dbe960c35c84451faf4e5b5c9233fa39 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.457028  3431 tablet_replica.cc:333] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.457607  3431 raft_consensus.cc:2243] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.458652  3431 raft_consensus.cc:2272] T ea533a99927c40d48b894cf00a75301d P dbe960c35c84451faf4e5b5c9233fa39 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:08.461264  3431 tablet_replica.cc:333] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.461843  3431 raft_consensus.cc:2243] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Raft consensus shutting down.
I20260709 12:28:08.462252  3431 pending_rounds.cc:70] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: Trying to abort 1 pending ops.
I20260709 12:28:08.462504  3431 pending_rounds.cc:77] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7305625129141669888 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "53908897ce974579947af90acaf0aed7" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } } } new_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "5c3201048f8346009aefca07428828f3" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45077 } } peers { permanent_uuid: "3a5aeedd11fe49938079fe86cb01232c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42599 } } peers { permanent_uuid: "49277a7653524787a715232ab908bbc6" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41049 } attrs { replace: true } } peers { permanent_uuid: "dbe960c35c84451faf4e5b5c9233fa39" member_type: NON_VOTER last_known_addr { host: "127.3.89.196" port: 39067 } attrs { promote: true } } } }
I20260709 12:28:08.463306  3431 raft_consensus.cc:2917] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 LEARNER]: Aborting config change with OpId 1.2: Aborted: Op aborted
I20260709 12:28:08.463686  3431 raft_consensus.cc:2889] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:28:08.463953  3431 raft_consensus.cc:2272] T 53908897ce974579947af90acaf0aed7 P dbe960c35c84451faf4e5b5c9233fa39 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260709 12:28:08.466261  3431 tablet_replica.cc:333] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39: stopping tablet replica
I20260709 12:28:08.466732  3431 raft_consensus.cc:2243] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:28:08.467823  3431 raft_consensus.cc:2272] T a8ee74acbd5c45f0934f6f37d2154bc7 P dbe960c35c84451faf4e5b5c9233fa39 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:09.535931  3431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 5370)
W20260709 12:28:10.536445  3431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 5370)
W20260709 12:28:10.548302  4486 catalog_manager.cc:4755] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 13ebb8f343c743a1a28db116a4881004: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
I20260709 12:28:11.372838  3431 tablet_server.cc:196] TabletServer@127.3.89.196:0 shutdown complete.
I20260709 12:28:11.451169  3431 master.cc:562] Master@127.3.89.254:36317 shutting down...
W20260709 12:28:11.487576  4550 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:28:11.487823  4550 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:28:11.488073  4550 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:28:19.128232  4486 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111) [suppressed 29 similar messages]
W20260709 12:28:19.132290  4486 catalog_manager.cc:4755] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 2d6e5f40333a47559cd852e4dfa7c86b: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:25.139974  4533 debug-util.cc:398] Leaking SignalData structure 0x7b08001060c0 after lost signal to thread 3434
W20260709 12:28:25.140489  4533 debug-util.cc:398] Leaking SignalData structure 0x7b080018c6c0 after lost signal to thread 4536
W20260709 12:28:26.983295  4486 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260709 12:28:26.987300  4486 catalog_manager.cc:4755] TS 5c3201048f8346009aefca07428828f3 (127.3.89.193:45077): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 13ebb8f343c743a1a28db116a4881004: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45077: connect: Connection refused (error 111)
W20260709 12:28:44.211242  4488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111)
W20260709 12:28:44.215008  4488 catalog_manager.cc:4755] TS 49277a7653524787a715232ab908bbc6 (127.3.89.195:41049): ChangeConfig:REMOVE_PEER RPC failed for tablet 3be428c36f9242c6a2b8aa853f2c8b2c: Network error: Client connection negotiation failed: client connection to 127.3.89.195:41049: connect: Connection refused (error 111)
I20260709 12:28:44.704823  3431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:44.705984  3431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:44.706408  3431 tablet_replica.cc:333] T 00000000000000000000000000000000 P edba2778b94f4e5aa07d5f710879b375: stopping tablet replica
I20260709 12:28:44.954119  3431 master.cc:584] Master@127.3.89.254:36317 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (269552 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:28:45.081681  3431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.3.89.254:45675
I20260709 12:28:45.082935  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:45.089128 12134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:45.089231 12135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:45.090018 12137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:45.090590  3431 server_base.cc:1061] running on GCE node
I20260709 12:28:45.094062  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:45.094297  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:45.094449  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600125094433 us; error 0 us; skew 500 ppm
I20260709 12:28:45.094969  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:45.097476  3431 webserver.cc:533] Webserver started at http://127.3.89.254:44639/ using document root <none> and password file <none>
I20260709 12:28:45.098135  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:45.098343  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:45.098618  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:45.099905  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/master-0-root/instance:
uuid: "6e9b094ea1e7401993ec5ad7a7e427d6"
format_stamp: "Formatted at 2026-07-09 12:28:45 on dist-test-slave-3hrd"
I20260709 12:28:45.104538  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:28:45.108400 12142 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:45.109408  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:28:45.109823  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/master-0-root
uuid: "6e9b094ea1e7401993ec5ad7a7e427d6"
format_stamp: "Formatted at 2026-07-09 12:28:45 on dist-test-slave-3hrd"
I20260709 12:28:45.110239  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:45.131466  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:45.132773  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:45.176730  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:45675
I20260709 12:28:45.176873 12193 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.254:45675 every 8 connection(s)
I20260709 12:28:45.181735 12194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.194173 12194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6: Bootstrap starting.
I20260709 12:28:45.199293 12194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.203912 12194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6: No bootstrap required, opened a new log
I20260709 12:28:45.206655 12194 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9b094ea1e7401993ec5ad7a7e427d6" member_type: VOTER }
I20260709 12:28:45.207111 12194 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.207387 12194 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e9b094ea1e7401993ec5ad7a7e427d6, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.208092 12194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9b094ea1e7401993ec5ad7a7e427d6" member_type: VOTER }
I20260709 12:28:45.208573 12194 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:28:45.208824 12194 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:28:45.209103 12194 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:45.214138 12194 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9b094ea1e7401993ec5ad7a7e427d6" member_type: VOTER }
I20260709 12:28:45.214675 12194 leader_election.cc:304] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e9b094ea1e7401993ec5ad7a7e427d6; no voters: 
I20260709 12:28:45.215768 12194 leader_election.cc:290] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:28:45.216192 12197 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:45.217593 12197 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [term 1 LEADER]: Becoming Leader. State: Replica: 6e9b094ea1e7401993ec5ad7a7e427d6, State: Running, Role: LEADER
I20260709 12:28:45.218320 12197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9b094ea1e7401993ec5ad7a7e427d6" member_type: VOTER }
I20260709 12:28:45.218914 12194 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:28:45.227150 12199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e9b094ea1e7401993ec5ad7a7e427d6. Latest consensus state: current_term: 1 leader_uuid: "6e9b094ea1e7401993ec5ad7a7e427d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9b094ea1e7401993ec5ad7a7e427d6" member_type: VOTER } }
I20260709 12:28:45.227414 12198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e9b094ea1e7401993ec5ad7a7e427d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9b094ea1e7401993ec5ad7a7e427d6" member_type: VOTER } }
I20260709 12:28:45.227993 12199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:45.228021 12198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:45.230150 12207 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:28:45.238358 12207 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:28:45.239452  3431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:28:45.248013 12207 catalog_manager.cc:1383] Generated new cluster ID: 2cd0ca3ba3ac44e3833eb59df4d6300b
I20260709 12:28:45.248322 12207 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:28:45.275105 12207 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:28:45.276467 12207 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:28:45.292135 12207 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6: Generated new TSK 0
I20260709 12:28:45.292768 12207 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:28:45.306267  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:45.312502 12215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:45.314169 12216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:45.315084 12218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:45.316397  3431 server_base.cc:1061] running on GCE node
I20260709 12:28:45.318883  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:45.319078  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:45.319202  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600125319192 us; error 0 us; skew 500 ppm
I20260709 12:28:45.319689  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:45.322325  3431 webserver.cc:533] Webserver started at http://127.3.89.193:34689/ using document root <none> and password file <none>
I20260709 12:28:45.322808  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:45.322974  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:45.323181  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:45.324391  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-0-root/instance:
uuid: "1dcded10caa0493da818e67e13176979"
format_stamp: "Formatted at 2026-07-09 12:28:45 on dist-test-slave-3hrd"
I20260709 12:28:45.329144  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:28:45.332938 12223 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:45.334024  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 12:28:45.334457  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-0-root
uuid: "1dcded10caa0493da818e67e13176979"
format_stamp: "Formatted at 2026-07-09 12:28:45 on dist-test-slave-3hrd"
I20260709 12:28:45.334972  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:45.346791  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:45.348055  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:45.349658  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:45.352130  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:45.352362  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:45.352596  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:45.352787  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:45.400660  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:37927
I20260709 12:28:45.400802 12285 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.193:37927 every 8 connection(s)
I20260709 12:28:45.406314  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:45.413861 12290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:45.414716 12291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:45.417984  3431 server_base.cc:1061] running on GCE node
W20260709 12:28:45.418464 12294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:45.421924 12286 heartbeater.cc:344] Connected to a master server at 127.3.89.254:45675
I20260709 12:28:45.421926  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:45.422492  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:45.422533 12286 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:45.422750  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600125422728 us; error 0 us; skew 500 ppm
I20260709 12:28:45.423666  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:45.423689 12286 heartbeater.cc:507] Master 127.3.89.254:45675 requested a full tablet report, sending...
I20260709 12:28:45.426596 12159 ts_manager.cc:194] Registered new tserver with Master: 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:45.426632  3431 webserver.cc:533] Webserver started at http://127.3.89.194:42401/ using document root <none> and password file <none>
I20260709 12:28:45.427315  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:45.427544  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:45.427819  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:45.428632 12159 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57016
I20260709 12:28:45.429198  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-1-root/instance:
uuid: "4e02e5f693ce49a3b161ad90848be7d4"
format_stamp: "Formatted at 2026-07-09 12:28:45 on dist-test-slave-3hrd"
I20260709 12:28:45.434185  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:28:45.438598 12298 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:45.439584  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:28:45.439857  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-1-root
uuid: "4e02e5f693ce49a3b161ad90848be7d4"
format_stamp: "Formatted at 2026-07-09 12:28:45 on dist-test-slave-3hrd"
I20260709 12:28:45.440110  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:45.453364  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:45.454828  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:45.456271  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:45.458662  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:45.458844  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:45.459138  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:45.459290  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:45.502569  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:33269
I20260709 12:28:45.502681 12360 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.194:33269 every 8 connection(s)
I20260709 12:28:45.508000  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:45.516850 12364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:45.517369 12365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:45.520931 12368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:45.520956  3431 server_base.cc:1061] running on GCE node
I20260709 12:28:45.521257 12361 heartbeater.cc:344] Connected to a master server at 127.3.89.254:45675
I20260709 12:28:45.521855 12361 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:45.522341  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:45.522652  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:45.522670 12361 heartbeater.cc:507] Master 127.3.89.254:45675 requested a full tablet report, sending...
I20260709 12:28:45.522920  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600125522902 us; error 0 us; skew 500 ppm
I20260709 12:28:45.523600  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:45.525202 12159 ts_manager.cc:194] Registered new tserver with Master: 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:45.526373  3431 webserver.cc:533] Webserver started at http://127.3.89.195:42293/ using document root <none> and password file <none>
I20260709 12:28:45.526815 12159 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57018
I20260709 12:28:45.526943  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:45.527264  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:45.527552  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:45.528915  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-2-root/instance:
uuid: "a2104c8a9b144a419ccb7c07e3f02596"
format_stamp: "Formatted at 2026-07-09 12:28:45 on dist-test-slave-3hrd"
I20260709 12:28:45.533459  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.003s
I20260709 12:28:45.537112 12372 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:45.538100  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:28:45.538422  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-2-root
uuid: "a2104c8a9b144a419ccb7c07e3f02596"
format_stamp: "Formatted at 2026-07-09 12:28:45 on dist-test-slave-3hrd"
I20260709 12:28:45.538674  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:45.548226  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:45.549507  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:45.551149  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:45.553514  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:45.553803  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:45.554085  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:45.554253  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:45.599355  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:40155
I20260709 12:28:45.599450 12434 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.195:40155 every 8 connection(s)
I20260709 12:28:45.604765  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:45.612093 12438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:45.614616 12439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:45.615445 12441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:45.616070  3431 server_base.cc:1061] running on GCE node
I20260709 12:28:45.617166  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:45.617441  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:45.617667  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600125617650 us; error 0 us; skew 500 ppm
I20260709 12:28:45.617865 12435 heartbeater.cc:344] Connected to a master server at 127.3.89.254:45675
I20260709 12:28:45.618276 12435 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:45.618456  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:45.619207 12435 heartbeater.cc:507] Master 127.3.89.254:45675 requested a full tablet report, sending...
I20260709 12:28:45.621583  3431 webserver.cc:533] Webserver started at http://127.3.89.196:44735/ using document root <none> and password file <none>
I20260709 12:28:45.622015 12159 ts_manager.cc:194] Registered new tserver with Master: a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:45.622305  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:45.622610  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:45.622934  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:45.623903 12159 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57026
I20260709 12:28:45.624526  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-3-root/instance:
uuid: "a800ae68e91f43a794e326badca3320a"
format_stamp: "Formatted at 2026-07-09 12:28:45 on dist-test-slave-3hrd"
I20260709 12:28:45.630043  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:28:45.633792 12446 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:45.634662  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:28:45.635046  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-3-root
uuid: "a800ae68e91f43a794e326badca3320a"
format_stamp: "Formatted at 2026-07-09 12:28:45 on dist-test-slave-3hrd"
I20260709 12:28:45.635317  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:45.647588  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:45.648962  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:45.650576  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:45.652922  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:45.653163  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:45.653404  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:45.653604  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:45.699843  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.196:45677
I20260709 12:28:45.699937 12508 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.196:45677 every 8 connection(s)
I20260709 12:28:45.721489 12509 heartbeater.cc:344] Connected to a master server at 127.3.89.254:45675
I20260709 12:28:45.722033 12509 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:45.722977 12509 heartbeater.cc:507] Master 127.3.89.254:45675 requested a full tablet report, sending...
I20260709 12:28:45.725224 12159 ts_manager.cc:194] Registered new tserver with Master: a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:45.726402  3431 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.022939241s
I20260709 12:28:45.727042 12159 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57042
I20260709 12:28:45.756230 12159 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57044:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:28:46.014114 12251 tablet_service.cc:1511] Processing CreateTablet for tablet d0c8b89ad96240b588abe04f13ea2f9d (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:28:46.015836 12251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c8b89ad96240b588abe04f13ea2f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.027530 12400 tablet_service.cc:1511] Processing CreateTablet for tablet d0c8b89ad96240b588abe04f13ea2f9d (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:28:46.027865 12399 tablet_service.cc:1511] Processing CreateTablet for tablet fd891675417c4265b77695b4227d980f (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:28:46.029095 12400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c8b89ad96240b588abe04f13ea2f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.029814 12399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd891675417c4265b77695b4227d980f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.035873 12398 tablet_service.cc:1511] Processing CreateTablet for tablet 2f01aed6d25447b0be01b95f4518c8ef (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:28:46.041942 12250 tablet_service.cc:1511] Processing CreateTablet for tablet f5ae8840033347bea42f3c0ceb4d316a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:28:46.043568 12250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5ae8840033347bea42f3c0ceb4d316a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.043349 12326 tablet_service.cc:1511] Processing CreateTablet for tablet d0c8b89ad96240b588abe04f13ea2f9d (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:28:46.044870 12326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c8b89ad96240b588abe04f13ea2f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.045010 12323 tablet_service.cc:1511] Processing CreateTablet for tablet 7540665883f74d94b96d1631ac30970d (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:28:46.046718 12323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7540665883f74d94b96d1631ac30970d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.046994 12249 tablet_service.cc:1511] Processing CreateTablet for tablet fd891675417c4265b77695b4227d980f (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:28:46.048406 12249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd891675417c4265b77695b4227d980f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.050956 12248 tablet_service.cc:1511] Processing CreateTablet for tablet 7540665883f74d94b96d1631ac30970d (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:28:46.052500 12248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7540665883f74d94b96d1631ac30970d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.054467 12324 tablet_service.cc:1511] Processing CreateTablet for tablet 2f01aed6d25447b0be01b95f4518c8ef (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:28:46.054893 12247 tablet_service.cc:1511] Processing CreateTablet for tablet b0b789be887a492fb0e51b08f2475a97 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:28:46.055634 12322 tablet_service.cc:1511] Processing CreateTablet for tablet b0b789be887a492fb0e51b08f2475a97 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:28:46.056413 12247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0b789be887a492fb0e51b08f2475a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.056501 12324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f01aed6d25447b0be01b95f4518c8ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.057386 12322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0b789be887a492fb0e51b08f2475a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.058820 12246 tablet_service.cc:1511] Processing CreateTablet for tablet 81f0d8651a6144ca9fd21ccc0f57edd9 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:28:46.060340 12246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f0d8651a6144ca9fd21ccc0f57edd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.063002 12245 tablet_service.cc:1511] Processing CreateTablet for tablet 05dfc5c361a94fe5af3e9a5462a0b12f (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:28:46.064639 12245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05dfc5c361a94fe5af3e9a5462a0b12f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.051916 12325 tablet_service.cc:1511] Processing CreateTablet for tablet f5ae8840033347bea42f3c0ceb4d316a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:28:46.065795 12321 tablet_service.cc:1511] Processing CreateTablet for tablet 0db639fdafc34f269b3d493540075616 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:28:46.066531 12325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5ae8840033347bea42f3c0ceb4d316a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.067241 12321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0db639fdafc34f269b3d493540075616. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.067828 12398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f01aed6d25447b0be01b95f4518c8ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.072659 12244 tablet_service.cc:1511] Processing CreateTablet for tablet 307ee3a81a6649ebb96772cbd5e29f92 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:28:46.073292 12242 tablet_service.cc:1511] Processing CreateTablet for tablet 69e3b47e111a479f8582c270c1445658 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:28:46.074317 12244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 307ee3a81a6649ebb96772cbd5e29f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.075038 12242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69e3b47e111a479f8582c270c1445658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.076185 12319 tablet_service.cc:1511] Processing CreateTablet for tablet 307ee3a81a6649ebb96772cbd5e29f92 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:28:46.077499 12320 tablet_service.cc:1511] Processing CreateTablet for tablet 05dfc5c361a94fe5af3e9a5462a0b12f (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:28:46.076187 12243 tablet_service.cc:1511] Processing CreateTablet for tablet 8b89bcc99e094d1bbaf83979c2b78081 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:28:46.079531 12243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b89bcc99e094d1bbaf83979c2b78081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.082020 12319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 307ee3a81a6649ebb96772cbd5e29f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.083084 12320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05dfc5c361a94fe5af3e9a5462a0b12f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.140903 12318 tablet_service.cc:1511] Processing CreateTablet for tablet 33683dafd0e7419e8f52bdf98e28e33c (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:28:46.142693 12318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33683dafd0e7419e8f52bdf98e28e33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.147148 12397 tablet_service.cc:1511] Processing CreateTablet for tablet 7540665883f74d94b96d1631ac30970d (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:28:46.147716 12317 tablet_service.cc:1511] Processing CreateTablet for tablet 69e3b47e111a479f8582c270c1445658 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:28:46.148728 12397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7540665883f74d94b96d1631ac30970d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.149118 12317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69e3b47e111a479f8582c270c1445658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.150871 12396 tablet_service.cc:1511] Processing CreateTablet for tablet 81f0d8651a6144ca9fd21ccc0f57edd9 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:28:46.152410 12396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f0d8651a6144ca9fd21ccc0f57edd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.154965 12395 tablet_service.cc:1511] Processing CreateTablet for tablet 0db639fdafc34f269b3d493540075616 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:28:46.156800 12395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0db639fdafc34f269b3d493540075616. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.158926 12394 tablet_service.cc:1511] Processing CreateTablet for tablet 05dfc5c361a94fe5af3e9a5462a0b12f (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:28:46.160431 12394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05dfc5c361a94fe5af3e9a5462a0b12f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.162873 12393 tablet_service.cc:1511] Processing CreateTablet for tablet 8b89bcc99e094d1bbaf83979c2b78081 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:28:46.164426 12393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b89bcc99e094d1bbaf83979c2b78081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.166814 12391 tablet_service.cc:1511] Processing CreateTablet for tablet 69e3b47e111a479f8582c270c1445658 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:28:46.172272 12392 tablet_service.cc:1511] Processing CreateTablet for tablet 33683dafd0e7419e8f52bdf98e28e33c (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:28:46.174835 12472 tablet_service.cc:1511] Processing CreateTablet for tablet 2f01aed6d25447b0be01b95f4518c8ef (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:28:46.175127 12474 tablet_service.cc:1511] Processing CreateTablet for tablet f5ae8840033347bea42f3c0ceb4d316a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:28:46.176301 12472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f01aed6d25447b0be01b95f4518c8ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.177062 12474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5ae8840033347bea42f3c0ceb4d316a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.178506 12471 tablet_service.cc:1511] Processing CreateTablet for tablet b0b789be887a492fb0e51b08f2475a97 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:28:46.179934 12471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0b789be887a492fb0e51b08f2475a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.182950 12470 tablet_service.cc:1511] Processing CreateTablet for tablet 81f0d8651a6144ca9fd21ccc0f57edd9 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:28:46.183389 12467 tablet_service.cc:1511] Processing CreateTablet for tablet 8b89bcc99e094d1bbaf83979c2b78081 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:28:46.183959 12244 tablet_service.cc:1511] Processing CreateTablet for tablet 485001bca51c4145939dc8bd38b1169a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:28:46.184433 12470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f0d8651a6144ca9fd21ccc0f57edd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.185876 12469 tablet_service.cc:1511] Processing CreateTablet for tablet 0db639fdafc34f269b3d493540075616 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:28:46.186755 12466 tablet_service.cc:1511] Processing CreateTablet for tablet 33683dafd0e7419e8f52bdf98e28e33c (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:28:46.187403 12469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0db639fdafc34f269b3d493540075616. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.188284 12466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33683dafd0e7419e8f52bdf98e28e33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.189548 12467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b89bcc99e094d1bbaf83979c2b78081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.194960 12473 tablet_service.cc:1511] Processing CreateTablet for tablet fd891675417c4265b77695b4227d980f (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:28:46.196516 12473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd891675417c4265b77695b4227d980f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.198702 12468 tablet_service.cc:1511] Processing CreateTablet for tablet 307ee3a81a6649ebb96772cbd5e29f92 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:28:46.200219 12468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 307ee3a81a6649ebb96772cbd5e29f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.204680 12392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33683dafd0e7419e8f52bdf98e28e33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.206135 12391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69e3b47e111a479f8582c270c1445658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.206533 12244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 485001bca51c4145939dc8bd38b1169a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.208353 12319 tablet_service.cc:1511] Processing CreateTablet for tablet 485001bca51c4145939dc8bd38b1169a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:28:46.237166 12243 tablet_service.cc:1511] Processing CreateTablet for tablet 83d73b1bb20e4e79819072554a1ee076 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:28:46.243544 12243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83d73b1bb20e4e79819072554a1ee076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.245496 12242 tablet_service.cc:1511] Processing CreateTablet for tablet 8d8f5c63c391477fad5c9155f5022b07 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:28:46.255273 12242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d8f5c63c391477fad5c9155f5022b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.254839 12322 tablet_service.cc:1511] Processing CreateTablet for tablet d2a2e0c681f24264a729f89d08042fd9 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:28:46.256984 12465 tablet_service.cc:1511] Processing CreateTablet for tablet 485001bca51c4145939dc8bd38b1169a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:28:46.258775 12465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 485001bca51c4145939dc8bd38b1169a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.260246 12325 tablet_service.cc:1511] Processing CreateTablet for tablet 8d8f5c63c391477fad5c9155f5022b07 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:28:46.262382 12527 tablet_bootstrap.cc:492] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:46.263062 12251 tablet_service.cc:1511] Processing CreateTablet for tablet 93481f7df95d427eaa757fbbe87895d9 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:28:46.270499 12529 tablet_bootstrap.cc:492] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:46.271250 12251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93481f7df95d427eaa757fbbe87895d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.285211 12319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 485001bca51c4145939dc8bd38b1169a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:46.291826 12206 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:46.292435 12529 tablet_bootstrap.cc:654] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.329818 12322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2a2e0c681f24264a729f89d08042fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.344135 12472 tablet_service.cc:1511] Processing CreateTablet for tablet d2a2e0c681f24264a729f89d08042fd9 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:28:46.345834 12472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2a2e0c681f24264a729f89d08042fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.354198 12528 tablet_bootstrap.cc:492] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:46.365725 12472 tablet_service.cc:1511] Processing CreateTablet for tablet 83d73b1bb20e4e79819072554a1ee076 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:28:46.366602 12324 tablet_service.cc:1511] Processing CreateTablet for tablet 93481f7df95d427eaa757fbbe87895d9 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:28:46.367426 12472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83d73b1bb20e4e79819072554a1ee076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.260246 12400 tablet_service.cc:1511] Processing CreateTablet for tablet d2a2e0c681f24264a729f89d08042fd9 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:28:46.374743 12400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2a2e0c681f24264a729f89d08042fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.384780 12392 tablet_service.cc:1511] Processing CreateTablet for tablet 83d73b1bb20e4e79819072554a1ee076 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:28:46.392870 12325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d8f5c63c391477fad5c9155f5022b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.398015 12527 tablet_bootstrap.cc:654] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.400738 12324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93481f7df95d427eaa757fbbe87895d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.401401 12392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83d73b1bb20e4e79819072554a1ee076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.373188 12470 tablet_service.cc:1511] Processing CreateTablet for tablet 93481f7df95d427eaa757fbbe87895d9 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:28:46.423655 12470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93481f7df95d427eaa757fbbe87895d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.424414 12471 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa6f87b452d4a7fbc6803c9fd74e9c3 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:28:46.425673 12529 tablet_bootstrap.cc:492] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:46.426100 12471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa6f87b452d4a7fbc6803c9fd74e9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.426251 12529 ts_tablet_manager.cc:1403] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.156s	user 0.012s	sys 0.015s
I20260709 12:28:46.429478 12529 raft_consensus.cc:359] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:46.430274 12529 raft_consensus.cc:385] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.430569 12529 raft_consensus.cc:740] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.431264 12529 consensus_queue.cc:260] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:46.432924 12531 tablet_bootstrap.cc:492] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:46.438522 12528 tablet_bootstrap.cc:654] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.439469 12529 ts_tablet_manager.cc:1434] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:28:46.440675 12529 tablet_bootstrap.cc:492] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:46.447938 12527 tablet_bootstrap.cc:492] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:46.448674 12527 ts_tablet_manager.cc:1403] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.187s	user 0.011s	sys 0.004s
I20260709 12:28:46.449445 12529 tablet_bootstrap.cc:654] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.451366 12531 tablet_bootstrap.cc:654] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.452139 12527 raft_consensus.cc:359] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:46.453111 12527 raft_consensus.cc:385] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.453423 12527 raft_consensus.cc:740] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.454327 12527 consensus_queue.cc:260] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:46.495846 12246 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa6f87b452d4a7fbc6803c9fd74e9c3 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:28:46.497845 12244 tablet_service.cc:1511] Processing CreateTablet for tablet dd19eddb6c2e45b7acefc5c295d92928 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:28:46.498776 12246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa6f87b452d4a7fbc6803c9fd74e9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.503067 12286 heartbeater.cc:499] Master 127.3.89.254:45675 was elected leader, sending a full tablet report...
I20260709 12:28:46.503703 12244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd19eddb6c2e45b7acefc5c295d92928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.507745 12245 tablet_service.cc:1511] Processing CreateTablet for tablet f4bdda56dfcd42de9ca75e3e5a9b6c73 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:28:46.510977 12250 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff454851114445da8be05eda167efc4 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:28:46.511464 12245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4bdda56dfcd42de9ca75e3e5a9b6c73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.523316 12318 tablet_service.cc:1511] Processing CreateTablet for tablet dd19eddb6c2e45b7acefc5c295d92928 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:28:46.527228 12317 tablet_service.cc:1511] Processing CreateTablet for tablet 74e672f5fa0b4b68ad22e6fd8002d68d (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:28:46.530300 12326 tablet_service.cc:1511] Processing CreateTablet for tablet 60d63f43ed8a4bedb35aa02c779fe4d0 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:28:46.534725 12250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff454851114445da8be05eda167efc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.534842 12361 heartbeater.cc:499] Master 127.3.89.254:45675 was elected leader, sending a full tablet report...
W20260709 12:28:46.546399 12362 tablet.cc:2406] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:28:46.548858 12397 tablet_service.cc:1511] Processing CreateTablet for tablet 8d8f5c63c391477fad5c9155f5022b07 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:28:46.552215 12397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d8f5c63c391477fad5c9155f5022b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.530435 12320 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff454851114445da8be05eda167efc4 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:28:46.562125 12318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd19eddb6c2e45b7acefc5c295d92928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.566062 12320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff454851114445da8be05eda167efc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.569981 12326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60d63f43ed8a4bedb35aa02c779fe4d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.569931 12394 tablet_service.cc:1511] Processing CreateTablet for tablet 74e672f5fa0b4b68ad22e6fd8002d68d (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:28:46.573220 12394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74e672f5fa0b4b68ad22e6fd8002d68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.592244 12529 tablet_bootstrap.cc:492] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:46.605521 12470 tablet_service.cc:1511] Processing CreateTablet for tablet 60d63f43ed8a4bedb35aa02c779fe4d0 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:28:46.606897 12469 tablet_service.cc:1511] Processing CreateTablet for tablet f4bdda56dfcd42de9ca75e3e5a9b6c73 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:28:46.607595 12470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60d63f43ed8a4bedb35aa02c779fe4d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.608508 12469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4bdda56dfcd42de9ca75e3e5a9b6c73. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:46.609043 12143 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 94119.7us
I20260709 12:28:46.610167 12534 raft_consensus.cc:493] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.613780 12317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74e672f5fa0b4b68ad22e6fd8002d68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.614035 12534 raft_consensus.cc:515] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:46.615554 12396 tablet_service.cc:1511] Processing CreateTablet for tablet dd19eddb6c2e45b7acefc5c295d92928 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
W20260709 12:28:46.618191 12146 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 60064.5us
I20260709 12:28:46.594807 12529 ts_tablet_manager.cc:1403] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.153s	user 0.008s	sys 0.005s
I20260709 12:28:46.638042 12396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd19eddb6c2e45b7acefc5c295d92928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.654563 12468 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff454851114445da8be05eda167efc4 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:28:46.657006 12468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff454851114445da8be05eda167efc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.659083 12435 heartbeater.cc:499] Master 127.3.89.254:45675 was elected leader, sending a full tablet report...
I20260709 12:28:46.658879 12466 tablet_service.cc:1511] Processing CreateTablet for tablet 74e672f5fa0b4b68ad22e6fd8002d68d (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:28:46.660506 12466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74e672f5fa0b4b68ad22e6fd8002d68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.660768 12474 tablet_service.cc:1511] Processing CreateTablet for tablet 6a1cc112d1984521b5479f66b4289cce (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:28:46.673291 12247 tablet_service.cc:1511] Processing CreateTablet for tablet 7916cfbec26b4115a9251411482bed41 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:28:46.674963 12247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7916cfbec26b4115a9251411482bed41. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:46.684522 12146 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 64789.3us
I20260709 12:28:46.688311 12474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a1cc112d1984521b5479f66b4289cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.684574 12243 tablet_service.cc:1511] Processing CreateTablet for tablet 6a1cc112d1984521b5479f66b4289cce (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:28:46.695920 12243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a1cc112d1984521b5479f66b4289cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.698747 12531 tablet_bootstrap.cc:492] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:46.699632 12531 ts_tablet_manager.cc:1403] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.277s	user 0.009s	sys 0.007s
I20260709 12:28:46.708460 12391 tablet_service.cc:1511] Processing CreateTablet for tablet 60d63f43ed8a4bedb35aa02c779fe4d0 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:28:46.710309 12527 ts_tablet_manager.cc:1434] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.261s	user 0.006s	sys 0.018s
I20260709 12:28:46.712347 12393 tablet_service.cc:1511] Processing CreateTablet for tablet 7916cfbec26b4115a9251411482bed41 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:28:46.713902 12393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7916cfbec26b4115a9251411482bed41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.720052 12528 tablet_bootstrap.cc:492] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:46.720566 12538 raft_consensus.cc:493] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.724391 12399 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa6f87b452d4a7fbc6803c9fd74e9c3 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:28:46.670814 12534 leader_election.cc:290] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:46.729679 12531 raft_consensus.cc:359] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:46.730871 12509 heartbeater.cc:499] Master 127.3.89.254:45675 was elected leader, sending a full tablet report...
I20260709 12:28:46.731917 12325 tablet_service.cc:1511] Processing CreateTablet for tablet 6a1cc112d1984521b5479f66b4289cce (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:28:46.741068 12325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a1cc112d1984521b5479f66b4289cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.733853 12322 tablet_service.cc:1511] Processing CreateTablet for tablet 7916cfbec26b4115a9251411482bed41 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:28:46.742245 12322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7916cfbec26b4115a9251411482bed41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.744565 12538 raft_consensus.cc:515] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:46.745712 12528 ts_tablet_manager.cc:1403] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.406s	user 0.016s	sys 0.024s
I20260709 12:28:46.747126 12531 raft_consensus.cc:385] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.747560 12531 raft_consensus.cc:740] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.732802 12529 raft_consensus.cc:359] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:46.733682 12527 tablet_bootstrap.cc:492] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:46.764549 12399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa6f87b452d4a7fbc6803c9fd74e9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.762758 12529 raft_consensus.cc:385] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.817795 12529 raft_consensus.cc:740] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.819001 12529 consensus_queue.cc:260] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:46.820559 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f01aed6d25447b0be01b95f4518c8ef" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:46.793743 12391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60d63f43ed8a4bedb35aa02c779fe4d0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:46.822734 12302 leader_election.cc:343] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.777416 12531 consensus_queue.cc:260] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:46.801038 12398 tablet_service.cc:1511] Processing CreateTablet for tablet f4bdda56dfcd42de9ca75e3e5a9b6c73 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:28:46.830492 12398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4bdda56dfcd42de9ca75e3e5a9b6c73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.804359 12538 leader_election.cc:290] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:28:46.831696 12143 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 48833.5us
I20260709 12:28:46.805763 12528 raft_consensus.cc:359] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:46.845880 12528 raft_consensus.cc:385] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.846282 12528 raft_consensus.cc:740] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.847188 12528 consensus_queue.cc:260] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:46.811759 12527 tablet_bootstrap.cc:654] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.858047 12529 ts_tablet_manager.cc:1434] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.223s	user 0.006s	sys 0.003s
I20260709 12:28:46.859418 12529 tablet_bootstrap.cc:492] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:46.865706 12531 ts_tablet_manager.cc:1434] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.166s	user 0.002s	sys 0.013s
I20260709 12:28:46.866976 12531 tablet_bootstrap.cc:492] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:46.898546 12393 tablet_service.cc:1511] Processing CreateTablet for tablet 7cf8c3b9ee65415785e06fb8c085ada7 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:28:46.900566 12395 tablet_service.cc:1511] Processing CreateTablet for tablet fe640756185b42d7874674a29af747da (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:28:46.901747 12393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cf8c3b9ee65415785e06fb8c085ada7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.917711 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f01aed6d25447b0be01b95f4518c8ef" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:46.918603 12484 raft_consensus.cc:2468] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:46.920220 12299 leader_election.cc:304] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:46.921629 12534 raft_consensus.cc:2804] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:46.922066 12534 raft_consensus.cc:493] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:46.922456 12534 raft_consensus.cc:3060] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.935791 12534 raft_consensus.cc:515] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:46.940490 12396 tablet_service.cc:1511] Processing CreateTablet for tablet 13100e49825f4fcfa78791b73b8d7555 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:28:46.946539 12531 tablet_bootstrap.cc:654] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.948884 12395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe640756185b42d7874674a29af747da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.970757 12529 tablet_bootstrap.cc:654] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.979749 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f01aed6d25447b0be01b95f4518c8ef" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:46.980708 12484 raft_consensus.cc:3060] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.975831 12396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13100e49825f4fcfa78791b73b8d7555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.989022 12528 ts_tablet_manager.cc:1434] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.243s	user 0.034s	sys 0.023s
I20260709 12:28:46.990270 12528 tablet_bootstrap.cc:492] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:46.991111 12484 raft_consensus.cc:2468] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:46.992798 12299 leader_election.cc:304] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:46.994553 12534 leader_election.cc:290] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:46.995338 12534 raft_consensus.cc:2804] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:46.995877 12534 raft_consensus.cc:697] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:47.009121 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f01aed6d25447b0be01b95f4518c8ef" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
W20260709 12:28:47.010895 12302 leader_election.cc:343] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.016552 12527 tablet_bootstrap.cc:492] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:47.017316 12527 ts_tablet_manager.cc:1403] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.306s	user 0.014s	sys 0.033s
I20260709 12:28:47.020447 12547 raft_consensus.cc:493] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.020284 12466 tablet_service.cc:1511] Processing CreateTablet for tablet 7cf8c3b9ee65415785e06fb8c085ada7 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:28:47.032707 12466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cf8c3b9ee65415785e06fb8c085ada7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.039650 12318 tablet_service.cc:1511] Processing CreateTablet for tablet fe640756185b42d7874674a29af747da (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:28:47.039650 12245 tablet_service.cc:1511] Processing CreateTablet for tablet 7cf8c3b9ee65415785e06fb8c085ada7 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:28:47.042519 12318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe640756185b42d7874674a29af747da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.043597 12245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cf8c3b9ee65415785e06fb8c085ada7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.046995 12470 tablet_service.cc:1511] Processing CreateTablet for tablet fe640756185b42d7874674a29af747da (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:28:47.048345 12470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe640756185b42d7874674a29af747da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.030973 12547 raft_consensus.cc:515] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:47.050209 12534 consensus_queue.cc:237] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:47.055092 12547 leader_election.cc:290] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:47.060350 12529 tablet_bootstrap.cc:492] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:47.060966 12529 ts_tablet_manager.cc:1403] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.202s	user 0.002s	sys 0.021s
I20260709 12:28:47.069660 12398 tablet_service.cc:1511] Processing CreateTablet for tablet e94567d0d49d4bf0882ca910a8757cbc (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:28:47.071314 12398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e94567d0d49d4bf0882ca910a8757cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.072158 12529 raft_consensus.cc:359] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:47.075625 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:47.077188 12528 tablet_bootstrap.cc:654] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:47.078439 12373 leader_election.cc:343] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.086084 12399 tablet_service.cc:1511] Processing CreateTablet for tablet 1a81175bbecf4ba6b6be50b490564d9b (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:28:47.087230 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:47.087800 12399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a81175bbecf4ba6b6be50b490564d9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.089103 12531 tablet_bootstrap.cc:492] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:47.089776 12531 ts_tablet_manager.cc:1403] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.223s	user 0.014s	sys 0.008s
I20260709 12:28:47.087108 12527 raft_consensus.cc:359] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:47.073351 12529 raft_consensus.cc:385] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.092927 12531 raft_consensus.cc:359] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:47.096577 12529 raft_consensus.cc:740] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.098497 12529 consensus_queue.cc:260] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:47.098647 12317 tablet_service.cc:1511] Processing CreateTablet for tablet 13100e49825f4fcfa78791b73b8d7555 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:28:47.100092 12317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13100e49825f4fcfa78791b73b8d7555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.100888 12529 ts_tablet_manager.cc:1434] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.001s
I20260709 12:28:47.101377 12247 tablet_service.cc:1511] Processing CreateTablet for tablet 13100e49825f4fcfa78791b73b8d7555 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:28:47.095058 12527 raft_consensus.cc:385] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.102594 12527 raft_consensus.cc:740] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
W20260709 12:28:47.094928 12373 leader_election.cc:343] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.103370 12247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13100e49825f4fcfa78791b73b8d7555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.103694 12373 leader_election.cc:304] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:28:47.104817 12538 raft_consensus.cc:2749] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:47.109259 12395 tablet_service.cc:1511] Processing CreateTablet for tablet 87c6d482d9f1412a8f1ec924aa6d1222 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:28:47.103377 12527 consensus_queue.cc:260] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:47.111235 12474 tablet_service.cc:1511] Processing CreateTablet for tablet 228aa7f69257458ba0c2b5e6c950f217 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:28:47.112640 12474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 228aa7f69257458ba0c2b5e6c950f217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.114373 12325 tablet_service.cc:1511] Processing CreateTablet for tablet 228aa7f69257458ba0c2b5e6c950f217 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:28:47.115149 12469 tablet_service.cc:1511] Processing CreateTablet for tablet e94567d0d49d4bf0882ca910a8757cbc (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:28:47.115710 12325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 228aa7f69257458ba0c2b5e6c950f217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.116462 12469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e94567d0d49d4bf0882ca910a8757cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.120620 12322 tablet_service.cc:1511] Processing CreateTablet for tablet 1a81175bbecf4ba6b6be50b490564d9b (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:28:47.122074 12322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a81175bbecf4ba6b6be50b490564d9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.125483 12395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87c6d482d9f1412a8f1ec924aa6d1222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.131220 12243 tablet_service.cc:1511] Processing CreateTablet for tablet 228aa7f69257458ba0c2b5e6c950f217 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:28:47.132642 12243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 228aa7f69257458ba0c2b5e6c950f217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.097077 12531 raft_consensus.cc:385] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.134913 12531 raft_consensus.cc:740] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.135870 12531 consensus_queue.cc:260] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:47.138638 12531 ts_tablet_manager.cc:1434] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.001s
I20260709 12:28:47.102058 12529 tablet_bootstrap.cc:492] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:47.141505 12244 tablet_service.cc:1511] Processing CreateTablet for tablet e94567d0d49d4bf0882ca910a8757cbc (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:28:47.142928 12244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e94567d0d49d4bf0882ca910a8757cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.143898 12527 ts_tablet_manager.cc:1434] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.123s	user 0.006s	sys 0.003s
I20260709 12:28:47.165752 12396 tablet_service.cc:1511] Processing CreateTablet for tablet c67e469fc10a43bf9660c48b5ae1335a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:28:47.165753 12468 tablet_service.cc:1511] Processing CreateTablet for tablet 1a81175bbecf4ba6b6be50b490564d9b (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:28:47.168080 12468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a81175bbecf4ba6b6be50b490564d9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.159775 12531 tablet_bootstrap.cc:492] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:47.175992 12396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c67e469fc10a43bf9660c48b5ae1335a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.177906 12326 tablet_service.cc:1511] Processing CreateTablet for tablet 87c6d482d9f1412a8f1ec924aa6d1222 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:28:47.179266 12326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87c6d482d9f1412a8f1ec924aa6d1222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.180476 12245 tablet_service.cc:1511] Processing CreateTablet for tablet 87c6d482d9f1412a8f1ec924aa6d1222 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:28:47.181870 12527 tablet_bootstrap.cc:492] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:47.183303 12245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87c6d482d9f1412a8f1ec924aa6d1222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.191402 12324 tablet_service.cc:1511] Processing CreateTablet for tablet e72262c8e2b24a8fa97e1cacbb60b49a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:28:47.192224 12467 tablet_service.cc:1511] Processing CreateTablet for tablet c67e469fc10a43bf9660c48b5ae1335a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:28:47.192988 12324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e72262c8e2b24a8fa97e1cacbb60b49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.194020 12467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c67e469fc10a43bf9660c48b5ae1335a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.198230 12534 raft_consensus.cc:493] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.202001 12534 raft_consensus.cc:515] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:47.199967 12246 tablet_service.cc:1511] Processing CreateTablet for tablet e72262c8e2b24a8fa97e1cacbb60b49a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:28:47.206229 12246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e72262c8e2b24a8fa97e1cacbb60b49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.219504 12538 raft_consensus.cc:493] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.220100 12538 raft_consensus.cc:515] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:47.222514 12538 leader_election.cc:290] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:47.225838 12157 catalog_manager.cc:5697] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:47.227626 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
W20260709 12:28:47.229390 12373 leader_election.cc:343] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.230086 12528 tablet_bootstrap.cc:492] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:47.230643 12528 ts_tablet_manager.cc:1403] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.241s	user 0.012s	sys 0.010s
I20260709 12:28:47.230746 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b789be887a492fb0e51b08f2475a97" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:47.232414 12250 tablet_service.cc:1511] Processing CreateTablet for tablet c67e469fc10a43bf9660c48b5ae1335a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260709 12:28:47.232928 12299 leader_election.cc:343] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.233939 12250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c67e469fc10a43bf9660c48b5ae1335a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.235921 12534 leader_election.cc:290] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:47.246306 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b789be887a492fb0e51b08f2475a97" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:47.247572 12397 tablet_service.cc:1511] Processing CreateTablet for tablet 3d163b2f24c845d8b11b35d222659673 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:28:47.248391 12529 tablet_bootstrap.cc:654] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.248240 12528 raft_consensus.cc:359] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:47.248705 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89bcc99e094d1bbaf83979c2b78081" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:47.248982 12397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d163b2f24c845d8b11b35d222659673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.249053 12528 raft_consensus.cc:385] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.249614 12528 raft_consensus.cc:740] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
W20260709 12:28:47.249898 12299 leader_election.cc:343] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.252027 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89bcc99e094d1bbaf83979c2b78081" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:47.250584 12299 leader_election.cc:304] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
W20260709 12:28:47.256796 12227 leader_election.cc:343] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:47.258008 12224 leader_election.cc:343] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.258744 12224 leader_election.cc:304] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
I20260709 12:28:47.259869 12554 raft_consensus.cc:2749] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:47.260314 12528 consensus_queue.cc:260] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:47.261399 12534 raft_consensus.cc:2749] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:47.269495 12325 tablet_service.cc:1511] Processing CreateTablet for tablet 3d163b2f24c845d8b11b35d222659673 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:28:47.278755 12325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d163b2f24c845d8b11b35d222659673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.250600 12469 tablet_service.cc:1511] Processing CreateTablet for tablet e72262c8e2b24a8fa97e1cacbb60b49a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:28:47.277432 12528 ts_tablet_manager.cc:1434] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.046s	user 0.002s	sys 0.006s
I20260709 12:28:47.290473 12528 tablet_bootstrap.cc:492] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:47.293505 12527 tablet_bootstrap.cc:654] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.312460 12534 raft_consensus.cc:493] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.313126 12534 raft_consensus.cc:515] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:47.299187 12250 tablet_service.cc:1511] Processing CreateTablet for tablet b89a75dc9fa64c1c94a790754889c07a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:28:47.316668 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:47.317426 12250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b89a75dc9fa64c1c94a790754889c07a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:47.318531 12299 leader_election.cc:343] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.319195 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:47.319939 12261 raft_consensus.cc:2468] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:47.321492 12299 leader_election.cc:304] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:28:47.322602 12529 tablet_bootstrap.cc:492] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:47.323181 12529 ts_tablet_manager.cc:1403] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.221s	user 0.006s	sys 0.010s
I20260709 12:28:47.323406 12326 tablet_service.cc:1511] Processing CreateTablet for tablet b89a75dc9fa64c1c94a790754889c07a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:28:47.324594 12397 tablet_service.cc:1511] Processing CreateTablet for tablet b89a75dc9fa64c1c94a790754889c07a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:28:47.323311 12552 raft_consensus.cc:2804] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:47.326195 12552 raft_consensus.cc:493] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.299719 12469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e72262c8e2b24a8fa97e1cacbb60b49a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.326609 12552 raft_consensus.cc:3060] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.327903 12467 tablet_service.cc:1511] Processing CreateTablet for tablet ab18688981ff456ca286ec171980c8de (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:28:47.324826 12326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b89a75dc9fa64c1c94a790754889c07a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.329310 12467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab18688981ff456ca286ec171980c8de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.328800 12247 tablet_service.cc:1511] Processing CreateTablet for tablet ab18688981ff456ca286ec171980c8de (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:28:47.329663 12317 tablet_service.cc:1511] Processing CreateTablet for tablet ab18688981ff456ca286ec171980c8de (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:28:47.330482 12247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab18688981ff456ca286ec171980c8de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.331130 12317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab18688981ff456ca286ec171980c8de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.331284 12529 raft_consensus.cc:359] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.332849 12529 raft_consensus.cc:385] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.333160 12529 raft_consensus.cc:740] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.334379 12397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b89a75dc9fa64c1c94a790754889c07a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.306747 12474 tablet_service.cc:1511] Processing CreateTablet for tablet 3d163b2f24c845d8b11b35d222659673 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:28:47.338392 12474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d163b2f24c845d8b11b35d222659673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.341998 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:47.342258 12529 consensus_queue.cc:260] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
W20260709 12:28:47.343986 12373 leader_election.cc:343] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.344506 12373 leader_election.cc:304] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:28:47.303426 12528 tablet_bootstrap.cc:654] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.345803 12538 raft_consensus.cc:2749] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:47.352591 12552 raft_consensus.cc:515] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:47.354012 12396 tablet_service.cc:1511] Processing CreateTablet for tablet 2e2871c5312642a28d3d3d65a5941a3a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:28:47.390347 12396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e2871c5312642a28d3d3d65a5941a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.325685 12534 leader_election.cc:290] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:47.358740 12466 tablet_service.cc:1511] Processing CreateTablet for tablet 2e2871c5312642a28d3d3d65a5941a3a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:28:47.394908 12466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e2871c5312642a28d3d3d65a5941a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.397349 12246 tablet_service.cc:1511] Processing CreateTablet for tablet 6417cb26ca8d401f8e73d3512aeab50b (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:28:47.302414 12531 tablet_bootstrap.cc:654] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.373289 12244 tablet_service.cc:1511] Processing CreateTablet for tablet 2e2871c5312642a28d3d3d65a5941a3a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:28:47.398937 12246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6417cb26ca8d401f8e73d3512aeab50b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.375778 12324 tablet_service.cc:1511] Processing CreateTablet for tablet eace44731c5747ca98999cf217b561a3 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:28:47.400396 12324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eace44731c5747ca98999cf217b561a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.402031 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
W20260709 12:28:47.403650 12299 leader_election.cc:343] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.406219 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:47.406050 12322 tablet_service.cc:1511] Processing CreateTablet for tablet 6417cb26ca8d401f8e73d3512aeab50b (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:28:47.406893 12261 raft_consensus.cc:3060] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.407845 12322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6417cb26ca8d401f8e73d3512aeab50b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:47.408859 12206 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:47.409698 12244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e2871c5312642a28d3d3d65a5941a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.414336 12399 tablet_service.cc:1511] Processing CreateTablet for tablet 6417cb26ca8d401f8e73d3512aeab50b (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:28:47.415881 12399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6417cb26ca8d401f8e73d3512aeab50b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.380021 12528 tablet_bootstrap.cc:492] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:47.381681 12398 tablet_service.cc:1511] Processing CreateTablet for tablet eace44731c5747ca98999cf217b561a3 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:28:47.386148 12529 ts_tablet_manager.cc:1434] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.063s	user 0.019s	sys 0.010s
I20260709 12:28:47.385721 12467 tablet_service.cc:1511] Processing CreateTablet for tablet eace44731c5747ca98999cf217b561a3 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:28:47.429018 12467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eace44731c5747ca98999cf217b561a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.437005 12391 tablet_service.cc:1511] Processing CreateTablet for tablet a03f6aed8fcb4ea2b45b6432634c00ae (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:28:47.437067 12318 tablet_service.cc:1511] Processing CreateTablet for tablet 606a9d73d38c472698c0f88704e59b34 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:28:47.439428 12318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 606a9d73d38c472698c0f88704e59b34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.441860 12528 ts_tablet_manager.cc:1403] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.152s	user 0.008s	sys 0.006s
I20260709 12:28:47.438182 12398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eace44731c5747ca98999cf217b561a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.442741 12469 tablet_service.cc:1511] Processing CreateTablet for tablet a03f6aed8fcb4ea2b45b6432634c00ae (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:28:47.444080 12469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a03f6aed8fcb4ea2b45b6432634c00ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.445991 12391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a03f6aed8fcb4ea2b45b6432634c00ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.437096 12247 tablet_service.cc:1511] Processing CreateTablet for tablet 606a9d73d38c472698c0f88704e59b34 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:28:47.450942 12529 tablet_bootstrap.cc:492] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:47.450942 12474 tablet_service.cc:1511] Processing CreateTablet for tablet 606a9d73d38c472698c0f88704e59b34 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:28:47.463517 12247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 606a9d73d38c472698c0f88704e59b34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.463637 12474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 606a9d73d38c472698c0f88704e59b34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.445365 12528 raft_consensus.cc:359] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.470427 12528 raft_consensus.cc:385] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.470782 12528 raft_consensus.cc:740] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.471555 12528 consensus_queue.cc:260] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.476438 12528 ts_tablet_manager.cc:1434] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.034s	user 0.001s	sys 0.006s
I20260709 12:28:47.477846 12250 tablet_service.cc:1511] Processing CreateTablet for tablet a03f6aed8fcb4ea2b45b6432634c00ae (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:28:47.479424 12250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a03f6aed8fcb4ea2b45b6432634c00ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.480924 12529 tablet_bootstrap.cc:654] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.486814 12261 raft_consensus.cc:2468] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:47.488603 12299 leader_election.cc:304] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:28:47.489779 12534 raft_consensus.cc:2804] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:47.490311 12534 raft_consensus.cc:697] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:47.491194 12534 consensus_queue.cc:237] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:47.495076 12395 tablet_service.cc:1511] Processing CreateTablet for tablet 436db7228d9d4e92a8472906005ae3fb (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:28:47.496547 12395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 436db7228d9d4e92a8472906005ae3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.426755 12552 leader_election.cc:290] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:47.499958 12528 tablet_bootstrap.cc:492] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:47.509632 12546 raft_consensus.cc:493] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.510274 12546 raft_consensus.cc:515] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:47.510742 12326 tablet_service.cc:1511] Processing CreateTablet for tablet 436db7228d9d4e92a8472906005ae3fb (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:28:47.512327 12326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 436db7228d9d4e92a8472906005ae3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.524279 12531 tablet_bootstrap.cc:492] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:47.525168 12531 ts_tablet_manager.cc:1403] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.386s	user 0.012s	sys 0.025s
I20260709 12:28:47.533897 12527 tablet_bootstrap.cc:492] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:47.552047 12243 tablet_service.cc:1511] Processing CreateTablet for tablet 864a0424f5d64be0a7d4d8b5d38907c6 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:28:47.553843 12243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864a0424f5d64be0a7d4d8b5d38907c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.538071 12527 ts_tablet_manager.cc:1403] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.356s	user 0.017s	sys 0.033s
I20260709 12:28:47.547361 12546 leader_election.cc:290] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:47.564939 12552 raft_consensus.cc:493] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.577383 12392 tablet_service.cc:1511] Processing CreateTablet for tablet 864a0424f5d64be0a7d4d8b5d38907c6 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:28:47.578871 12392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864a0424f5d64be0a7d4d8b5d38907c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.580137 12325 tablet_service.cc:1511] Processing CreateTablet for tablet 864a0424f5d64be0a7d4d8b5d38907c6 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:28:47.581521 12325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864a0424f5d64be0a7d4d8b5d38907c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.584081 12159 catalog_manager.cc:5697] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:47.595587 12302 consensus_peers.cc:597] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.570680 12552 raft_consensus.cc:515] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.600996 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:47.602911 12299 leader_election.cc:343] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.605460 12468 tablet_service.cc:1511] Processing CreateTablet for tablet 436db7228d9d4e92a8472906005ae3fb (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:28:47.605860 12467 tablet_service.cc:1511] Processing CreateTablet for tablet 1263de4693a84afe93a508aff7f1dc29 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:28:47.607076 12468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 436db7228d9d4e92a8472906005ae3fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.610666 12245 tablet_service.cc:1511] Processing CreateTablet for tablet 1263de4693a84afe93a508aff7f1dc29 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:28:47.611568 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:47.612268 12410 raft_consensus.cc:2468] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:47.614028 12302 leader_election.cc:304] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:47.615578 12534 raft_consensus.cc:2804] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:47.616051 12534 raft_consensus.cc:493] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.616771 12534 raft_consensus.cc:3060] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.618690 12245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1263de4693a84afe93a508aff7f1dc29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.619112 12552 leader_election.cc:290] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:47.618738 12467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1263de4693a84afe93a508aff7f1dc29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.628512 12531 raft_consensus.cc:359] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:47.630112 12324 tablet_service.cc:1511] Processing CreateTablet for tablet 1263de4693a84afe93a508aff7f1dc29 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:28:47.631809 12324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1263de4693a84afe93a508aff7f1dc29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.621034 12528 tablet_bootstrap.cc:654] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.629436 12531 raft_consensus.cc:385] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.644837 12531 raft_consensus.cc:740] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.645867 12531 consensus_queue.cc:260] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:47.648483 12531 ts_tablet_manager.cc:1434] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.114s	user 0.004s	sys 0.004s
I20260709 12:28:47.636806 12527 raft_consensus.cc:359] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.662036 12527 raft_consensus.cc:385] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.662403 12527 raft_consensus.cc:740] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.663439 12527 consensus_queue.cc:260] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.665439 12529 tablet_bootstrap.cc:492] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:47.666144 12529 ts_tablet_manager.cc:1403] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.216s	user 0.012s	sys 0.004s
I20260709 12:28:47.666481 12527 ts_tablet_manager.cc:1434] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.102s	user 0.005s	sys 0.002s
I20260709 12:28:47.666200 12552 consensus_queue.cc:1048] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:47.667647 12527 tablet_bootstrap.cc:492] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:47.670187 12529 raft_consensus.cc:359] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:47.676154 12531 tablet_bootstrap.cc:492] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:47.679579 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2a2e0c681f24264a729f89d08042fd9" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:28:47.684495 12450 leader_election.cc:343] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.689436 12534 raft_consensus.cc:515] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.692864 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:47.694252 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2a2e0c681f24264a729f89d08042fd9" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
W20260709 12:28:47.694532 12299 leader_election.cc:343] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:47.695962 12447 leader_election.cc:343] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.696532 12447 leader_election.cc:304] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:47.697438 12546 raft_consensus.cc:2749] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:47.697688 12398 tablet_service.cc:1511] Processing CreateTablet for tablet f39a7bfe4bff44b18d1007ced129ae09 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:28:47.698403 12250 tablet_service.cc:1511] Processing CreateTablet for tablet f39a7bfe4bff44b18d1007ced129ae09 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:28:47.699891 12250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f39a7bfe4bff44b18d1007ced129ae09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.700789 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:47.701476 12410 raft_consensus.cc:3060] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.705710 12529 raft_consensus.cc:385] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.706141 12529 raft_consensus.cc:740] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.707008 12529 consensus_queue.cc:260] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:47.710314 12529 ts_tablet_manager.cc:1434] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.044s	user 0.005s	sys 0.002s
I20260709 12:28:47.711586 12529 tablet_bootstrap.cc:492] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:47.711966 12534 leader_election.cc:290] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:47.713984 12410 raft_consensus.cc:2468] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:47.715698 12302 leader_election.cc:304] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:47.717140 12527 tablet_bootstrap.cc:654] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.717701 12534 raft_consensus.cc:2804] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:47.719105 12534 raft_consensus.cc:697] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:47.720160 12534 consensus_queue.cc:237] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.720664 12326 tablet_service.cc:1511] Processing CreateTablet for tablet d6682a011b3945c59d105bb6951d62f3 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:28:47.722234 12326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6682a011b3945c59d105bb6951d62f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.726003 12531 tablet_bootstrap.cc:654] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.699126 12398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f39a7bfe4bff44b18d1007ced129ae09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.736769 12554 raft_consensus.cc:493] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.739636 12554 raft_consensus.cc:515] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.740065 12392 tablet_service.cc:1511] Processing CreateTablet for tablet d6682a011b3945c59d105bb6951d62f3 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:28:47.751646 12392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6682a011b3945c59d105bb6951d62f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.754675 12554 leader_election.cc:290] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:47.758071 12467 tablet_service.cc:1511] Processing CreateTablet for tablet f39a7bfe4bff44b18d1007ced129ae09 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:28:47.758560 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e3b47e111a479f8582c270c1445658" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:47.758677 12469 tablet_service.cc:1511] Processing CreateTablet for tablet d6682a011b3945c59d105bb6951d62f3 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:28:47.760317 12469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6682a011b3945c59d105bb6951d62f3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:47.760782 12227 leader_election.cc:343] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.768923 12528 tablet_bootstrap.cc:492] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:47.771318 12528 ts_tablet_manager.cc:1403] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.294s	user 0.011s	sys 0.016s
I20260709 12:28:47.765262 12159 catalog_manager.cc:5697] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:47.780436 12527 tablet_bootstrap.cc:492] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:47.781164 12318 tablet_service.cc:1511] Processing CreateTablet for tablet bf7c870391ab41aa99577da648d9ad60 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:28:47.782801 12318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7c870391ab41aa99577da648d9ad60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.782945 12531 tablet_bootstrap.cc:492] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:47.783497 12531 ts_tablet_manager.cc:1403] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.108s	user 0.007s	sys 0.011s
I20260709 12:28:47.790134 12467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f39a7bfe4bff44b18d1007ced129ae09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.781185 12527 ts_tablet_manager.cc:1403] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.114s	user 0.009s	sys 0.023s
I20260709 12:28:47.801079 12395 tablet_service.cc:1511] Processing CreateTablet for tablet bf7c870391ab41aa99577da648d9ad60 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:28:47.804100 12546 raft_consensus.cc:493] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.809453 12531 raft_consensus.cc:359] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:47.820606 12474 tablet_service.cc:1511] Processing CreateTablet for tablet 164179fdd9ee45ff80ea3530e210088a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:28:47.811115 12529 tablet_bootstrap.cc:654] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.826706 12546 raft_consensus.cc:515] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:47.828032 12395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7c870391ab41aa99577da648d9ad60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.829212 12531 raft_consensus.cc:385] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.831501 12531 raft_consensus.cc:740] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.832325 12531 consensus_queue.cc:260] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:47.832859 12546 leader_election.cc:290] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:47.831274 12474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 164179fdd9ee45ff80ea3530e210088a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.834216 12468 tablet_service.cc:1511] Processing CreateTablet for tablet 2ad2a742adea4bf989def5e758a3a81c (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:28:47.835507 12468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ad2a742adea4bf989def5e758a3a81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.842862 12528 raft_consensus.cc:359] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.845465 12528 raft_consensus.cc:385] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.845821 12528 raft_consensus.cc:740] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.846611 12528 consensus_queue.cc:260] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.848706 12318 tablet_service.cc:1511] Processing CreateTablet for tablet 164179fdd9ee45ff80ea3530e210088a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:28:47.844887 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f0d8651a6144ca9fd21ccc0f57edd9" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:47.857513 12324 tablet_service.cc:1511] Processing CreateTablet for tablet 76b0d4a9a29a4de9bc3a8ab529535d68 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:28:47.858973 12528 ts_tablet_manager.cc:1434] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.087s	user 0.004s	sys 0.004s
I20260709 12:28:47.859115 12324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76b0d4a9a29a4de9bc3a8ab529535d68. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:47.859546 12450 leader_election.cc:343] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.863336 12528 tablet_bootstrap.cc:492] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:47.865761 12318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 164179fdd9ee45ff80ea3530e210088a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.871367 12527 raft_consensus.cc:359] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:47.872563 12527 raft_consensus.cc:385] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.872908 12527 raft_consensus.cc:740] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.874401 12398 tablet_service.cc:1511] Processing CreateTablet for tablet 2ad2a742adea4bf989def5e758a3a81c (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:28:47.874774 12527 consensus_queue.cc:260] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:47.875833 12398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ad2a742adea4bf989def5e758a3a81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.878782 12392 tablet_service.cc:1511] Processing CreateTablet for tablet 76b0d4a9a29a4de9bc3a8ab529535d68 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:28:47.879323 12243 tablet_service.cc:1511] Processing CreateTablet for tablet 164179fdd9ee45ff80ea3530e210088a (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:28:47.879844 12245 tablet_service.cc:1511] Processing CreateTablet for tablet 2ad2a742adea4bf989def5e758a3a81c (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:28:47.880690 12243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 164179fdd9ee45ff80ea3530e210088a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.881423 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e3b47e111a479f8582c270c1445658" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:47.882028 12250 tablet_service.cc:1511] Processing CreateTablet for tablet bf7c870391ab41aa99577da648d9ad60 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:28:47.883278 12250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7c870391ab41aa99577da648d9ad60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.877704 12527 ts_tablet_manager.cc:1434] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.051s	user 0.003s	sys 0.006s
I20260709 12:28:47.880280 12392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76b0d4a9a29a4de9bc3a8ab529535d68. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:47.883147 12224 leader_election.cc:343] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.890965 12245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ad2a742adea4bf989def5e758a3a81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.897593 12528 tablet_bootstrap.cc:654] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.887348 12224 leader_election.cc:304] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:47.914392 12554 raft_consensus.cc:2749] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:47.899971 12529 tablet_bootstrap.cc:492] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:47.901683 12469 tablet_service.cc:1511] Processing CreateTablet for tablet 76b0d4a9a29a4de9bc3a8ab529535d68 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:28:47.915467 12529 ts_tablet_manager.cc:1403] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.204s	user 0.012s	sys 0.010s
I20260709 12:28:47.916013 12469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76b0d4a9a29a4de9bc3a8ab529535d68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.909560 12531 ts_tablet_manager.cc:1434] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.126s	user 0.020s	sys 0.003s
I20260709 12:28:47.917151 12247 tablet_service.cc:1511] Processing CreateTablet for tablet 75f9d02452bc41bbab1eb4cb7e1ccf44 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:28:47.918826 12247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f9d02452bc41bbab1eb4cb7e1ccf44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.918967 12529 raft_consensus.cc:359] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.920178 12529 raft_consensus.cc:385] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.920488 12529 raft_consensus.cc:740] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.918908 12531 tablet_bootstrap.cc:492] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:47.924561 12326 tablet_service.cc:1511] Processing CreateTablet for tablet 75f9d02452bc41bbab1eb4cb7e1ccf44 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:28:47.926085 12326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f9d02452bc41bbab1eb4cb7e1ccf44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.930083 12527 tablet_bootstrap.cc:492] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:47.930091 12529 consensus_queue.cc:260] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.943943 12392 tablet_service.cc:1511] Processing CreateTablet for tablet 75f9d02452bc41bbab1eb4cb7e1ccf44 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:28:47.945324 12392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f9d02452bc41bbab1eb4cb7e1ccf44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.948678 12244 tablet_service.cc:1511] Processing CreateTablet for tablet fcca3076fc03410aa86e82c060c29335 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:28:47.950112 12244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcca3076fc03410aa86e82c060c29335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.951176 12529 ts_tablet_manager.cc:1434] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.035s	user 0.002s	sys 0.005s
I20260709 12:28:47.952296 12529 tablet_bootstrap.cc:492] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:47.954372 12325 tablet_service.cc:1511] Processing CreateTablet for tablet fcca3076fc03410aa86e82c060c29335 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:28:47.955674 12325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcca3076fc03410aa86e82c060c29335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.962441 12531 tablet_bootstrap.cc:654] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.969411 12474 tablet_service.cc:1511] Processing CreateTablet for tablet fcca3076fc03410aa86e82c060c29335 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:28:47.973978 12474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcca3076fc03410aa86e82c060c29335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.976481 12528 tablet_bootstrap.cc:492] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:47.977066 12528 ts_tablet_manager.cc:1403] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.114s	user 0.009s	sys 0.002s
I20260709 12:28:47.978811 12395 tablet_service.cc:1511] Processing CreateTablet for tablet ce7f904b0c124a17881b5b725eaa1bdc (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:28:47.980221 12395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce7f904b0c124a17881b5b725eaa1bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.980285 12528 raft_consensus.cc:359] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:47.981107 12528 raft_consensus.cc:385] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.988744 12317 tablet_service.cc:1511] Processing CreateTablet for tablet 4fe65f1e2e7d4ae796006f23575cb681 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:28:47.989250 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f0d8651a6144ca9fd21ccc0f57edd9" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:47.990326 12317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fe65f1e2e7d4ae796006f23575cb681. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:47.990634 12447 leader_election.cc:343] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.991094 12447 leader_election.cc:304] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:47.992354 12581 raft_consensus.cc:2749] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:47.989866 12467 tablet_service.cc:1511] Processing CreateTablet for tablet ce7f904b0c124a17881b5b725eaa1bdc (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:28:47.993949 12467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce7f904b0c124a17881b5b725eaa1bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:48.003316 12531 tablet_bootstrap.cc:492] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:48.003926 12531 ts_tablet_manager.cc:1403] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.085s	user 0.009s	sys 0.004s
I20260709 12:28:47.981412 12528 raft_consensus.cc:740] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.006567 12528 consensus_queue.cc:260] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.007125 12244 tablet_service.cc:1511] Processing CreateTablet for tablet ce7f904b0c124a17881b5b725eaa1bdc (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:28:48.007721 12398 tablet_service.cc:1511] Processing CreateTablet for tablet 4fe65f1e2e7d4ae796006f23575cb681 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:28:48.008545 12244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce7f904b0c124a17881b5b725eaa1bdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:48.009012 12398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fe65f1e2e7d4ae796006f23575cb681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:48.011261 12531 raft_consensus.cc:359] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.009057 12528 ts_tablet_manager.cc:1434] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.032s	user 0.002s	sys 0.006s
I20260709 12:28:48.013720 12528 tablet_bootstrap.cc:492] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:48.016008 12245 tablet_service.cc:1511] Processing CreateTablet for tablet c8d9482906bb4ca4a34c22c21f178cab (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:28:48.017115 12469 tablet_service.cc:1511] Processing CreateTablet for tablet 4fe65f1e2e7d4ae796006f23575cb681 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:28:48.017441 12245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8d9482906bb4ca4a34c22c21f178cab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:48.018422 12469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fe65f1e2e7d4ae796006f23575cb681. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:48.023427 12299 consensus_peers.cc:597] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Couldn't send request to peer a800ae68e91f43a794e326badca3320a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.023937 12325 tablet_service.cc:1511] Processing CreateTablet for tablet c8d9482906bb4ca4a34c22c21f178cab (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:28:48.025003 12552 raft_consensus.cc:493] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.025233 12325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8d9482906bb4ca4a34c22c21f178cab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:48.025499 12552 raft_consensus.cc:515] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.027699 12552 leader_election.cc:290] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:48.029109 12538 raft_consensus.cc:493] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.029645 12538 raft_consensus.cc:515] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.031113 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ae8840033347bea42f3c0ceb4d316a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:48.033982 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7540665883f74d94b96d1631ac30970d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:48.034468 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ae8840033347bea42f3c0ceb4d316a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:48.034710 12260 raft_consensus.cc:2468] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
W20260709 12:28:48.035941 12299 leader_election.cc:343] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.035991 12373 leader_election.cc:304] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
W20260709 12:28:48.036782 12299 leader_election.cc:343] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.037231 12299 leader_election.cc:304] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:28:48.037822 12538 leader_election.cc:290] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:48.038424 12538 raft_consensus.cc:2804] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.038560 12552 raft_consensus.cc:2749] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:48.038915 12538 raft_consensus.cc:493] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.039453 12538 raft_consensus.cc:3060] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.031837 12531 raft_consensus.cc:385] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.049947 12531 raft_consensus.cc:740] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.049984 12396 tablet_service.cc:1511] Processing CreateTablet for tablet c8d9482906bb4ca4a34c22c21f178cab (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:28:48.050866 12531 consensus_queue.cc:260] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.051539 12396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8d9482906bb4ca4a34c22c21f178cab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:48.053309 12531 ts_tablet_manager.cc:1434] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.001s
I20260709 12:28:48.066876 12531 tablet_bootstrap.cc:492] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:48.031177 12528 tablet_bootstrap.cc:654] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.038118 12527 tablet_bootstrap.cc:700] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596: Time spent opening tablet: real 0.102s	user 0.003s	sys 0.000s
I20260709 12:28:48.067404 12581 raft_consensus.cc:493] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.067966 12527 tablet_bootstrap.cc:654] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.067997 12581 raft_consensus.cc:515] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.058413 12392 tablet_service.cc:1511] Processing CreateTablet for tablet b6cc6fec74b6423bb49415fbcdd93e55 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:28:48.070407 12581 leader_election.cc:290] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:48.070618 12392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6cc6fec74b6423bb49415fbcdd93e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:48.063470 12474 tablet_service.cc:1511] Processing CreateTablet for tablet a6b87f00ff8c4e1c858037069e3bbb6e (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:28:48.074282 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:48.075511 12474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6b87f00ff8c4e1c858037069e3bbb6e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:48.075899 12447 leader_election.cc:343] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.063593 12244 tablet_service.cc:1511] Processing CreateTablet for tablet a6b87f00ff8c4e1c858037069e3bbb6e (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:28:48.078655 12244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6b87f00ff8c4e1c858037069e3bbb6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:48.078904 12552 consensus_queue.cc:1048] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:48.073212 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
W20260709 12:28:48.085867 12447 leader_election.cc:343] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.086346 12447 leader_election.cc:304] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:28:48.087559 12581 raft_consensus.cc:2749] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:48.096697 12538 raft_consensus.cc:515] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.098625 12467 tablet_service.cc:1511] Processing CreateTablet for tablet b6cc6fec74b6423bb49415fbcdd93e55 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
W20260709 12:28:48.099576 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 63671.6us
I20260709 12:28:48.119158 12467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6cc6fec74b6423bb49415fbcdd93e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:48.121148 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7540665883f74d94b96d1631ac30970d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:48.121801 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7540665883f74d94b96d1631ac30970d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:48.120580 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7540665883f74d94b96d1631ac30970d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:48.122831 12260 raft_consensus.cc:3060] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.101181 12324 tablet_service.cc:1511] Processing CreateTablet for tablet a6b87f00ff8c4e1c858037069e3bbb6e (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:28:48.107146 12243 tablet_service.cc:1511] Processing CreateTablet for tablet b6cc6fec74b6423bb49415fbcdd93e55 (DEFAULT_TABLE table=test-workload [id=218e0637b65a45d39ad805781e522c1e]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:28:48.124346 12586 raft_consensus.cc:493] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.124660 12243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6cc6fec74b6423bb49415fbcdd93e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:48.125200 12324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6b87f00ff8c4e1c858037069e3bbb6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:48.108258 12581 raft_consensus.cc:493] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.112478 12527 tablet_bootstrap.cc:492] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:48.121923 12336 raft_consensus.cc:2468] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:48.107502 12538 leader_election.cc:290] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:48.124878 12586 raft_consensus.cc:515] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.125872 12581 raft_consensus.cc:515] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.129108 12527 ts_tablet_manager.cc:1403] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.199s	user 0.011s	sys 0.003s
I20260709 12:28:48.065708 12529 tablet_bootstrap.cc:654] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.153918 12259 raft_consensus.cc:3060] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.163762 12552 consensus_queue.cc:1048] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:48.164784 12527 raft_consensus.cc:359] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.165620 12527 raft_consensus.cc:385] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.165865 12554 raft_consensus.cc:493] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.166066 12527 raft_consensus.cc:740] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.166500 12554 raft_consensus.cc:515] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.167663 12586 leader_election.cc:290] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:48.168973 12554 leader_election.cc:290] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:48.169376 12258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b789be887a492fb0e51b08f2475a97" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:48.170214 12527 consensus_queue.cc:260] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.170758 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:48.170768 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd891675417c4265b77695b4227d980f" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
W20260709 12:28:48.172386 12447 leader_election.cc:343] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.173319 12528 tablet_bootstrap.cc:492] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:48.173381 12531 tablet_bootstrap.cc:654] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.173609 12258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd891675417c4265b77695b4227d980f" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:48.174084 12528 ts_tablet_manager.cc:1403] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.161s	user 0.030s	sys 0.024s
I20260709 12:28:48.175607 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:48.176234 12410 raft_consensus.cc:2468] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:48.176712 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b789be887a492fb0e51b08f2475a97" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:48.177343 12336 raft_consensus.cc:2468] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:48.177304 12528 raft_consensus.cc:359] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.178251 12528 raft_consensus.cc:385] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.178341 12227 leader_election.cc:304] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:28:48.178560 12528 raft_consensus.cc:740] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.179013 12447 leader_election.cc:304] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:48.179826 12546 raft_consensus.cc:2804] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.179505 12528 consensus_queue.cc:260] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.180250 12546 raft_consensus.cc:493] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.180683 12546 raft_consensus.cc:3060] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.188652 12546 raft_consensus.cc:515] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.191664 12528 ts_tablet_manager.cc:1434] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.009s
I20260709 12:28:48.192255 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b789be887a492fb0e51b08f2475a97" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:48.193254 12528 tablet_bootstrap.cc:492] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
W20260709 12:28:48.194034 12447 leader_election.cc:343] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:48.195012 12373 leader_election.cc:343] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:48.195861 12373 leader_election.cc:343] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.196738 12373 leader_election.cc:304] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:28:48.197664 12538 raft_consensus.cc:2749] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:48.198848 12581 leader_election.cc:290] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:48.201536 12528 tablet_bootstrap.cc:654] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.201630 12527 ts_tablet_manager.cc:1434] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.004s
I20260709 12:28:48.206095 12527 tablet_bootstrap.cc:492] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:48.212445 12531 tablet_bootstrap.cc:492] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:48.213097 12531 ts_tablet_manager.cc:1403] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.146s	user 0.011s	sys 0.003s
I20260709 12:28:48.215447 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b789be887a492fb0e51b08f2475a97" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:48.216110 12336 raft_consensus.cc:3060] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.217100 12528 tablet_bootstrap.cc:492] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:48.216795 12531 raft_consensus.cc:359] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.217662 12528 ts_tablet_manager.cc:1403] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.000s
I20260709 12:28:48.217669 12531 raft_consensus.cc:385] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.218209 12531 raft_consensus.cc:740] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.219978 12527 tablet_bootstrap.cc:654] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.221706 12528 raft_consensus.cc:359] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.222536 12528 raft_consensus.cc:385] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.222926 12528 raft_consensus.cc:740] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.226658 12528 consensus_queue.cc:260] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.223755 12546 leader_election.cc:290] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:48.229354 12528 ts_tablet_manager.cc:1434] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:28:48.232606 12528 tablet_bootstrap.cc:492] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:48.237856 12336 raft_consensus.cc:2468] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:48.219327 12531 consensus_queue.cc:260] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.239610 12447 leader_election.cc:304] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:48.240814 12531 ts_tablet_manager.cc:1434] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 12:28:48.240994 12581 raft_consensus.cc:2804] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.244836 12581 raft_consensus.cc:697] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:48.245375 12531 tablet_bootstrap.cc:492] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:48.245709 12581 consensus_queue.cc:237] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.251616 12531 tablet_bootstrap.cc:654] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.252506 12527 tablet_bootstrap.cc:492] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:48.252650 12528 tablet_bootstrap.cc:654] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.253070 12527 ts_tablet_manager.cc:1403] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.047s	user 0.003s	sys 0.010s
I20260709 12:28:48.256450 12527 raft_consensus.cc:359] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.257205 12527 raft_consensus.cc:385] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.251546 12529 tablet_bootstrap.cc:492] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:48.257506 12527 raft_consensus.cc:740] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.259769 12529 ts_tablet_manager.cc:1403] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.308s	user 0.016s	sys 0.012s
I20260709 12:28:48.262080 12527 consensus_queue.cc:260] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.263190 12528 tablet_bootstrap.cc:492] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:48.263940 12528 ts_tablet_manager.cc:1403] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.002s
I20260709 12:28:48.263751 12529 raft_consensus.cc:359] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.264777 12529 raft_consensus.cc:385] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.265086 12529 raft_consensus.cc:740] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.268314 12529 consensus_queue.cc:260] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.271790 12529 ts_tablet_manager.cc:1434] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:28:48.272622 12552 consensus_queue.cc:1048] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:48.279333 12527 ts_tablet_manager.cc:1434] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 12:28:48.284731 12528 raft_consensus.cc:359] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.285693 12528 raft_consensus.cc:385] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.286013 12528 raft_consensus.cc:740] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.289671 12157 catalog_manager.cc:5697] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.292842 12554 raft_consensus.cc:2804] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:48.295423 12554 raft_consensus.cc:493] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:28:48.295872 12554 raft_consensus.cc:3060] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:48.297070 12529 tablet_bootstrap.cc:492] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:48.304461 12531 tablet_bootstrap.cc:492] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:48.305204 12531 ts_tablet_manager.cc:1403] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.060s	user 0.023s	sys 0.015s
I20260709 12:28:48.287945 12527 tablet_bootstrap.cc:492] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:48.288344 12528 consensus_queue.cc:260] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.312909 12528 ts_tablet_manager.cc:1434] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.000s
I20260709 12:28:48.314101 12528 tablet_bootstrap.cc:492] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:48.309845 12531 raft_consensus.cc:359] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.337549 12531 raft_consensus.cc:385] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.338121 12531 raft_consensus.cc:740] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.339377 12531 consensus_queue.cc:260] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.236455 12260 raft_consensus.cc:2468] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:48.341675 12373 leader_election.cc:304] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:28:48.350404 12528 tablet_bootstrap.cc:654] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.350732 12586 raft_consensus.cc:2804] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.341104 12581 raft_consensus.cc:493] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.351361 12586 raft_consensus.cc:697] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:28:48.351521 12581 raft_consensus.cc:515] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.354562 12552 raft_consensus.cc:493] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.355126 12529 tablet_bootstrap.cc:654] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.355437 12552 raft_consensus.cc:515] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.343835 12531 ts_tablet_manager.cc:1434] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 12:28:48.352861 12586 consensus_queue.cc:237] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.364959 12552 leader_election.cc:290] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:48.365159 12593 raft_consensus.cc:493] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.365804 12593 raft_consensus.cc:515] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.368381 12593 leader_election.cc:290] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:48.362676 12527 tablet_bootstrap.cc:654] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.375810 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:48.376634 12410 raft_consensus.cc:2468] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:48.378261 12302 leader_election.cc:304] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:48.378810 12531 tablet_bootstrap.cc:492] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:48.379761 12552 raft_consensus.cc:2804] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.380280 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:48.380355 12552 raft_consensus.cc:493] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.381153 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:48.381115 12552 raft_consensus.cc:3060] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.386081 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:48.389791 12484 raft_consensus.cc:2468] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
W20260709 12:28:48.382114 12299 leader_election.cc:343] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.391255 12224 leader_election.cc:304] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
W20260709 12:28:48.396634 12450 leader_election.cc:343] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.383462 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:48.397048 12593 raft_consensus.cc:2804] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.398934 12593 raft_consensus.cc:493] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.401825 12593 raft_consensus.cc:3060] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.402115 12554 raft_consensus.cc:515] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.403324 12261 raft_consensus.cc:1240] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Rejecting Update request from peer 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:48.403844 12528 tablet_bootstrap.cc:492] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
W20260709 12:28:48.404125 12224 leader_election.cc:343] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.404448 12528 ts_tablet_manager.cc:1403] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.091s	user 0.012s	sys 0.014s
I20260709 12:28:48.405159 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:48.406965 12447 leader_election.cc:343] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.407495 12447 leader_election.cc:304] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:48.408491 12546 raft_consensus.cc:2749] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:48.399210 12581 leader_election.cc:290] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:48.409451 12534 consensus_queue.cc:1059] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:48.410887 12565 raft_consensus.cc:3055] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:48.411283 12565 raft_consensus.cc:740] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
W20260709 12:28:48.412729 12206 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:48.412257 12565 consensus_queue.cc:260] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.413689 12565 raft_consensus.cc:3060] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:48.414335 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:48.414264 12155 catalog_manager.cc:5697] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:48.418776 12531 tablet_bootstrap.cc:654] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.419478 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:48.421815 12527 tablet_bootstrap.cc:492] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:48.422345 12527 ts_tablet_manager.cc:1403] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.135s	user 0.009s	sys 0.007s
I20260709 12:28:48.425422 12527 raft_consensus.cc:359] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.425720 12554 leader_election.cc:290] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:48.416966 12593 raft_consensus.cc:515] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.429222 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:48.429833 12484 raft_consensus.cc:3060] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.431882 12552 raft_consensus.cc:515] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.432992 12529 tablet_bootstrap.cc:492] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:48.432739 12528 raft_consensus.cc:359] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.433460 12528 raft_consensus.cc:385] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.433522 12529 ts_tablet_manager.cc:1403] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.137s	user 0.024s	sys 0.011s
I20260709 12:28:48.433859 12528 raft_consensus.cc:740] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.434337 12552 leader_election.cc:290] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:48.435036 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:48.435621 12409 raft_consensus.cc:3060] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.436188 12529 raft_consensus.cc:359] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.436889 12529 raft_consensus.cc:385] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.437315 12529 raft_consensus.cc:740] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.436816 12528 consensus_queue.cc:260] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.438134 12529 consensus_queue.cc:260] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.439466 12528 ts_tablet_manager.cc:1434] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:28:48.440776 12528 tablet_bootstrap.cc:492] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:48.442948 12531 tablet_bootstrap.cc:492] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:48.443519 12531 ts_tablet_manager.cc:1403] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.070s	user 0.012s	sys 0.005s
I20260709 12:28:48.445158 12529 ts_tablet_manager.cc:1434] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 12:28:48.447045 12529 tablet_bootstrap.cc:492] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:48.446862 12531 raft_consensus.cc:359] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.447669 12531 raft_consensus.cc:385] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.448014 12531 raft_consensus.cc:740] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.448799 12531 consensus_queue.cc:260] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.450625 12586 raft_consensus.cc:493] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.451279 12586 raft_consensus.cc:515] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.453814 12586 leader_election.cc:290] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:48.454562 12529 tablet_bootstrap.cc:654] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.455412 12531 ts_tablet_manager.cc:1434] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 12:28:48.456508 12531 tablet_bootstrap.cc:492] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:48.457456 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05dfc5c361a94fe5af3e9a5462a0b12f" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:48.458210 12261 raft_consensus.cc:2468] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:48.459645 12373 leader_election.cc:304] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:48.460971 12586 raft_consensus.cc:2804] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.461400 12586 raft_consensus.cc:493] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.461776 12586 raft_consensus.cc:3060] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.464855 12531 tablet_bootstrap.cc:654] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.467150 12529 tablet_bootstrap.cc:492] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:48.467097 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05dfc5c361a94fe5af3e9a5462a0b12f" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:48.467983 12335 raft_consensus.cc:2468] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:48.471065 12529 ts_tablet_manager.cc:1403] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.004s
I20260709 12:28:48.476234 12529 raft_consensus.cc:359] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.477386 12529 raft_consensus.cc:385] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.477912 12529 raft_consensus.cc:740] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.479254 12529 consensus_queue.cc:260] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.482005 12528 tablet_bootstrap.cc:654] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.483145 12529 ts_tablet_manager.cc:1434] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:28:48.485800 12529 tablet_bootstrap.cc:492] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:48.486706 12531 tablet_bootstrap.cc:492] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:48.487270 12531 ts_tablet_manager.cc:1403] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.016s
I20260709 12:28:48.487099 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:48.489866 12554 raft_consensus.cc:493] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.490494 12554 raft_consensus.cc:515] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.493022 12529 tablet_bootstrap.cc:654] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:48.489912 12299 leader_election.cc:343] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.494880 12409 raft_consensus.cc:2468] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:48.496001 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ae8840033347bea42f3c0ceb4d316a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:48.496385 12302 leader_election.cc:304] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:28:48.496759 12335 raft_consensus.cc:2468] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:48.498708 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ae8840033347bea42f3c0ceb4d316a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:48.499061 12565 raft_consensus.cc:2804] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.499637 12565 raft_consensus.cc:697] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
W20260709 12:28:48.500663 12224 leader_election.cc:343] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.500509 12565 consensus_queue.cc:237] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.502609 12586 raft_consensus.cc:515] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.505147 12224 leader_election.cc:304] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:28:48.506561 12594 raft_consensus.cc:2804] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.506606 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05dfc5c361a94fe5af3e9a5462a0b12f" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:48.507062 12594 raft_consensus.cc:493] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.507304 12261 raft_consensus.cc:3060] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.507490 12594 raft_consensus.cc:3060] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.492983 12554 leader_election.cc:290] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:48.512419 12531 raft_consensus.cc:359] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.513177 12531 raft_consensus.cc:385] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.513494 12531 raft_consensus.cc:740] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.426327 12527 raft_consensus.cc:385] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.514218 12527 raft_consensus.cc:740] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.515384 12527 consensus_queue.cc:260] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.517315 12594 raft_consensus.cc:515] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.519249 12528 tablet_bootstrap.cc:492] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:48.520378 12528 ts_tablet_manager.cc:1403] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.080s	user 0.019s	sys 0.003s
I20260709 12:28:48.519589 12531 consensus_queue.cc:260] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.521879 12529 tablet_bootstrap.cc:492] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:48.522441 12529 ts_tablet_manager.cc:1403] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.006s
I20260709 12:28:48.523787 12531 ts_tablet_manager.cc:1434] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:28:48.525843 12529 raft_consensus.cc:359] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.526598 12529 raft_consensus.cc:385] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.526925 12529 raft_consensus.cc:740] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.527628 12529 consensus_queue.cc:260] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.527876 12528 raft_consensus.cc:359] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.530208 12528 raft_consensus.cc:385] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.530987 12528 raft_consensus.cc:740] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.533782 12336 raft_consensus.cc:2468] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:28:48.536599 12529 ts_tablet_manager.cc:1434] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.001s
I20260709 12:28:48.537081 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05dfc5c361a94fe5af3e9a5462a0b12f" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:48.537770 12335 raft_consensus.cc:3060] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.538075 12593 leader_election.cc:290] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:48.539530 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ae8840033347bea42f3c0ceb4d316a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
W20260709 12:28:48.541113 12224 leader_election.cc:343] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.542580 12594 leader_election.cc:290] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:48.542594 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:48.543748 12529 tablet_bootstrap.cc:492] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:48.542766 12224 leader_election.cc:304] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:48.545676 12594 raft_consensus.cc:2804] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:48.546227 12594 raft_consensus.cc:697] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
W20260709 12:28:48.546578 12224 leader_election.cc:343] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.547467 12594 consensus_queue.cc:237] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.552174 12552 raft_consensus.cc:493] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.552598 12538 raft_consensus.cc:493] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.552762 12552 raft_consensus.cc:515] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.553099 12538 raft_consensus.cc:515] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.555035 12552 leader_election.cc:290] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:48.555922 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e3b47e111a479f8582c270c1445658" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:48.556666 12260 raft_consensus.cc:2468] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:48.556787 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e3b47e111a479f8582c270c1445658" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:48.557021 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f0d8651a6144ca9fd21ccc0f57edd9" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:48.557790 12483 raft_consensus.cc:2468] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:48.558185 12299 leader_election.cc:304] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:28:48.557411 12159 catalog_manager.cc:5697] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 1 to 2, leader changed from 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.559087 12373 leader_election.cc:304] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
W20260709 12:28:48.559120 12302 leader_election.cc:343] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.533298 12484 raft_consensus.cc:2468] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:48.560889 12538 leader_election.cc:290] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:48.561269 12594 raft_consensus.cc:493] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.529376 12531 tablet_bootstrap.cc:492] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:48.534875 12586 leader_election.cc:290] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:48.559312 12552 raft_consensus.cc:2804] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.561605 12224 leader_election.cc:304] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:28:48.564324 12335 raft_consensus.cc:2468] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:48.564863 12593 raft_consensus.cc:2804] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.565415 12373 leader_election.cc:304] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:48.565728 12593 raft_consensus.cc:697] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:48.566399 12586 raft_consensus.cc:2804] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.566641 12593 consensus_queue.cc:237] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.567162 12586 raft_consensus.cc:697] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:28:48.568811 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ae8840033347bea42f3c0ceb4d316a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:48.568464 12586 consensus_queue.cc:237] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.569454 12336 raft_consensus.cc:3060] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.574620 12531 tablet_bootstrap.cc:654] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.562820 12615 raft_consensus.cc:2804] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.576411 12529 tablet_bootstrap.cc:654] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.563122 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f0d8651a6144ca9fd21ccc0f57edd9" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:48.577458 12615 raft_consensus.cc:493] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.577950 12615 raft_consensus.cc:3060] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.553078 12528 consensus_queue.cc:260] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.584062 12527 ts_tablet_manager.cc:1434] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.161s	user 0.007s	sys 0.000s
I20260709 12:28:48.584316 12528 ts_tablet_manager.cc:1434] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.001s
I20260709 12:28:48.586689 12261 raft_consensus.cc:2468] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:48.587713 12615 raft_consensus.cc:515] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.585487 12528 tablet_bootstrap.cc:492] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:48.591635 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f0d8651a6144ca9fd21ccc0f57edd9" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:48.592273 12484 raft_consensus.cc:3060] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.593268 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f0d8651a6144ca9fd21ccc0f57edd9" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:48.595767 12615 leader_election.cc:290] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:48.563516 12552 raft_consensus.cc:493] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.606348 12552 raft_consensus.cc:3060] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.606559 12336 raft_consensus.cc:2468] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:48.606433 12157 catalog_manager.cc:5697] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:48.607967 12224 leader_election.cc:304] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:28:48.608954 12554 raft_consensus.cc:2804] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.585551 12527 tablet_bootstrap.cc:492] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:48.600467 12529 tablet_bootstrap.cc:492] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:48.611516 12529 ts_tablet_manager.cc:1403] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.074s	user 0.016s	sys 0.000s
I20260709 12:28:48.607822 12159 catalog_manager.cc:5697] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.614835 12529 raft_consensus.cc:359] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.615811 12529 raft_consensus.cc:385] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.616117 12529 raft_consensus.cc:740] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.616726 12484 raft_consensus.cc:2468] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:48.618147 12373 leader_election.cc:304] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:48.619343 12586 raft_consensus.cc:2804] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.620009 12586 raft_consensus.cc:697] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:28:48.561838 12594 raft_consensus.cc:515] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.614845 12552 raft_consensus.cc:515] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.620831 12586 consensus_queue.cc:237] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.621520 12260 raft_consensus.cc:2468] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:48.624234 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e3b47e111a479f8582c270c1445658" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:48.624933 12260 raft_consensus.cc:3060] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.628909 12594 leader_election.cc:290] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:48.629312 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f0d8651a6144ca9fd21ccc0f57edd9" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:48.630097 12528 tablet_bootstrap.cc:654] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.616863 12529 consensus_queue.cc:260] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.632864 12554 raft_consensus.cc:697] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:48.629485 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f0d8651a6144ca9fd21ccc0f57edd9" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:48.633373 12529 ts_tablet_manager.cc:1434] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:28:48.633960 12484 raft_consensus.cc:2393] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1dcded10caa0493da818e67e13176979 in current term 1: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:28:48.634521 12529 tablet_bootstrap.cc:492] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:48.634056 12554 consensus_queue.cc:237] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.635183 12224 leader_election.cc:304] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
I20260709 12:28:48.636719 12620 raft_consensus.cc:3060] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.637089 12565 raft_consensus.cc:493] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.637598 12565 raft_consensus.cc:515] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.638265 12527 tablet_bootstrap.cc:654] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.660468 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:48.646000 12531 tablet_bootstrap.cc:492] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:48.662752 12531 ts_tablet_manager.cc:1403] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.134s	user 0.010s	sys 0.009s
I20260709 12:28:48.648409 12620 raft_consensus.cc:2749] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:28:48.665699 12302 leader_election.cc:343] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.665938 12565 leader_election.cc:290] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:48.666831 12531 raft_consensus.cc:359] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.667541 12531 raft_consensus.cc:385] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.641384 12157 catalog_manager.cc:5697] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.667812 12531 raft_consensus.cc:740] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.669797 12531 consensus_queue.cc:260] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.668608 12260 raft_consensus.cc:2468] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:48.680985 12299 leader_election.cc:304] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:28:48.682806 12531 ts_tablet_manager.cc:1434] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:28:48.672467 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:48.685688 12484 raft_consensus.cc:2468] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:48.688696 12299 leader_election.cc:304] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:48.689957 12534 raft_consensus.cc:2804] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.691951 12565 raft_consensus.cc:2804] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.692469 12565 raft_consensus.cc:493] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.692924 12565 raft_consensus.cc:3060] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.695935 12531 tablet_bootstrap.cc:492] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
W20260709 12:28:48.696286 12447 consensus_peers.cc:597] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:48.670958 12155 catalog_manager.cc:5697] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.702385 12552 leader_election.cc:290] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:48.703632 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e3b47e111a479f8582c270c1445658" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
W20260709 12:28:48.705510 12302 leader_election.cc:343] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.706717 12620 raft_consensus.cc:493] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.707583 12531 tablet_bootstrap.cc:654] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.707281 12620 raft_consensus.cc:515] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.709863 12620 leader_election.cc:290] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:48.712855 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:48.712993 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:48.713987 12484 raft_consensus.cc:2468] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
W20260709 12:28:48.714497 12227 leader_election.cc:343] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.715565 12224 leader_election.cc:304] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:48.716619 12531 tablet_bootstrap.cc:492] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:48.717005 12528 tablet_bootstrap.cc:492] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:48.717443 12531 ts_tablet_manager.cc:1403] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.005s
I20260709 12:28:48.716907 12620 raft_consensus.cc:2804] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.717861 12528 ts_tablet_manager.cc:1403] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.133s	user 0.017s	sys 0.000s
I20260709 12:28:48.718057 12620 raft_consensus.cc:493] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.718757 12620 raft_consensus.cc:3060] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.721228 12531 raft_consensus.cc:359] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.722038 12531 raft_consensus.cc:385] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.722313 12531 raft_consensus.cc:740] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.722647 12528 raft_consensus.cc:359] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.723433 12528 raft_consensus.cc:385] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.723755 12528 raft_consensus.cc:740] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.724041 12529 tablet_bootstrap.cc:654] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.726898 12528 consensus_queue.cc:260] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.728399 12531 consensus_queue.cc:260] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.728840 12620 raft_consensus.cc:515] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.728794 12534 raft_consensus.cc:697] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:48.729800 12528 ts_tablet_manager.cc:1434] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:28:48.730783 12528 tablet_bootstrap.cc:492] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:48.730543 12534 consensus_queue.cc:237] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.732347 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
W20260709 12:28:48.733955 12227 leader_election.cc:343] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.733990 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:48.734719 12484 raft_consensus.cc:3060] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.743398 12620 leader_election.cc:290] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:48.744218 12531 ts_tablet_manager.cc:1434] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:28:48.744059 12484 raft_consensus.cc:2468] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:48.744627 12159 catalog_manager.cc:5697] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.745949 12224 leader_election.cc:304] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:48.747327 12528 tablet_bootstrap.cc:654] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.747534 12620 raft_consensus.cc:2804] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.748531 12620 raft_consensus.cc:697] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:48.749460 12620 consensus_queue.cc:237] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.753962 12531 tablet_bootstrap.cc:492] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:48.763288 12528 tablet_bootstrap.cc:492] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:48.763386 12565 raft_consensus.cc:515] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.766800 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:48.767678 12531 tablet_bootstrap.cc:654] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.767988 12581 consensus_queue.cc:1048] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:48.768256 12302 leader_election.cc:343] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.764027 12528 ts_tablet_manager.cc:1403] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.000s
I20260709 12:28:48.772423 12155 catalog_manager.cc:5697] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:48.773586 12527 tablet_bootstrap.cc:492] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:48.776312 12527 ts_tablet_manager.cc:1403] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.191s	user 0.035s	sys 0.054s
I20260709 12:28:48.776510 12528 raft_consensus.cc:359] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.776580 12586 consensus_queue.cc:1048] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:48.777370 12528 raft_consensus.cc:385] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.777733 12528 raft_consensus.cc:740] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.778404 12528 consensus_queue.cc:260] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.779121 12527 raft_consensus.cc:359] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.779814 12527 raft_consensus.cc:385] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.780068 12527 raft_consensus.cc:740] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.780359 12528 ts_tablet_manager.cc:1434] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:28:48.780751 12527 consensus_queue.cc:260] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.782999 12527 ts_tablet_manager.cc:1434] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:28:48.781260 12528 tablet_bootstrap.cc:492] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:48.788689 12529 tablet_bootstrap.cc:492] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:48.789220 12529 ts_tablet_manager.cc:1403] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.155s	user 0.012s	sys 0.000s
I20260709 12:28:48.791450 12528 tablet_bootstrap.cc:654] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.791831 12159 catalog_manager.cc:5697] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.793756 12527 tablet_bootstrap.cc:492] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:48.794334 12565 leader_election.cc:290] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:48.795441 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:48.796066 12484 raft_consensus.cc:3060] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.797462 12529 raft_consensus.cc:359] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.798956 12529 raft_consensus.cc:385] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.799273 12529 raft_consensus.cc:740] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.800510 12529 consensus_queue.cc:260] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.811139 12546 raft_consensus.cc:493] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.811710 12546 raft_consensus.cc:515] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.813750 12581 raft_consensus.cc:493] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.814263 12581 raft_consensus.cc:515] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.814652 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd891675417c4265b77695b4227d980f" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:48.815361 12409 raft_consensus.cc:2468] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:48.816460 12527 tablet_bootstrap.cc:654] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.816365 12581 leader_election.cc:290] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:48.816679 12450 leader_election.cc:304] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:48.818457 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:48.819657 12529 ts_tablet_manager.cc:1434] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 12:28:48.818567 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:48.820516 12529 tablet_bootstrap.cc:492] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:48.821208 12581 raft_consensus.cc:2804] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.821776 12581 raft_consensus.cc:493] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.822091 12546 leader_election.cc:290] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:48.822224 12581 raft_consensus.cc:3060] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.823143 12484 raft_consensus.cc:2468] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:48.822636 12546 raft_consensus.cc:493] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.824272 12299 leader_election.cc:304] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:48.824972 12546 raft_consensus.cc:515] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.825686 12565 raft_consensus.cc:2804] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.826625 12565 raft_consensus.cc:697] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:48.827347 12546 leader_election.cc:290] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:48.827620 12529 tablet_bootstrap.cc:654] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.833179 12527 tablet_bootstrap.cc:492] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:48.833891 12527 ts_tablet_manager.cc:1403] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.004s
I20260709 12:28:48.827489 12565 consensus_queue.cc:237] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.838392 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:48.839234 12409 raft_consensus.cc:2393] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a800ae68e91f43a794e326badca3320a in current term 1: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:28:48.843199 12335 raft_consensus.cc:3060] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.849246 12528 tablet_bootstrap.cc:492] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:48.850565 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd891675417c4265b77695b4227d980f" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:48.851285 12261 raft_consensus.cc:2468] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:48.852072 12528 ts_tablet_manager.cc:1403] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.001s
I20260709 12:28:48.853137 12531 tablet_bootstrap.cc:492] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:48.854485 12531 ts_tablet_manager.cc:1403] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.101s	user 0.018s	sys 0.008s
I20260709 12:28:48.855583 12528 raft_consensus.cc:359] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.856380 12528 raft_consensus.cc:385] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.856746 12528 raft_consensus.cc:740] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.857800 12528 consensus_queue.cc:260] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.858309 12581 raft_consensus.cc:515] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.860260 12528 ts_tablet_manager.cc:1434] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:48.860270 12531 raft_consensus.cc:359] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.862862 12546 raft_consensus.cc:493] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.863041 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd891675417c4265b77695b4227d980f" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:48.864630 12531 raft_consensus.cc:385] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.865370 12531 raft_consensus.cc:740] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.865664 12528 tablet_bootstrap.cc:492] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:48.866349 12546 raft_consensus.cc:515] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.869745 12409 raft_consensus.cc:3060] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.871239 12484 raft_consensus.cc:3060] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.869326 12546 leader_election.cc:290] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:28:48.878618 12447 leader_election.cc:343] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.879020 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
W20260709 12:28:48.880810 12447 leader_election.cc:343] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.880960 12586 consensus_queue.cc:1048] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:48.885165 12615 raft_consensus.cc:493] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.888229 12531 consensus_queue.cc:260] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.887823 12615 raft_consensus.cc:515] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.885406 12447 leader_election.cc:304] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:28:48.886391 12527 raft_consensus.cc:359] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.894946 12527 raft_consensus.cc:385] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.889104 12620 raft_consensus.cc:493] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.895821 12620 raft_consensus.cc:515] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.899056 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b789be887a492fb0e51b08f2475a97" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:48.901513 12620 leader_election.cc:290] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:48.895331 12527 raft_consensus.cc:740] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.906805 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b789be887a492fb0e51b08f2475a97" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:48.908869 12615 leader_election.cc:290] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:48.911928 12636 raft_consensus.cc:2749] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:48.912202 12224 leader_election.cc:304] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:28:48.912431 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89bcc99e094d1bbaf83979c2b78081" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:48.913203 12261 raft_consensus.cc:2468] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:48.913625 12620 raft_consensus.cc:2749] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:48.902614 12581 leader_election.cc:290] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:48.914352 12409 raft_consensus.cc:2468] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:48.915392 12527 consensus_queue.cc:260] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:48.917860 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:48.918632 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89bcc99e094d1bbaf83979c2b78081" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:48.918819 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:48.919106 12258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd891675417c4265b77695b4227d980f" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:48.919426 12447 leader_election.cc:304] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:48.920033 12373 leader_election.cc:304] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:48.922040 12531 ts_tablet_manager.cc:1434] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 12:28:48.932432 12531 tablet_bootstrap.cc:492] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:48.930393 12258 raft_consensus.cc:3060] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.937589 12527 ts_tablet_manager.cc:1434] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.103s	user 0.005s	sys 0.004s
I20260709 12:28:48.934414 12586 raft_consensus.cc:2804] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.934779 12581 raft_consensus.cc:3060] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.941109 12527 tablet_bootstrap.cc:492] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:48.938387 12450 leader_election.cc:304] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:48.942282 12586 raft_consensus.cc:493] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:48.946067 12373 leader_election.cc:343] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.946369 12636 raft_consensus.cc:2804] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.948022 12636 raft_consensus.cc:697] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:48.948844 12636 consensus_queue.cc:237] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.947383 12586 raft_consensus.cc:3060] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.943557 12155 catalog_manager.cc:5697] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.955397 12258 raft_consensus.cc:2468] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:48.957965 12529 tablet_bootstrap.cc:492] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:48.961848 12529 ts_tablet_manager.cc:1403] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.141s	user 0.015s	sys 0.004s
I20260709 12:28:48.964890 12529 raft_consensus.cc:359] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.965718 12529 raft_consensus.cc:385] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.966040 12529 raft_consensus.cc:740] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.966882 12529 consensus_queue.cc:260] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:48.959457 12586 raft_consensus.cc:515] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:48.971699 12447 leader_election.cc:304] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:28:48.972414 12527 tablet_bootstrap.cc:654] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.972225 12529 ts_tablet_manager.cc:1434] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:28:48.975116 12529 tablet_bootstrap.cc:492] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:48.975723 12528 tablet_bootstrap.cc:654] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.980499 12531 tablet_bootstrap.cc:654] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.981735 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89bcc99e094d1bbaf83979c2b78081" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:48.980232 12581 raft_consensus.cc:2749] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:48.983546 12258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89bcc99e094d1bbaf83979c2b78081" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:48.984279 12258 raft_consensus.cc:3060] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.989711 12644 raft_consensus.cc:2749] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:28:48.990046 12373 leader_election.cc:343] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.991837 12586 leader_election.cc:290] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:48.993151 12531 tablet_bootstrap.cc:492] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:48.993767 12531 ts_tablet_manager.cc:1403] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.062s	user 0.015s	sys 0.004s
I20260709 12:28:48.996052 12586 raft_consensus.cc:493] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.996672 12586 raft_consensus.cc:515] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:48.999640 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f01aed6d25447b0be01b95f4518c8ef" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:48.999377 12531 raft_consensus.cc:359] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.000378 12531 raft_consensus.cc:385] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.000732 12531 raft_consensus.cc:740] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.001550 12531 consensus_queue.cc:260] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.003149 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f01aed6d25447b0be01b95f4518c8ef" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:49.004468 12531 ts_tablet_manager.cc:1434] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:28:49.005524 12531 tablet_bootstrap.cc:492] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:49.006582 12373 leader_election.cc:304] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:28:49.007637 12615 raft_consensus.cc:2749] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:49.007834 12586 leader_election.cc:290] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:49.010239 12528 tablet_bootstrap.cc:492] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:49.008577 12529 tablet_bootstrap.cc:654] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.010809 12528 ts_tablet_manager.cc:1403] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.146s	user 0.010s	sys 0.003s
I20260709 12:28:49.010416 12155 catalog_manager.cc:5697] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:49.011672 12531 tablet_bootstrap.cc:654] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.011758 12565 raft_consensus.cc:493] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.012403 12565 raft_consensus.cc:515] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.016153 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2a2e0c681f24264a729f89d08042fd9" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:28:49.017693 12302 leader_election.cc:343] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.021018 12565 leader_election.cc:290] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:49.021906 12552 raft_consensus.cc:493] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.022449 12552 raft_consensus.cc:515] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.023159 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2a2e0c681f24264a729f89d08042fd9" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:49.023769 12258 raft_consensus.cc:2468] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:49.024809 12552 leader_election.cc:290] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:49.025426 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d8f5c63c391477fad5c9155f5022b07" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:49.025280 12373 leader_election.cc:304] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:28:49.023815 12484 raft_consensus.cc:2468] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:49.026880 12586 raft_consensus.cc:2804] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:49.027349 12299 leader_election.cc:304] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:49.027549 12586 raft_consensus.cc:697] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:28:49.028540 12552 raft_consensus.cc:2804] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.028834 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d8f5c63c391477fad5c9155f5022b07" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:49.028604 12586 consensus_queue.cc:237] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.029006 12552 raft_consensus.cc:493] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.030650 12552 raft_consensus.cc:3060] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:49.032213 12302 leader_election.cc:343] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.034248 12335 raft_consensus.cc:1275] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:49.036387 12554 consensus_queue.cc:1048] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:49.047451 12615 consensus_queue.cc:1048] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.052659 12620 consensus_queue.cc:1048] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.053803 12565 consensus_queue.cc:1048] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.064355 12527 tablet_bootstrap.cc:492] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:49.064973 12527 ts_tablet_manager.cc:1403] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.124s	user 0.008s	sys 0.005s
I20260709 12:28:49.067107 12528 raft_consensus.cc:359] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.067983 12528 raft_consensus.cc:385] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.068337 12528 raft_consensus.cc:740] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.068113 12527 raft_consensus.cc:359] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.068921 12527 raft_consensus.cc:385] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.069271 12527 raft_consensus.cc:740] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.070362 12527 consensus_queue.cc:260] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.069090 12528 consensus_queue.cc:260] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.083132 12529 tablet_bootstrap.cc:492] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:49.084007 12529 ts_tablet_manager.cc:1403] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.111s	user 0.031s	sys 0.015s
I20260709 12:28:49.088416 12529 raft_consensus.cc:359] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.092639 12261 raft_consensus.cc:3060] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.089468 12529 raft_consensus.cc:385] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.093503 12529 raft_consensus.cc:740] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.094487 12529 consensus_queue.cc:260] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.099308 12528 ts_tablet_manager.cc:1434] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.002s
I20260709 12:28:49.100723 12528 tablet_bootstrap.cc:492] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:49.101303 12529 ts_tablet_manager.cc:1434] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 12:28:49.078002 12632 consensus_queue.cc:1048] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 12:28:49.102946 12409 raft_consensus.cc:3060] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:49.103458 12484 raft_consensus.cc:3060] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.107384 12552 raft_consensus.cc:515] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
W20260709 12:28:49.096385 12299 leader_election.cc:343] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.117808 12527 ts_tablet_manager.cc:1434] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.005s
I20260709 12:28:49.118799 12527 tablet_bootstrap.cc:492] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:49.120882 12299 leader_election.cc:304] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:49.123899 12624 raft_consensus.cc:2749] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:49.124014 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2a2e0c681f24264a729f89d08042fd9" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:49.124918 12482 raft_consensus.cc:3060] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.126488 12538 consensus_queue.cc:1048] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.129769 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2a2e0c681f24264a729f89d08042fd9" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:49.132335 12529 tablet_bootstrap.cc:492] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:49.110967 12552 leader_election.cc:290] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:49.140619 12531 tablet_bootstrap.cc:492] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:49.141361 12531 ts_tablet_manager.cc:1403] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.136s	user 0.019s	sys 0.015s
W20260709 12:28:49.144660 12302 leader_election.cc:343] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.146713 12531 raft_consensus.cc:359] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.147574 12531 raft_consensus.cc:385] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.148262 12531 raft_consensus.cc:740] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.148715 12408 raft_consensus.cc:3060] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.156623 12336 raft_consensus.cc:3060] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.157796 12620 consensus_queue.cc:1048] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:49.149371 12409 raft_consensus.cc:1275] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:49.167318 12527 tablet_bootstrap.cc:654] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.169963 12531 consensus_queue.cc:260] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.175062 12620 consensus_queue.cc:1048] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:49.176517 12554 consensus_queue.cc:1048] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:49.180131 12538 raft_consensus.cc:493] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.180714 12538 raft_consensus.cc:515] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.182895 12538 leader_election.cc:290] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:49.185940 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7916cfbec26b4115a9251411482bed41" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:49.187587 12373 leader_election.cc:343] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.189944 12554 consensus_queue.cc:1048] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.165472 12482 raft_consensus.cc:2468] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:49.192117 12528 tablet_bootstrap.cc:654] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.192379 12299 leader_election.cc:304] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:49.193676 12565 raft_consensus.cc:2804] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:49.204017 12531 ts_tablet_manager.cc:1434] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.001s
I20260709 12:28:49.196338 12565 raft_consensus.cc:697] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:49.223613 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7916cfbec26b4115a9251411482bed41" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:49.209141 12565 consensus_queue.cc:237] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.243235 12538 consensus_queue.cc:1048] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.245870 12410 raft_consensus.cc:3060] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.245596 12155 catalog_manager.cc:5697] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:49.279632 12529 tablet_bootstrap.cc:654] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.287089 12593 consensus_queue.cc:1048] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 12:28:49.290241 12302 consensus_peers.cc:597] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:49.289749 12373 leader_election.cc:343] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.303190 12373 leader_election.cc:304] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:28:49.304098 12538 raft_consensus.cc:2749] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:49.325529 12663 consensus_queue.cc:1048] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.327445 12531 tablet_bootstrap.cc:492] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
W20260709 12:28:49.290844 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 69776.9us
I20260709 12:28:49.341558 12593 raft_consensus.cc:493] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.342417 12593 raft_consensus.cc:515] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.353808 12554 raft_consensus.cc:493] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.355549 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d8f5c63c391477fad5c9155f5022b07" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:49.356462 12336 raft_consensus.cc:2468] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:49.358057 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d8f5c63c391477fad5c9155f5022b07" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:49.359310 12528 tablet_bootstrap.cc:492] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:49.359494 12663 raft_consensus.cc:493] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:49.359639 12227 leader_election.cc:343] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.360077 12663 raft_consensus.cc:515] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.361938 12538 consensus_queue.cc:1048] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.362782 12663 leader_election.cc:290] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:49.354429 12554 raft_consensus.cc:515] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.365348 12527 tablet_bootstrap.cc:492] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:49.365895 12527 ts_tablet_manager.cc:1403] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.247s	user 0.028s	sys 0.017s
I20260709 12:28:49.365402 12554 leader_election.cc:290] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:49.354434 12593 leader_election.cc:290] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:49.368779 12527 raft_consensus.cc:359] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.369491 12527 raft_consensus.cc:385] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.371171 12527 raft_consensus.cc:740] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
W20260709 12:28:49.371428 12227 consensus_peers.cc:597] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:49.371896 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:49.373585 12483 raft_consensus.cc:2468] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:49.374331 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:49.371896 12527 consensus_queue.cc:260] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.372881 12224 leader_election.cc:304] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:49.376798 12224 leader_election.cc:304] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:49.377048 12593 raft_consensus.cc:2804] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.377327 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff454851114445da8be05eda167efc4" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:49.378298 12593 raft_consensus.cc:493] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.377873 12620 raft_consensus.cc:2804] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.379070 12620 raft_consensus.cc:493] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:49.379323 12660 log.cc:927] Time spent T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a: Append to log took a long time: real 0.125s	user 0.004s	sys 0.000s
I20260709 12:28:49.379508 12620 raft_consensus.cc:3060] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.379873 12529 tablet_bootstrap.cc:492] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:49.380367 12529 ts_tablet_manager.cc:1403] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.248s	user 0.019s	sys 0.008s
W20260709 12:28:49.381358 12224 leader_election.cc:343] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.386659 12529 raft_consensus.cc:359] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.387499 12529 raft_consensus.cc:385] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.387764 12529 raft_consensus.cc:740] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.378718 12593 raft_consensus.cc:3060] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.388631 12529 consensus_queue.cc:260] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.391054 12529 ts_tablet_manager.cc:1434] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:28:49.391149 12527 ts_tablet_manager.cc:1434] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.002s
I20260709 12:28:49.392200 12529 tablet_bootstrap.cc:492] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:49.392359 12527 tablet_bootstrap.cc:492] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:49.359861 12528 ts_tablet_manager.cc:1403] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.259s	user 0.033s	sys 0.011s
I20260709 12:28:49.395319 12593 raft_consensus.cc:515] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.395906 12528 raft_consensus.cc:359] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.396615 12528 raft_consensus.cc:385] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.398360 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d8f5c63c391477fad5c9155f5022b07" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:49.398979 12336 raft_consensus.cc:3060] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.400012 12527 tablet_bootstrap.cc:654] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.404878 12529 tablet_bootstrap.cc:654] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.407997 12336 raft_consensus.cc:2468] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:49.409257 12224 leader_election.cc:304] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:28:49.410480 12531 tablet_bootstrap.cc:654] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.396900 12528 raft_consensus.cc:740] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.412957 12594 raft_consensus.cc:2804] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:49.413585 12594 raft_consensus.cc:697] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:49.414577 12258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff454851114445da8be05eda167efc4" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:49.414448 12594 consensus_queue.cc:237] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.414106 12528 consensus_queue.cc:260] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.416620 12258 raft_consensus.cc:2468] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:49.387876 12620 raft_consensus.cc:515] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.418355 12644 consensus_queue.cc:1048] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:49.420435 12299 leader_election.cc:304] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:28:49.421409 12565 raft_consensus.cc:2804] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.421931 12565 raft_consensus.cc:493] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.422322 12565 raft_consensus.cc:3060] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.423249 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:49.423854 12336 raft_consensus.cc:3060] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.424937 12620 leader_election.cc:290] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:49.426893 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:49.427520 12484 raft_consensus.cc:3060] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.428984 12581 raft_consensus.cc:493] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.429502 12581 raft_consensus.cc:515] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.431710 12581 leader_election.cc:290] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:49.432001 12336 raft_consensus.cc:2468] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:49.433231 12224 leader_election.cc:304] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:28:49.433503 12565 raft_consensus.cc:515] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.434412 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:49.435647 12565 leader_election.cc:290] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
W20260709 12:28:49.435739 12450 leader_election.cc:343] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:49.421401 12299 leader_election.cc:343] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.436749 12258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:49.438134 12620 raft_consensus.cc:2804] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:28:49.438412 12447 leader_election.cc:343] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.438721 12620 raft_consensus.cc:697] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:49.439002 12447 leader_election.cc:304] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:49.439875 12259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff454851114445da8be05eda167efc4" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:49.439646 12620 consensus_queue.cc:237] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.440445 12259 raft_consensus.cc:3060] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.440567 12644 consensus_queue.cc:1048] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:49.441916 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff454851114445da8be05eda167efc4" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:49.442582 12644 raft_consensus.cc:2749] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:49.442772 12632 consensus_queue.cc:1048] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:49.400543 12155 catalog_manager.cc:5697] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:49.461862 12299 leader_election.cc:343] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.449330 12632 consensus_queue.cc:1048] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:28:49.467929 12259 raft_consensus.cc:2468] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:49.469326 12484 raft_consensus.cc:2468] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:49.486907 12593 leader_election.cc:290] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:49.498521 12528 ts_tablet_manager.cc:1434] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.105s	user 0.010s	sys 0.011s
I20260709 12:28:49.500062 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d8f5c63c391477fad5c9155f5022b07" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:49.511525 12528 tablet_bootstrap.cc:492] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:49.518311 12531 tablet_bootstrap.cc:492] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:49.518991 12531 ts_tablet_manager.cc:1403] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.192s	user 0.013s	sys 0.003s
I20260709 12:28:49.490432 12299 leader_election.cc:304] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a800ae68e91f43a794e326badca3320a
W20260709 12:28:49.530625 12206 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:28:49.532413 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 46395.6us
I20260709 12:28:49.533030 12632 raft_consensus.cc:2804] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:49.533512 12529 tablet_bootstrap.cc:492] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:49.533653 12632 raft_consensus.cc:697] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:49.534250 12529 ts_tablet_manager.cc:1403] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.142s	user 0.004s	sys 0.020s
I20260709 12:28:49.534878 12632 consensus_queue.cc:237] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
W20260709 12:28:49.536330 12227 leader_election.cc:343] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.562383 12529 raft_consensus.cc:359] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.563213 12529 raft_consensus.cc:385] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.563478 12529 raft_consensus.cc:740] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.569818 12529 consensus_queue.cc:260] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.575443 12529 ts_tablet_manager.cc:1434] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.041s	user 0.004s	sys 0.002s
I20260709 12:28:49.576464 12529 tablet_bootstrap.cc:492] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:49.582893 12531 raft_consensus.cc:359] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.584023 12531 raft_consensus.cc:385] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.584380 12531 raft_consensus.cc:740] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.587041 12644 raft_consensus.cc:493] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.587845 12644 raft_consensus.cc:515] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.589946 12529 tablet_bootstrap.cc:654] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.592809 12527 tablet_bootstrap.cc:492] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:49.593477 12527 ts_tablet_manager.cc:1403] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.201s	user 0.031s	sys 0.009s
I20260709 12:28:49.607275 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89bcc99e094d1bbaf83979c2b78081" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:49.606719 12527 raft_consensus.cc:359] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.608086 12527 raft_consensus.cc:385] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.608474 12527 raft_consensus.cc:740] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.609326 12527 consensus_queue.cc:260] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.612600 12527 ts_tablet_manager.cc:1434] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:28:49.613994 12527 tablet_bootstrap.cc:492] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:49.622416 12527 tablet_bootstrap.cc:654] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.632375 12644 leader_election.cc:290] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:49.634399 12480 raft_consensus.cc:3060] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.638595 12528 tablet_bootstrap.cc:654] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.649084 12529 tablet_bootstrap.cc:492] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:49.649705 12529 ts_tablet_manager.cc:1403] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.073s	user 0.015s	sys 0.010s
I20260709 12:28:49.651412 12527 tablet_bootstrap.cc:492] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:49.651943 12527 ts_tablet_manager.cc:1403] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.038s	user 0.015s	sys 0.000s
I20260709 12:28:49.652626 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89bcc99e094d1bbaf83979c2b78081" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:49.653672 12257 raft_consensus.cc:2393] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a800ae68e91f43a794e326badca3320a in current term 1: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:28:49.653522 12586 consensus_queue.cc:1048] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.655123 12447 leader_election.cc:304] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:49.655400 12527 raft_consensus.cc:359] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.656149 12527 raft_consensus.cc:385] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.656424 12527 raft_consensus.cc:740] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.652701 12529 raft_consensus.cc:359] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.656883 12529 raft_consensus.cc:385] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.657320 12529 raft_consensus.cc:740] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.657207 12636 raft_consensus.cc:2749] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:49.657202 12527 consensus_queue.cc:260] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.658315 12529 consensus_queue.cc:260] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.660270 12527 ts_tablet_manager.cc:1434] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:28:49.660720 12529 ts_tablet_manager.cc:1434] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:28:49.661263 12527 tablet_bootstrap.cc:492] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:49.661695 12529 tablet_bootstrap.cc:492] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:49.669652 12586 consensus_queue.cc:1048] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:49.672174 12531 consensus_queue.cc:260] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.674122 12528 tablet_bootstrap.cc:492] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:49.674654 12528 ts_tablet_manager.cc:1403] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.163s	user 0.014s	sys 0.002s
I20260709 12:28:49.675510 12527 tablet_bootstrap.cc:654] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.677522 12528 raft_consensus.cc:359] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.678009 12531 ts_tablet_manager.cc:1434] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.159s	user 0.012s	sys 0.000s
I20260709 12:28:49.678342 12528 raft_consensus.cc:385] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.678754 12528 raft_consensus.cc:740] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.679500 12528 consensus_queue.cc:260] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.685084 12529 tablet_bootstrap.cc:654] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.679502 12531 tablet_bootstrap.cc:492] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:49.679973 12155 catalog_manager.cc:5697] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:49.690817 12155 catalog_manager.cc:5697] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:49.692358 12636 raft_consensus.cc:493] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.692914 12636 raft_consensus.cc:515] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.689458 12528 ts_tablet_manager.cc:1434] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 12:28:49.694700 12528 tablet_bootstrap.cc:492] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:49.698269 12636 leader_election.cc:290] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:49.699066 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4bdda56dfcd42de9ca75e3e5a9b6c73" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:28:49.700695 12450 leader_election.cc:343] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.701488 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4bdda56dfcd42de9ca75e3e5a9b6c73" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:49.704619 12447 leader_election.cc:343] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.705097 12447 leader_election.cc:304] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:49.708740 12529 tablet_bootstrap.cc:492] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:49.709245 12529 ts_tablet_manager.cc:1403] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.048s	user 0.016s	sys 0.002s
I20260709 12:28:49.708340 12158 catalog_manager.cc:5697] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:49.712244 12636 raft_consensus.cc:2749] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:49.712376 12529 raft_consensus.cc:359] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.713255 12529 raft_consensus.cc:385] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.713609 12529 raft_consensus.cc:740] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.716208 12528 tablet_bootstrap.cc:654] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.719070 12409 raft_consensus.cc:3060] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.719687 12529 consensus_queue.cc:260] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.722874 12529 ts_tablet_manager.cc:1434] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:28:49.724014 12529 tablet_bootstrap.cc:492] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:49.747094 12529 tablet_bootstrap.cc:654] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.750118 12531 tablet_bootstrap.cc:654] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.759620 12527 tablet_bootstrap.cc:492] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:49.760229 12527 ts_tablet_manager.cc:1403] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.099s	user 0.023s	sys 0.023s
I20260709 12:28:49.763371 12527 raft_consensus.cc:359] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.764086 12527 raft_consensus.cc:385] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.764353 12527 raft_consensus.cc:740] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.766589 12527 consensus_queue.cc:260] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.769708 12527 ts_tablet_manager.cc:1434] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:28:49.771171 12527 tablet_bootstrap.cc:492] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:49.773114 12531 tablet_bootstrap.cc:492] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:49.776906 12531 ts_tablet_manager.cc:1403] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.098s	user 0.021s	sys 0.003s
I20260709 12:28:49.780073 12527 tablet_bootstrap.cc:654] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.779997 12531 raft_consensus.cc:359] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.780908 12531 raft_consensus.cc:385] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.781251 12531 raft_consensus.cc:740] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.768901 12528 tablet_bootstrap.cc:492] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:49.782188 12528 ts_tablet_manager.cc:1403] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.088s	user 0.006s	sys 0.016s
I20260709 12:28:49.785343 12528 raft_consensus.cc:359] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.786115 12528 raft_consensus.cc:385] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.786424 12528 raft_consensus.cc:740] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.787134 12528 consensus_queue.cc:260] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.789101 12528 ts_tablet_manager.cc:1434] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:28:49.792793 12531 consensus_queue.cc:260] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.794382 12527 tablet_bootstrap.cc:492] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:49.794931 12527 ts_tablet_manager.cc:1403] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.007s
I20260709 12:28:49.795241 12531 ts_tablet_manager.cc:1434] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:28:49.796773 12531 tablet_bootstrap.cc:492] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:49.797724 12528 tablet_bootstrap.cc:492] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:49.798889 12527 raft_consensus.cc:359] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.799620 12527 raft_consensus.cc:385] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.799902 12527 raft_consensus.cc:740] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.800586 12527 consensus_queue.cc:260] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.803007 12527 ts_tablet_manager.cc:1434] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:28:49.804324 12527 tablet_bootstrap.cc:492] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:49.805754 12529 tablet_bootstrap.cc:492] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:49.806318 12529 ts_tablet_manager.cc:1403] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.083s	user 0.010s	sys 0.007s
I20260709 12:28:49.809360 12529 raft_consensus.cc:359] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.810698 12527 tablet_bootstrap.cc:654] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.810989 12636 raft_consensus.cc:493] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.811545 12636 raft_consensus.cc:515] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.814674 12636 leader_election.cc:290] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:49.815145 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff454851114445da8be05eda167efc4" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:49.815989 12261 raft_consensus.cc:2393] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a800ae68e91f43a794e326badca3320a in current term 1: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:28:49.816438 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff454851114445da8be05eda167efc4" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:49.818475 12447 leader_election.cc:304] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:28:49.819717 12636 raft_consensus.cc:3060] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.820364 12538 raft_consensus.cc:493] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.820955 12538 raft_consensus.cc:515] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.810174 12529 raft_consensus.cc:385] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.822783 12529 raft_consensus.cc:740] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.823498 12538 leader_election.cc:290] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:49.824090 12620 consensus_queue.cc:1048] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:49.825107 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf8c3b9ee65415785e06fb8c085ada7" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:49.825438 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf8c3b9ee65415785e06fb8c085ada7" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:49.826783 12373 leader_election.cc:343] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:49.827584 12373 leader_election.cc:343] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.823665 12529 consensus_queue.cc:260] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.828063 12373 leader_election.cc:304] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:28:49.830523 12538 raft_consensus.cc:2749] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:49.839962 12636 raft_consensus.cc:2749] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:49.842427 12529 ts_tablet_manager.cc:1434] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:28:49.863059 12529 tablet_bootstrap.cc:492] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:49.865149 12527 tablet_bootstrap.cc:492] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:49.865846 12527 ts_tablet_manager.cc:1403] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.062s	user 0.023s	sys 0.029s
I20260709 12:28:49.869422 12529 tablet_bootstrap.cc:654] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.869175 12527 raft_consensus.cc:359] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.870024 12527 raft_consensus.cc:385] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.870404 12527 raft_consensus.cc:740] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.870370 12632 consensus_queue.cc:1048] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.849357 12531 tablet_bootstrap.cc:654] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.850135 12528 tablet_bootstrap.cc:654] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.851466 12662 raft_consensus.cc:493] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.872524 12662 raft_consensus.cc:515] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
W20260709 12:28:49.854786 12227 consensus_peers.cc:597] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:49.861083 12409 raft_consensus.cc:3060] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.879516 12662 leader_election.cc:290] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:49.880369 12531 tablet_bootstrap.cc:492] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:49.880910 12531 ts_tablet_manager.cc:1403] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.084s	user 0.006s	sys 0.004s
I20260709 12:28:49.881265 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7916cfbec26b4115a9251411482bed41" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:49.882052 12410 raft_consensus.cc:2468] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:49.883358 12302 leader_election.cc:304] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:49.883641 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7916cfbec26b4115a9251411482bed41" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:49.884119 12531 raft_consensus.cc:359] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.884776 12632 raft_consensus.cc:2804] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.884848 12531 raft_consensus.cc:385] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.885371 12531 raft_consensus.cc:740] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.885337 12632 raft_consensus.cc:493] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.885995 12632 raft_consensus.cc:3060] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:49.887185 12299 leader_election.cc:343] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.890528 12529 tablet_bootstrap.cc:492] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:49.891077 12529 ts_tablet_manager.cc:1403] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.000s
I20260709 12:28:49.871551 12527 consensus_queue.cc:260] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.893121 12662 consensus_queue.cc:1048] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:49.894330 12531 consensus_queue.cc:260] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.895977 12632 raft_consensus.cc:515] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.921293 12527 ts_tablet_manager.cc:1434] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.055s	user 0.022s	sys 0.012s
I20260709 12:28:49.922540 12527 tablet_bootstrap.cc:492] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:49.923180 12662 raft_consensus.cc:493] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.923802 12662 raft_consensus.cc:515] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.925913 12538 raft_consensus.cc:493] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.926506 12538 raft_consensus.cc:515] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.928465 12662 leader_election.cc:290] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:49.928640 12538 leader_election.cc:290] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:49.929874 12527 tablet_bootstrap.cc:654] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.930917 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:49.931509 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:49.932155 12408 raft_consensus.cc:2468] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
W20260709 12:28:49.932592 12373 leader_election.cc:343] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.933220 12480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:49.934175 12662 consensus_queue.cc:1048] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:49.934721 12299 leader_election.cc:343] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.935007 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:49.935796 12334 raft_consensus.cc:2468] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:49.934194 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7916cfbec26b4115a9251411482bed41" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:49.937954 12302 leader_election.cc:304] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:28:49.939298 12373 leader_election.cc:304] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:28:49.940055 12666 raft_consensus.cc:2804] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.940188 12538 raft_consensus.cc:2804] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.940601 12666 raft_consensus.cc:493] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.940732 12538 raft_consensus.cc:493] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.941210 12666 raft_consensus.cc:3060] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.941351 12538 raft_consensus.cc:3060] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.943087 12529 raft_consensus.cc:359] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.943831 12529 raft_consensus.cc:385] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.944092 12529 raft_consensus.cc:740] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
W20260709 12:28:49.946260 12299 leader_election.cc:343] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.939327 12531 ts_tablet_manager.cc:1434] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.005s
I20260709 12:28:49.954634 12531 tablet_bootstrap.cc:492] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:49.956313 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7916cfbec26b4115a9251411482bed41" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:49.958199 12632 leader_election.cc:290] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:49.958297 12408 raft_consensus.cc:3060] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.966439 12531 tablet_bootstrap.cc:654] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.953187 12538 raft_consensus.cc:515] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:49.968611 12620 raft_consensus.cc:493] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.970144 12620 raft_consensus.cc:515] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:49.970571 12538 leader_election.cc:290] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:49.971239 12586 raft_consensus.cc:493] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.971792 12586 raft_consensus.cc:515] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:49.974026 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:49.974673 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:49.975714 12483 raft_consensus.cc:2468] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:49.975934 12480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:49.977758 12224 leader_election.cc:304] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
W20260709 12:28:49.978451 12373 leader_election.cc:343] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.979214 12594 raft_consensus.cc:2804] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.979851 12594 raft_consensus.cc:493] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.980379 12594 raft_consensus.cc:3060] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.980470 12480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:49.982949 12586 leader_election.cc:290] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:49.983656 12531 tablet_bootstrap.cc:492] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:49.984215 12531 ts_tablet_manager.cc:1403] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.005s
I20260709 12:28:49.984978 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:49.986573 12408 raft_consensus.cc:2468] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:49.956249 12529 consensus_queue.cc:260] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.988104 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:49.992887 12336 raft_consensus.cc:2468] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:49.994395 12663 raft_consensus.cc:493] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.994918 12663 raft_consensus.cc:515] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:49.996012 12302 leader_election.cc:304] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:49.986714 12373 leader_election.cc:304] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:28:49.997892 12662 raft_consensus.cc:2804] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:49.998375 12662 raft_consensus.cc:697] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:49.999275 12538 raft_consensus.cc:2749] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:49.999228 12662 consensus_queue.cc:237] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:50.004330 12663 leader_election.cc:290] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:50.006704 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd19eddb6c2e45b7acefc5c295d92928" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:50.007299 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd19eddb6c2e45b7acefc5c295d92928" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:50.007541 12410 raft_consensus.cc:2468] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:49.990314 12531 raft_consensus.cc:359] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.010424 12531 raft_consensus.cc:385] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.011116 12302 leader_election.cc:304] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
W20260709 12:28:50.011806 12299 leader_election.cc:343] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.012295 12632 raft_consensus.cc:2804] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.012781 12632 raft_consensus.cc:493] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.013164 12632 raft_consensus.cc:3060] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.013504 12624 consensus_queue.cc:1048] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.015977 12529 ts_tablet_manager.cc:1434] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.125s	user 0.008s	sys 0.000s
I20260709 12:28:49.988866 12620 leader_election.cc:290] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:50.025607 12531 raft_consensus.cc:740] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.026575 12620 consensus_queue.cc:1048] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:50.027637 12531 consensus_queue.cc:260] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.029260 12624 raft_consensus.cc:493] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.030063 12624 raft_consensus.cc:515] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.035156 12624 leader_election.cc:290] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:50.035344 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:28:50.037218 12302 leader_election.cc:343] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.038710 12531 ts_tablet_manager.cc:1434] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.054s	user 0.002s	sys 0.005s
I20260709 12:28:50.017073 12529 tablet_bootstrap.cc:492] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:50.042477 12632 raft_consensus.cc:515] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:50.039937 12531 tablet_bootstrap.cc:492] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:50.046329 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd19eddb6c2e45b7acefc5c295d92928" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:50.047000 12410 raft_consensus.cc:3060] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.049935 12632 leader_election.cc:290] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:50.054515 12531 tablet_bootstrap.cc:654] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.061213 12620 consensus_queue.cc:1048] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.067046 12528 tablet_bootstrap.cc:492] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:50.067723 12528 ts_tablet_manager.cc:1403] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.270s	user 0.015s	sys 0.011s
I20260709 12:28:50.068785 12531 tablet_bootstrap.cc:492] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:50.069322 12531 ts_tablet_manager.cc:1403] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.008s
I20260709 12:28:50.072335 12531 raft_consensus.cc:359] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.073096 12531 raft_consensus.cc:385] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.073431 12531 raft_consensus.cc:740] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.076764 12531 consensus_queue.cc:260] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.079152 12531 ts_tablet_manager.cc:1434] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:28:50.080206 12531 tablet_bootstrap.cc:492] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:50.081056 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd19eddb6c2e45b7acefc5c295d92928" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:50.083618 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:50.084537 12484 raft_consensus.cc:2468] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:50.088637 12528 raft_consensus.cc:359] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:50.089527 12528 raft_consensus.cc:385] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.096884 12528 raft_consensus.cc:740] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.096917 12410 raft_consensus.cc:2468] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:50.098268 12302 leader_election.cc:304] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:50.096506 12531 tablet_bootstrap.cc:654] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.099304 12624 raft_consensus.cc:2804] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.088310 12666 raft_consensus.cc:515] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.100219 12624 raft_consensus.cc:697] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:50.100020 12594 raft_consensus.cc:515] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:50.101023 12334 raft_consensus.cc:2393] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2104c8a9b144a419ccb7c07e3f02596 in current term 1: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:28:50.104796 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:50.105796 12409 raft_consensus.cc:2393] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 in current term 1: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:28:50.106009 12373 leader_election.cc:304] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:28:50.107788 12586 raft_consensus.cc:2749] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.101336 12624 consensus_queue.cc:237] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:50.095402 12299 leader_election.cc:304] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:50.112272 12662 raft_consensus.cc:2804] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.097819 12528 consensus_queue.cc:260] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:50.112670 12662 raft_consensus.cc:493] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.113197 12662 raft_consensus.cc:3060] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.114923 12528 ts_tablet_manager.cc:1434] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.047s	user 0.008s	sys 0.000s
I20260709 12:28:50.116549 12529 tablet_bootstrap.cc:654] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:50.131081 12299 leader_election.cc:343] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.139686 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:50.140661 12483 raft_consensus.cc:3060] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.126864 12662 raft_consensus.cc:515] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.149271 12157 catalog_manager.cc:5697] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.154599 12594 leader_election.cc:290] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:50.163538 12528 tablet_bootstrap.cc:492] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:50.169427 12531 tablet_bootstrap.cc:492] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:50.169967 12531 ts_tablet_manager.cc:1403] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.090s	user 0.031s	sys 0.007s
I20260709 12:28:50.162068 12666 leader_election.cc:290] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:50.171717 12620 raft_consensus.cc:493] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.165915 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:50.172468 12334 raft_consensus.cc:3060] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.172297 12620 raft_consensus.cc:515] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.174403 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:50.175887 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:50.176085 12681 raft_consensus.cc:493] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:50.176266 12302 leader_election.cc:343] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.172286 12662 leader_election.cc:290] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:50.176645 12681 raft_consensus.cc:515] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
W20260709 12:28:50.177459 12227 leader_election.cc:343] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.178855 12681 leader_election.cc:290] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:50.179713 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:50.179903 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd19eddb6c2e45b7acefc5c295d92928" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:50.180418 12484 raft_consensus.cc:2468] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:50.188359 12527 tablet_bootstrap.cc:492] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:50.182763 12620 leader_election.cc:290] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:50.189167 12527 ts_tablet_manager.cc:1403] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.267s	user 0.055s	sys 0.117s
I20260709 12:28:50.192847 12527 raft_consensus.cc:359] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.193835 12527 raft_consensus.cc:385] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.194911 12527 raft_consensus.cc:740] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.196149 12529 tablet_bootstrap.cc:492] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:50.196699 12529 ts_tablet_manager.cc:1403] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.180s	user 0.014s	sys 0.015s
I20260709 12:28:50.198499 12224 leader_election.cc:304] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:50.200073 12681 raft_consensus.cc:2804] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.200456 12681 raft_consensus.cc:493] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.200776 12681 raft_consensus.cc:3060] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.205281 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd19eddb6c2e45b7acefc5c295d92928" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:50.205628 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:50.206213 12410 raft_consensus.cc:2393] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1dcded10caa0493da818e67e13176979 in current term 1: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:28:50.206373 12484 raft_consensus.cc:3060] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.208070 12480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:50.208772 12480 raft_consensus.cc:3060] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.203912 12483 raft_consensus.cc:2468] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:50.212582 12224 leader_election.cc:304] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:50.214879 12620 raft_consensus.cc:2804] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.214957 12528 tablet_bootstrap.cc:654] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.215631 12620 raft_consensus.cc:697] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:50.223721 12620 consensus_queue.cc:237] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:50.228906 12227 leader_election.cc:304] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:50.229925 12594 raft_consensus.cc:3060] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.222041 12334 raft_consensus.cc:2468] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:50.230439 12527 consensus_queue.cc:260] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.224536 12531 raft_consensus.cc:359] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.236606 12531 raft_consensus.cc:385] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.236161 12484 raft_consensus.cc:2468] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:50.217594 12480 raft_consensus.cc:2468] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:50.236923 12531 raft_consensus.cc:740] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.238155 12681 raft_consensus.cc:515] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.238984 12299 leader_election.cc:304] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:50.240056 12666 raft_consensus.cc:2804] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.240556 12666 raft_consensus.cc:697] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:50.241398 12666 consensus_queue.cc:237] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.241446 12529 raft_consensus.cc:359] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.242653 12529 raft_consensus.cc:385] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.243049 12529 raft_consensus.cc:740] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.243177 12586 raft_consensus.cc:493] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.243753 12586 raft_consensus.cc:515] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.245934 12299 leader_election.cc:304] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:50.247164 12662 raft_consensus.cc:2804] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.247695 12662 raft_consensus.cc:697] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:50.248023 12409 raft_consensus.cc:3060] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.248778 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:50.238942 12531 consensus_queue.cc:260] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.250130 12529 consensus_queue.cc:260] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.260947 12480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:50.261854 12480 raft_consensus.cc:3060] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.262691 12586 leader_election.cc:290] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
W20260709 12:28:50.264508 12227 leader_election.cc:343] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.265527 12157 catalog_manager.cc:5697] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.266314 12594 raft_consensus.cc:2749] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.249706 12662 consensus_queue.cc:237] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.265657 12681 leader_election.cc:290] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:50.282356 12529 ts_tablet_manager.cc:1434] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.085s	user 0.007s	sys 0.002s
I20260709 12:28:50.283350 12531 ts_tablet_manager.cc:1434] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.113s	user 0.006s	sys 0.002s
I20260709 12:28:50.283455 12529 tablet_bootstrap.cc:492] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:50.284931 12531 tablet_bootstrap.cc:492] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:50.306020 12528 tablet_bootstrap.cc:492] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:50.306581 12528 ts_tablet_manager.cc:1403] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.143s	user 0.024s	sys 0.007s
I20260709 12:28:50.314622 12528 raft_consensus.cc:359] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.315585 12528 raft_consensus.cc:385] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.314684 12155 catalog_manager.cc:5697] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.315866 12528 raft_consensus.cc:740] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.316690 12528 consensus_queue.cc:260] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
W20260709 12:28:50.317492 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 65577us
I20260709 12:28:50.318926 12531 tablet_bootstrap.cc:654] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.319981 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:50.320616 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:50.321476 12483 raft_consensus.cc:2393] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2104c8a9b144a419ccb7c07e3f02596 in current term 1: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:28:50.323024 12527 ts_tablet_manager.cc:1434] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.133s	user 0.011s	sys 0.004s
I20260709 12:28:50.325865 12527 tablet_bootstrap.cc:492] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:50.327498 12529 tablet_bootstrap.cc:654] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.332294 12528 ts_tablet_manager.cc:1434] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:28:50.341912 12527 tablet_bootstrap.cc:654] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.336061 12480 raft_consensus.cc:2468] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:50.346117 12224 leader_election.cc:304] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:50.349107 12594 raft_consensus.cc:2804] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.348532 12158 catalog_manager.cc:5697] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.338140 12373 leader_election.cc:304] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:28:50.349675 12594 raft_consensus.cc:697] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:50.350798 12158 catalog_manager.cc:5697] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.351269 12594 consensus_queue.cc:237] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.353157 12723 raft_consensus.cc:3060] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.365541 12527 tablet_bootstrap.cc:492] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:50.366430 12527 ts_tablet_manager.cc:1403] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.043s	user 0.010s	sys 0.004s
I20260709 12:28:50.367275 12531 tablet_bootstrap.cc:492] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:50.367498 12528 tablet_bootstrap.cc:492] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:50.367799 12531 ts_tablet_manager.cc:1403] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.083s	user 0.013s	sys 0.005s
I20260709 12:28:50.369288 12527 raft_consensus.cc:359] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:50.370093 12527 raft_consensus.cc:385] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.370368 12527 raft_consensus.cc:740] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.371052 12527 consensus_queue.cc:260] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:50.371212 12531 raft_consensus.cc:359] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.373440 12527 ts_tablet_manager.cc:1434] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:50.374017 12723 raft_consensus.cc:2749] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.374598 12527 tablet_bootstrap.cc:492] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:50.374848 12531 raft_consensus.cc:385] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.375211 12531 raft_consensus.cc:740] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.376215 12531 consensus_queue.cc:260] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.379423 12529 tablet_bootstrap.cc:492] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:50.379951 12529 ts_tablet_manager.cc:1403] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.097s	user 0.015s	sys 0.001s
I20260709 12:28:50.380110 12528 tablet_bootstrap.cc:654] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.382000 12527 tablet_bootstrap.cc:654] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.384547 12529 raft_consensus.cc:359] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.385260 12529 raft_consensus.cc:385] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.388036 12531 ts_tablet_manager.cc:1434] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 12:28:50.385521 12529 raft_consensus.cc:740] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.390061 12531 tablet_bootstrap.cc:492] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:50.393915 12529 consensus_queue.cc:260] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.396409 12529 ts_tablet_manager.cc:1434] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 12:28:50.402036 12529 tablet_bootstrap.cc:492] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:50.408568 12529 tablet_bootstrap.cc:654] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.411561 12409 raft_consensus.cc:3060] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.419725 12531 tablet_bootstrap.cc:654] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.425320 12527 tablet_bootstrap.cc:492] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:50.425850 12527 ts_tablet_manager.cc:1403] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.052s	user 0.019s	sys 0.023s
I20260709 12:28:50.427680 12528 tablet_bootstrap.cc:492] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:50.429997 12528 ts_tablet_manager.cc:1403] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.000s
I20260709 12:28:50.428593 12527 raft_consensus.cc:359] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.433096 12527 raft_consensus.cc:385] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.433521 12527 raft_consensus.cc:740] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.434587 12527 consensus_queue.cc:260] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.435647 12158 catalog_manager.cc:5697] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:50.440160 12528 raft_consensus.cc:359] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:50.440917 12528 raft_consensus.cc:385] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.441169 12528 raft_consensus.cc:740] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.441969 12528 consensus_queue.cc:260] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:50.442461 12527 ts_tablet_manager.cc:1434] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 12:28:50.443548 12527 tablet_bootstrap.cc:492] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:50.444458 12528 ts_tablet_manager.cc:1434] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:28:50.445482 12528 tablet_bootstrap.cc:492] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:50.451187 12527 tablet_bootstrap.cc:654] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.451556 12528 tablet_bootstrap.cc:654] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.453390 12531 tablet_bootstrap.cc:492] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:50.454295 12531 ts_tablet_manager.cc:1403] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.065s	user 0.020s	sys 0.006s
I20260709 12:28:50.458261 12531 raft_consensus.cc:359] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:50.459206 12531 raft_consensus.cc:385] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.459553 12531 raft_consensus.cc:740] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.461964 12531 consensus_queue.cc:260] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:50.464697 12531 ts_tablet_manager.cc:1434] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:28:50.465940 12531 tablet_bootstrap.cc:492] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:50.489811 12636 raft_consensus.cc:493] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.490795 12636 raft_consensus.cc:515] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.499238 12531 tablet_bootstrap.cc:654] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.500459 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:50.504087 12529 tablet_bootstrap.cc:492] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:50.504616 12529 ts_tablet_manager.cc:1403] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.103s	user 0.013s	sys 0.000s
I20260709 12:28:50.507571 12529 raft_consensus.cc:359] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
W20260709 12:28:50.503947 12450 leader_election.cc:343] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.508267 12529 raft_consensus.cc:385] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.508555 12529 raft_consensus.cc:740] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.510951 12636 leader_election.cc:290] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:50.512923 12529 consensus_queue.cc:260] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:50.517241 12529 ts_tablet_manager.cc:1434] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:28:50.518218 12529 tablet_bootstrap.cc:492] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:50.524530 12528 tablet_bootstrap.cc:492] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:50.525149 12528 ts_tablet_manager.cc:1403] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.080s	user 0.019s	sys 0.028s
I20260709 12:28:50.524587 12527 tablet_bootstrap.cc:492] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:50.526418 12527 ts_tablet_manager.cc:1403] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.083s	user 0.015s	sys 0.027s
I20260709 12:28:50.528270 12528 raft_consensus.cc:359] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.529006 12528 raft_consensus.cc:385] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.529275 12528 raft_consensus.cc:740] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.531213 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:50.532495 12527 raft_consensus.cc:359] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.537596 12527 raft_consensus.cc:385] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.537896 12527 raft_consensus.cc:740] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.538609 12527 consensus_queue.cc:260] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.540901 12527 ts_tablet_manager.cc:1434] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 12:28:50.542068 12527 tablet_bootstrap.cc:492] T fe640756185b42d7874674a29af747da P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:50.538997 12528 consensus_queue.cc:260] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.537084 12529 tablet_bootstrap.cc:654] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.545989 12528 ts_tablet_manager.cc:1434] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.001s
I20260709 12:28:50.547130 12528 tablet_bootstrap.cc:492] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
W20260709 12:28:50.547353 12447 leader_election.cc:343] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.547938 12447 leader_election.cc:304] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:50.550128 12738 raft_consensus.cc:2749] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.553331 12527 tablet_bootstrap.cc:654] T fe640756185b42d7874674a29af747da P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.558357 12738 raft_consensus.cc:493] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.559038 12738 raft_consensus.cc:515] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.565914 12528 tablet_bootstrap.cc:654] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.572006 12738 leader_election.cc:290] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:50.575088 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe640756185b42d7874674a29af747da" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:50.576037 12334 raft_consensus.cc:2468] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:50.577524 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe640756185b42d7874674a29af747da" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:28:50.587267 12450 leader_election.cc:343] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.585071 12666 consensus_queue.cc:1048] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:28:50.595341 12257 raft_consensus.cc:3060] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.596527 12447 leader_election.cc:304] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:50.597544 12738 raft_consensus.cc:2804] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.599359 12738 raft_consensus.cc:493] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.599707 12738 raft_consensus.cc:3060] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.608875 12738 raft_consensus.cc:515] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.612426 12738 leader_election.cc:290] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:50.616822 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe640756185b42d7874674a29af747da" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:50.617863 12334 raft_consensus.cc:3060] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.627341 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe640756185b42d7874674a29af747da" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
W20260709 12:28:50.631889 12450 leader_election.cc:343] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.640348 12662 consensus_queue.cc:1048] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.659232 12662 consensus_queue.cc:1048] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.672281 12662 consensus_queue.cc:1048] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.678232 12666 consensus_queue.cc:1048] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.682751 12620 raft_consensus.cc:493] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.683418 12620 raft_consensus.cc:515] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.686023 12620 leader_election.cc:290] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:50.686868 12624 raft_consensus.cc:493] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.687494 12624 raft_consensus.cc:515] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.689147 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4bdda56dfcd42de9ca75e3e5a9b6c73" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:50.690135 12483 raft_consensus.cc:2468] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:50.692253 12224 leader_election.cc:304] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:50.692457 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a81175bbecf4ba6b6be50b490564d9b" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:50.693508 12620 raft_consensus.cc:2804] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.694059 12620 raft_consensus.cc:493] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.694531 12620 raft_consensus.cc:3060] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:50.694881 12302 leader_election.cc:343] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.697134 12624 leader_election.cc:290] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:50.699982 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a81175bbecf4ba6b6be50b490564d9b" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
W20260709 12:28:50.701997 12299 leader_election.cc:343] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.702647 12299 leader_election.cc:304] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
I20260709 12:28:50.703831 12624 raft_consensus.cc:2749] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.715015 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4bdda56dfcd42de9ca75e3e5a9b6c73" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:50.716065 12409 raft_consensus.cc:2468] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:50.743554 12662 consensus_queue.cc:1048] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.797474 12751 raft_consensus.cc:493] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.802538 12751 raft_consensus.cc:515] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.800130 12723 raft_consensus.cc:493] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.805352 12723 raft_consensus.cc:515] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.814858 12334 raft_consensus.cc:2468] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:50.815696 12723 leader_election.cc:290] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:50.817859 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf8c3b9ee65415785e06fb8c085ada7" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:50.819149 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:50.821424 12750 consensus_queue.cc:1048] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:50.821903 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:50.829895 12751 leader_election.cc:290] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:50.883544 12481 raft_consensus.cc:2393] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2104c8a9b144a419ccb7c07e3f02596 in current term 1: Already voted for candidate 1dcded10caa0493da818e67e13176979 in this term.
I20260709 12:28:50.836207 12620 raft_consensus.cc:515] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.899492 12531 tablet_bootstrap.cc:492] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:50.900291 12531 ts_tablet_manager.cc:1403] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.435s	user 0.002s	sys 0.016s
W20260709 12:28:50.902220 12753 log.cc:927] Time spent T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979: Append to log took a long time: real 0.059s	user 0.000s	sys 0.003s
I20260709 12:28:50.919309 12632 raft_consensus.cc:493] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.920212 12620 leader_election.cc:290] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:50.920161 12632 raft_consensus.cc:515] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:50.921552 12447 leader_election.cc:304] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:50.922920 12738 raft_consensus.cc:2804] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.923413 12738 raft_consensus.cc:697] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:50.923723 12666 consensus_queue.cc:1048] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.924243 12738 consensus_queue.cc:237] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:50.904662 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf8c3b9ee65415785e06fb8c085ada7" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:50.935181 12529 tablet_bootstrap.cc:492] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:50.936583 12410 raft_consensus.cc:2468] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:50.937672 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:50.938246 12739 raft_consensus.cc:493] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.938935 12527 tablet_bootstrap.cc:492] T fe640756185b42d7874674a29af747da P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
W20260709 12:28:50.915493 12224 leader_election.cc:343] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:50.902837 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 79175.3us
I20260709 12:28:50.940938 12529 ts_tablet_manager.cc:1403] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.423s	user 0.021s	sys 0.083s
I20260709 12:28:50.943135 12527 ts_tablet_manager.cc:1403] T fe640756185b42d7874674a29af747da P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.401s	user 0.025s	sys 0.075s
I20260709 12:28:50.952534 12739 raft_consensus.cc:515] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:50.954344 12528 tablet_bootstrap.cc:492] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:50.954924 12528 ts_tablet_manager.cc:1403] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.408s	user 0.097s	sys 0.122s
I20260709 12:28:50.955005 12739 leader_election.cc:290] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:50.958420 12632 leader_election.cc:290] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
W20260709 12:28:50.956749 12224 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 47855.1us
I20260709 12:28:50.971951 12531 raft_consensus.cc:359] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.974913 12531 raft_consensus.cc:385] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.975318 12531 raft_consensus.cc:740] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.959666 12484 raft_consensus.cc:2468] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:50.979457 12531 consensus_queue.cc:260] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.986073 12528 raft_consensus.cc:359] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.988168 12528 raft_consensus.cc:385] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.988747 12531 ts_tablet_manager.cc:1434] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.088s	user 0.006s	sys 0.005s
I20260709 12:28:50.988533 12528 raft_consensus.cc:740] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.993232 12528 consensus_queue.cc:260] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:50.999511 12527 raft_consensus.cc:359] T fe640756185b42d7874674a29af747da P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.000322 12527 raft_consensus.cc:385] T fe640756185b42d7874674a29af747da P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.000622 12527 raft_consensus.cc:740] T fe640756185b42d7874674a29af747da P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.001320 12527 consensus_queue.cc:260] T fe640756185b42d7874674a29af747da P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.004001 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:51.004745 12334 raft_consensus.cc:2468] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:51.006937 12227 leader_election.cc:304] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:28:50.998863 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4bdda56dfcd42de9ca75e3e5a9b6c73" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:51.010731 12528 ts_tablet_manager.cc:1434] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.000s
I20260709 12:28:51.013996 12480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4bdda56dfcd42de9ca75e3e5a9b6c73" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:51.014520 12528 tablet_bootstrap.cc:492] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:51.014655 12480 raft_consensus.cc:3060] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.011801 12751 raft_consensus.cc:2804] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.015336 12751 raft_consensus.cc:493] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.015692 12751 raft_consensus.cc:3060] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.020766 12407 raft_consensus.cc:3060] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.022817 12447 leader_election.cc:304] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:51.023938 12739 raft_consensus.cc:2804] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.024287 12739 raft_consensus.cc:493] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.024597 12739 raft_consensus.cc:3060] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.025431 12480 raft_consensus.cc:2468] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:51.026417 12679 consensus_queue.cc:1048] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.029034 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:51.035506 12259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:51.036289 12407 raft_consensus.cc:2468] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:51.036675 12410 raft_consensus.cc:3060] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:51.037844 12299 leader_election.cc:343] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.038141 12528 tablet_bootstrap.cc:654] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.046979 12527 ts_tablet_manager.cc:1434] T fe640756185b42d7874674a29af747da P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.000s
I20260709 12:28:51.048518 12527 tablet_bootstrap.cc:492] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:51.053082 12529 raft_consensus.cc:359] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.056775 12529 raft_consensus.cc:385] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.057096 12529 raft_consensus.cc:740] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.059404 12529 consensus_queue.cc:260] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.054216 12739 raft_consensus.cc:515] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.066377 12224 leader_election.cc:304] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:51.068212 12739 leader_election.cc:290] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:28:51.069900 12447 leader_election.cc:343] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:51.071029 12759 log.cc:927] Time spent T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a: Append to log took a long time: real 0.067s	user 0.000s	sys 0.002s
I20260709 12:28:51.074970 12529 ts_tablet_manager.cc:1434] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.118s	user 0.006s	sys 0.001s
I20260709 12:28:51.078174 12259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:51.079305 12681 raft_consensus.cc:2804] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.079871 12681 raft_consensus.cc:697] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:51.079972 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:51.080641 12334 raft_consensus.cc:3060] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.080812 12681 consensus_queue.cc:237] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.084561 12529 tablet_bootstrap.cc:492] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
W20260709 12:28:51.091310 12758 log.cc:927] Time spent T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a: Append to log took a long time: real 0.087s	user 0.003s	sys 0.000s
I20260709 12:28:51.092024 12755 consensus_queue.cc:1048] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:28:51.097254 12447 leader_election.cc:343] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.087361 12373 leader_election.cc:304] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:28:51.110828 12751 raft_consensus.cc:515] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.113293 12751 leader_election.cc:290] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:51.113890 12620 consensus_queue.cc:1048] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.119174 12531 tablet_bootstrap.cc:492] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:51.120306 12754 raft_consensus.cc:2749] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:51.124220 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf8c3b9ee65415785e06fb8c085ada7" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:51.124989 12481 raft_consensus.cc:3060] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:51.125203 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 48027.5us
I20260709 12:28:51.126609 12527 tablet_bootstrap.cc:654] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.138855 12529 tablet_bootstrap.cc:654] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.139760 12531 tablet_bootstrap.cc:654] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:51.131039 12206 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:28:51.137897 12227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 48991.9us
I20260709 12:28:51.149801 12528 tablet_bootstrap.cc:492] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:51.150538 12528 ts_tablet_manager.cc:1403] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.138s	user 0.015s	sys 0.015s
I20260709 12:28:51.151093 12334 raft_consensus.cc:2468] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:51.153024 12447 leader_election.cc:304] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:51.156482 12299 leader_election.cc:304] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:51.159502 12739 raft_consensus.cc:2804] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.160094 12739 raft_consensus.cc:697] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:51.160429 12662 raft_consensus.cc:2764] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:28:51.160987 12739 consensus_queue.cc:237] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.168455 12620 consensus_queue.cc:1048] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.170394 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf8c3b9ee65415785e06fb8c085ada7" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:51.182750 12529 tablet_bootstrap.cc:492] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:51.183336 12529 ts_tablet_manager.cc:1403] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.106s	user 0.015s	sys 0.001s
I20260709 12:28:51.187124 12529 raft_consensus.cc:359] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.187913 12529 raft_consensus.cc:385] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.176805 12407 raft_consensus.cc:3060] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.188194 12529 raft_consensus.cc:740] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.188385 12155 catalog_manager.cc:5697] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:51.189254 12529 consensus_queue.cc:260] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.198661 12528 raft_consensus.cc:359] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.199381 12528 raft_consensus.cc:385] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.199635 12528 raft_consensus.cc:740] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.202454 12528 consensus_queue.cc:260] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.215693 12407 raft_consensus.cc:2468] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:51.219435 12529 ts_tablet_manager.cc:1434] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.000s
I20260709 12:28:51.219748 12227 leader_election.cc:304] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:51.220837 12529 tablet_bootstrap.cc:492] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:51.221983 12681 raft_consensus.cc:2804] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.222569 12681 raft_consensus.cc:697] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:51.223483 12681 consensus_queue.cc:237] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.232489 12531 tablet_bootstrap.cc:492] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:51.233093 12531 ts_tablet_manager.cc:1403] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.114s	user 0.015s	sys 0.000s
I20260709 12:28:51.235622 12528 ts_tablet_manager.cc:1434] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.085s	user 0.007s	sys 0.001s
I20260709 12:28:51.238333 12528 tablet_bootstrap.cc:492] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:51.239964 12531 raft_consensus.cc:359] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.241093 12531 raft_consensus.cc:385] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.241392 12531 raft_consensus.cc:740] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.242415 12531 consensus_queue.cc:260] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.245018 12529 tablet_bootstrap.cc:654] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.252106 12531 ts_tablet_manager.cc:1434] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:28:51.254182 12481 raft_consensus.cc:2468] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:51.261050 12531 tablet_bootstrap.cc:492] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:51.261897 12527 tablet_bootstrap.cc:492] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:51.262449 12527 ts_tablet_manager.cc:1403] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.214s	user 0.024s	sys 0.015s
I20260709 12:28:51.267110 12527 raft_consensus.cc:359] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.267684 12527 raft_consensus.cc:385] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.267940 12527 raft_consensus.cc:740] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.270219 12527 consensus_queue.cc:260] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.271868 12531 tablet_bootstrap.cc:654] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.277872 12527 ts_tablet_manager.cc:1434] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:28:51.279311 12527 tablet_bootstrap.cc:492] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:51.292457 12528 tablet_bootstrap.cc:654] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.295370 12527 tablet_bootstrap.cc:654] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.304494 12531 tablet_bootstrap.cc:492] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:51.304980 12531 ts_tablet_manager.cc:1403] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.014s
I20260709 12:28:51.311170 12529 tablet_bootstrap.cc:492] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:51.310989 12531 raft_consensus.cc:359] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.311995 12531 raft_consensus.cc:385] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.312175 12529 ts_tablet_manager.cc:1403] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.092s	user 0.028s	sys 0.001s
I20260709 12:28:51.312292 12531 raft_consensus.cc:740] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.312925 12527 tablet_bootstrap.cc:492] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:51.313400 12527 ts_tablet_manager.cc:1403] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.000s
I20260709 12:28:51.313247 12531 consensus_queue.cc:260] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.315078 12529 raft_consensus.cc:359] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.315722 12531 ts_tablet_manager.cc:1434] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:28:51.316061 12527 raft_consensus.cc:359] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.316720 12531 tablet_bootstrap.cc:492] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:51.319315 12528 tablet_bootstrap.cc:492] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:51.319864 12528 ts_tablet_manager.cc:1403] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.083s	user 0.009s	sys 0.006s
I20260709 12:28:51.320279 12681 raft_consensus.cc:493] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.321009 12681 raft_consensus.cc:515] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.315757 12529 raft_consensus.cc:385] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.322047 12529 raft_consensus.cc:740] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.322777 12529 consensus_queue.cc:260] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.323931 12531 tablet_bootstrap.cc:654] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.325006 12528 raft_consensus.cc:359] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.325750 12528 raft_consensus.cc:385] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.325543 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:51.326201 12528 raft_consensus.cc:740] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.316708 12527 raft_consensus.cc:385] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.329406 12527 raft_consensus.cc:740] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.330724 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:51.330942 12681 leader_election.cc:290] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:51.331622 12333 raft_consensus.cc:2393] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1dcded10caa0493da818e67e13176979 in current term 1: Already voted for candidate a800ae68e91f43a794e326badca3320a in this term.
I20260709 12:28:51.331393 12528 consensus_queue.cc:260] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.333268 12224 leader_election.cc:304] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:28:51.333635 12528 ts_tablet_manager.cc:1434] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:28:51.334211 12681 raft_consensus.cc:3060] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.334815 12528 tablet_bootstrap.cc:492] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:51.330543 12527 consensus_queue.cc:260] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.339273 12527 ts_tablet_manager.cc:1434] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:28:51.340507 12527 tablet_bootstrap.cc:492] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:51.342276 12739 consensus_queue.cc:1048] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.342881 12681 raft_consensus.cc:2749] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:51.345825 12529 ts_tablet_manager.cc:1434] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 12:28:51.347028 12529 tablet_bootstrap.cc:492] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:51.349478 12662 raft_consensus.cc:493] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.350124 12662 raft_consensus.cc:515] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.353112 12529 tablet_bootstrap.cc:654] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.354103 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13100e49825f4fcfa78791b73b8d7555" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:51.360982 12299 leader_election.cc:343] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.353891 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13100e49825f4fcfa78791b73b8d7555" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:51.362432 12410 raft_consensus.cc:2468] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:51.357697 12662 leader_election.cc:290] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:51.364408 12302 leader_election.cc:304] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:51.365788 12662 raft_consensus.cc:2804] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.366211 12662 raft_consensus.cc:493] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.366586 12662 raft_consensus.cc:3060] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.369994 12410 raft_consensus.cc:3060] T fe640756185b42d7874674a29af747da P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.375342 12662 raft_consensus.cc:515] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.378759 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13100e49825f4fcfa78791b73b8d7555" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:51.382058 12154 catalog_manager.cc:5697] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:51.388235 12299 leader_election.cc:343] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.389113 12527 tablet_bootstrap.cc:654] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.389933 12528 tablet_bootstrap.cc:654] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.390508 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13100e49825f4fcfa78791b73b8d7555" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:51.391211 12407 raft_consensus.cc:3060] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.384025 12154 catalog_manager.cc:5697] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.397812 12528 tablet_bootstrap.cc:492] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:51.399384 12662 leader_election.cc:290] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:51.399860 12528 ts_tablet_manager.cc:1403] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.065s	user 0.013s	sys 0.000s
I20260709 12:28:51.403306 12528 raft_consensus.cc:359] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.404066 12528 raft_consensus.cc:385] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.404349 12528 raft_consensus.cc:740] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.399281 12529 tablet_bootstrap.cc:492] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:51.406675 12529 ts_tablet_manager.cc:1403] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.060s	user 0.007s	sys 0.005s
I20260709 12:28:51.405085 12528 consensus_queue.cc:260] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.399281 12407 raft_consensus.cc:2468] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:51.409637 12529 raft_consensus.cc:359] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.410044 12636 consensus_queue.cc:1048] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.410621 12529 raft_consensus.cc:385] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.410979 12529 raft_consensus.cc:740] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.411808 12529 consensus_queue.cc:260] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.412675 12528 ts_tablet_manager.cc:1434] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:28:51.413283 12302 leader_election.cc:304] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:51.413854 12528 tablet_bootstrap.cc:492] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:51.415238 12529 ts_tablet_manager.cc:1434] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:28:51.415185 12632 raft_consensus.cc:2804] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.415954 12632 raft_consensus.cc:697] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:51.420715 12632 consensus_queue.cc:237] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.423601 12531 tablet_bootstrap.cc:492] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:51.424587 12531 ts_tablet_manager.cc:1403] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.108s	user 0.022s	sys 0.005s
I20260709 12:28:51.428069 12529 tablet_bootstrap.cc:492] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:51.430162 12158 catalog_manager.cc:5697] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:51.440587 12531 raft_consensus.cc:359] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.441659 12531 raft_consensus.cc:385] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.442008 12531 raft_consensus.cc:740] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.442873 12531 consensus_queue.cc:260] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.447949 12528 tablet_bootstrap.cc:654] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.449587 12531 ts_tablet_manager.cc:1434] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:28:51.450649 12531 tablet_bootstrap.cc:492] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:51.463573 12529 tablet_bootstrap.cc:654] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.470480 12159 catalog_manager.cc:5697] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.477736 12527 tablet_bootstrap.cc:492] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:51.478367 12527 ts_tablet_manager.cc:1403] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.138s	user 0.025s	sys 0.013s
I20260709 12:28:51.483589 12531 tablet_bootstrap.cc:654] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.489579 12681 raft_consensus.cc:493] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.490236 12681 raft_consensus.cc:515] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.481415 12527 raft_consensus.cc:359] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.495512 12527 raft_consensus.cc:385] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.495812 12527 raft_consensus.cc:740] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.496517 12527 consensus_queue.cc:260] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.496984 12681 leader_election.cc:290] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:51.497665 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:51.498044 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:51.498309 12481 raft_consensus.cc:2468] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:51.498787 12407 raft_consensus.cc:2468] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:51.499019 12527 ts_tablet_manager.cc:1434] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:28:51.499823 12224 leader_election.cc:304] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:51.501245 12527 tablet_bootstrap.cc:492] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:51.501245 12681 raft_consensus.cc:2804] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.501971 12681 raft_consensus.cc:493] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.502378 12681 raft_consensus.cc:3060] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.507575 12528 tablet_bootstrap.cc:492] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:51.508042 12527 tablet_bootstrap.cc:654] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.511044 12528 ts_tablet_manager.cc:1403] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.097s	user 0.013s	sys 0.004s
I20260709 12:28:51.515131 12529 tablet_bootstrap.cc:492] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:51.515664 12529 ts_tablet_manager.cc:1403] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.088s	user 0.014s	sys 0.009s
I20260709 12:28:51.526991 12531 tablet_bootstrap.cc:492] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:51.531682 12531 ts_tablet_manager.cc:1403] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.081s	user 0.013s	sys 0.003s
I20260709 12:28:51.519919 12528 raft_consensus.cc:359] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.537626 12528 raft_consensus.cc:385] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.538116 12528 raft_consensus.cc:740] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.518482 12681 raft_consensus.cc:515] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.542459 12528 consensus_queue.cc:260] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.522311 12529 raft_consensus.cc:359] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.547530 12529 raft_consensus.cc:385] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.547701 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:51.544977 12681 leader_election.cc:290] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:51.547981 12529 raft_consensus.cc:740] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.548481 12481 raft_consensus.cc:3060] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.542606 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:51.549672 12407 raft_consensus.cc:3060] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.536823 12531 raft_consensus.cc:359] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.550343 12531 raft_consensus.cc:385] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.550693 12531 raft_consensus.cc:740] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.552062 12528 ts_tablet_manager.cc:1434] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.000s
I20260709 12:28:51.553143 12528 tablet_bootstrap.cc:492] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:51.552824 12529 consensus_queue.cc:260] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.559805 12527 tablet_bootstrap.cc:492] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:51.559913 12531 consensus_queue.cc:260] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.560607 12527 ts_tablet_manager.cc:1403] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.060s	user 0.022s	sys 0.014s
I20260709 12:28:51.561785 12529 ts_tablet_manager.cc:1434] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.046s	user 0.013s	sys 0.000s
I20260709 12:28:51.563664 12527 raft_consensus.cc:359] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.564831 12407 raft_consensus.cc:2468] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:51.566457 12531 ts_tablet_manager.cc:1434] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.004s
I20260709 12:28:51.569734 12529 tablet_bootstrap.cc:492] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:51.569933 12531 tablet_bootstrap.cc:492] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:51.569937 12227 leader_election.cc:304] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:51.577111 12527 raft_consensus.cc:385] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.577600 12527 raft_consensus.cc:740] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.577862 12528 tablet_bootstrap.cc:654] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.578685 12529 tablet_bootstrap.cc:654] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.578419 12527 consensus_queue.cc:260] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.568203 12481 raft_consensus.cc:2468] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:51.577208 12531 tablet_bootstrap.cc:654] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.584086 12527 ts_tablet_manager.cc:1434] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:28:51.585497 12527 tablet_bootstrap.cc:492] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:51.585688 12679 raft_consensus.cc:2804] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.586295 12679 raft_consensus.cc:697] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:51.587189 12679 consensus_queue.cc:237] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.592701 12527 tablet_bootstrap.cc:654] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.594352 12529 tablet_bootstrap.cc:492] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:51.594936 12529 ts_tablet_manager.cc:1403] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.025s	user 0.005s	sys 0.012s
I20260709 12:28:51.597506 12636 consensus_queue.cc:1048] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.599169 12529 raft_consensus.cc:359] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.600193 12529 raft_consensus.cc:385] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.600566 12529 raft_consensus.cc:740] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.600811 12749 raft_consensus.cc:493] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.601432 12749 raft_consensus.cc:515] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.606902 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6417cb26ca8d401f8e73d3512aeab50b" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:51.607299 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6417cb26ca8d401f8e73d3512aeab50b" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:51.609864 12302 leader_election.cc:343] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.610949 12749 leader_election.cc:290] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:51.601778 12529 consensus_queue.cc:260] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
W20260709 12:28:51.612552 12299 leader_election.cc:343] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.613237 12299 leader_election.cc:304] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:51.616693 12632 raft_consensus.cc:2749] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:51.617681 12632 raft_consensus.cc:493] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.618290 12632 raft_consensus.cc:515] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.630338 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:51.633488 12299 leader_election.cc:343] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.635119 12632 leader_election.cc:290] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:51.637727 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:28:51.641362 12302 leader_election.cc:343] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.641906 12302 leader_election.cc:304] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:51.643733 12750 raft_consensus.cc:2749] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:51.645069 12157 catalog_manager.cc:5697] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:51.653645 12636 raft_consensus.cc:493] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.654467 12636 raft_consensus.cc:515] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.655048 12528 tablet_bootstrap.cc:492] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:51.655364 12529 ts_tablet_manager.cc:1434] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.000s
I20260709 12:28:51.655603 12528 ts_tablet_manager.cc:1403] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.103s	user 0.005s	sys 0.012s
I20260709 12:28:51.656379 12529 tablet_bootstrap.cc:492] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:51.656965 12636 leader_election.cc:290] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:51.657920 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67e469fc10a43bf9660c48b5ae1335a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:51.657956 12739 raft_consensus.cc:493] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.659549 12739 raft_consensus.cc:515] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
W20260709 12:28:51.662772 12450 leader_election.cc:343] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.664487 12738 raft_consensus.cc:493] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.665119 12738 raft_consensus.cc:515] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.666007 12739 leader_election.cc:290] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:51.667662 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d163b2f24c845d8b11b35d222659673" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:51.667994 12738 leader_election.cc:290] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:51.668593 12409 raft_consensus.cc:2468] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:51.669909 12450 leader_election.cc:304] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:51.671608 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:51.672763 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:51.673484 12336 raft_consensus.cc:2468] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:51.674230 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67e469fc10a43bf9660c48b5ae1335a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:51.675118 12257 raft_consensus.cc:2468] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:51.671485 12681 consensus_queue.cc:1048] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:28:51.677788 12739 raft_consensus.cc:2804] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.678306 12739 raft_consensus.cc:493] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.678692 12739 raft_consensus.cc:3060] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.680915 12447 leader_election.cc:304] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:51.682917 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d163b2f24c845d8b11b35d222659673" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:51.688843 12679 consensus_queue.cc:1048] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:28:51.693735 12636 raft_consensus.cc:2804] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.694849 12636 raft_consensus.cc:493] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.695297 12752 consensus_queue.cc:1048] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.695055 12620 consensus_queue.cc:1048] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:28:51.702239 12528 raft_consensus.cc:359] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.706600 12528 raft_consensus.cc:385] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.706905 12528 raft_consensus.cc:740] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.708362 12528 consensus_queue.cc:260] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.712229 12528 ts_tablet_manager.cc:1434] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.004s
I20260709 12:28:51.695770 12636 raft_consensus.cc:3060] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.733512 12529 tablet_bootstrap.cc:654] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.756740 12531 tablet_bootstrap.cc:492] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:51.757650 12531 ts_tablet_manager.cc:1403] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.188s	user 0.010s	sys 0.004s
W20260709 12:28:51.750813 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 70161.1us
I20260709 12:28:51.769766 12723 raft_consensus.cc:493] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.764743 12632 raft_consensus.cc:493] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.775208 12528 tablet_bootstrap.cc:492] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:51.789165 12620 raft_consensus.cc:493] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.779475 12632 raft_consensus.cc:515] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:51.789994 12620 raft_consensus.cc:515] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.792447 12632 leader_election.cc:290] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:51.774358 12723 raft_consensus.cc:515] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.793493 12531 raft_consensus.cc:359] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.793933 12259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e72262c8e2b24a8fa97e1cacbb60b49a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:51.794379 12531 raft_consensus.cc:385] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.794629 12259 raft_consensus.cc:2468] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:51.795863 12723 leader_election.cc:290] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:51.794696 12531 raft_consensus.cc:740] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.801257 12531 consensus_queue.cc:260] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.807430 12299 leader_election.cc:304] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:28:51.808596 12632 raft_consensus.cc:2804] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.809206 12632 raft_consensus.cc:493] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.809748 12632 raft_consensus.cc:3060] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.813618 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a81175bbecf4ba6b6be50b490564d9b" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:51.814427 12334 raft_consensus.cc:2468] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:51.818060 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e72262c8e2b24a8fa97e1cacbb60b49a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:51.818821 12483 raft_consensus.cc:2468] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:51.820612 12447 leader_election.cc:304] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:51.821830 12799 raft_consensus.cc:2804] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:28:51.821877 12447 leader_election.cc:343] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.822422 12799 raft_consensus.cc:493] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.822814 12799 raft_consensus.cc:3060] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:51.823467 12447 leader_election.cc:343] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.829952 12373 leader_election.cc:304] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:51.830644 12529 tablet_bootstrap.cc:492] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:51.830960 12723 raft_consensus.cc:2804] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.831239 12529 ts_tablet_manager.cc:1403] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.175s	user 0.015s	sys 0.006s
I20260709 12:28:51.831494 12723 raft_consensus.cc:493] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.832099 12723 raft_consensus.cc:3060] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.835433 12620 leader_election.cc:290] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:51.836848 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13100e49825f4fcfa78791b73b8d7555" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:51.838433 12409 raft_consensus.cc:2393] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1dcded10caa0493da818e67e13176979 in current term 1: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:28:51.842062 12800 consensus_queue.cc:1048] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.837219 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13100e49825f4fcfa78791b73b8d7555" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:51.850625 12224 leader_election.cc:304] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:51.851630 12752 raft_consensus.cc:3060] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.854048 12529 raft_consensus.cc:359] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.857137 12529 raft_consensus.cc:385] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.858248 12529 raft_consensus.cc:740] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.841393 12739 raft_consensus.cc:515] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.864202 12739 leader_election.cc:290] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:51.860703 12529 consensus_queue.cc:260] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.868566 12750 raft_consensus.cc:493] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.869056 12750 raft_consensus.cc:515] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:51.871374 12750 leader_election.cc:290] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:51.873265 12529 ts_tablet_manager.cc:1434] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.041s	user 0.009s	sys 0.000s
I20260709 12:28:51.875396 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:51.876060 12529 tablet_bootstrap.cc:492] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:51.874785 12528 tablet_bootstrap.cc:654] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.872550 12259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:51.877971 12527 tablet_bootstrap.cc:492] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:51.878522 12527 ts_tablet_manager.cc:1403] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.293s	user 0.072s	sys 0.076s
I20260709 12:28:51.865371 12739 raft_consensus.cc:493] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.884187 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a81175bbecf4ba6b6be50b490564d9b" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:51.886488 12739 raft_consensus.cc:515] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:51.893754 12739 leader_election.cc:290] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:51.893178 12527 raft_consensus.cc:359] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.900022 12527 raft_consensus.cc:385] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.900616 12527 raft_consensus.cc:740] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
W20260709 12:28:51.884558 12302 leader_election.cc:343] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.905879 12527 consensus_queue.cc:260] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.914008 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eace44731c5747ca98999cf217b561a3" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:51.922372 12531 ts_tablet_manager.cc:1434] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.164s	user 0.011s	sys 0.035s
I20260709 12:28:51.923878 12527 ts_tablet_manager.cc:1434] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.005s
I20260709 12:28:51.924772 12527 tablet_bootstrap.cc:492] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
W20260709 12:28:51.916455 12450 leader_election.cc:343] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.901857 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d163b2f24c845d8b11b35d222659673" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:51.927341 12531 tablet_bootstrap.cc:492] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:51.896485 12636 raft_consensus.cc:515] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:51.931608 12409 raft_consensus.cc:3060] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:51.917380 12299 leader_election.cc:343] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.982355 12527 tablet_bootstrap.cc:654] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.982563 12636 leader_election.cc:290] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:51.983517 12531 tablet_bootstrap.cc:654] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.984347 12299 leader_election.cc:304] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:51.986157 12755 consensus_queue.cc:1048] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:28:51.985428 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 85031.7us
I20260709 12:28:51.991155 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67e469fc10a43bf9660c48b5ae1335a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:51.991914 12408 raft_consensus.cc:3060] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.992466 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d163b2f24c845d8b11b35d222659673" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:51.993130 12334 raft_consensus.cc:3060] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.998452 12749 raft_consensus.cc:2749] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:51.998893 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67e469fc10a43bf9660c48b5ae1335a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:51.998486 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eace44731c5747ca98999cf217b561a3" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:51.999725 12257 raft_consensus.cc:3060] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.002694 12409 raft_consensus.cc:2468] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:52.003541 12632 raft_consensus.cc:515] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:52.019122 12632 leader_election.cc:290] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:52.003765 12662 consensus_queue.cc:1048] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.939213 12799 raft_consensus.cc:515] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:52.008205 12334 raft_consensus.cc:2468] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:52.008387 12723 raft_consensus.cc:515] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:52.011502 12752 raft_consensus.cc:2749] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:52.021508 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e72262c8e2b24a8fa97e1cacbb60b49a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:52.029654 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e72262c8e2b24a8fa97e1cacbb60b49a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:52.052390 12260 raft_consensus.cc:3060] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.045853 12482 raft_consensus.cc:3060] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:52.030714 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 73876.7us
W20260709 12:28:52.055928 12447 leader_election.cc:343] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.065213 12450 leader_election.cc:304] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:52.097994 12739 raft_consensus.cc:2804] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:28:52.099469 12373 leader_election.cc:343] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.099704 12799 leader_election.cc:290] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:52.098692 12739 raft_consensus.cc:697] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:52.102854 12408 raft_consensus.cc:2468] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:52.103098 12257 raft_consensus.cc:2468] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:52.103920 12739 consensus_queue.cc:237] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:52.083890 12447 leader_election.cc:304] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
W20260709 12:28:52.111615 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 60751.1us
I20260709 12:28:52.114256 12636 raft_consensus.cc:2749] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:52.122877 12450 leader_election.cc:304] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:52.124876 12806 raft_consensus.cc:2804] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:52.125653 12806 raft_consensus.cc:697] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:52.126724 12723 leader_election.cc:290] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:52.126921 12806 consensus_queue.cc:237] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:52.129871 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:52.137285 12257 raft_consensus.cc:3060] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.144682 12529 tablet_bootstrap.cc:654] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.132386 12260 raft_consensus.cc:2468] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:52.203605 12531 tablet_bootstrap.cc:492] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:52.204372 12531 ts_tablet_manager.cc:1403] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.281s	user 0.023s	sys 0.018s
I20260709 12:28:52.206162 12299 leader_election.cc:304] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
W20260709 12:28:52.214766 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 113663us
I20260709 12:28:52.229555 12816 raft_consensus.cc:2804] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:52.230304 12816 raft_consensus.cc:697] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:52.231333 12816 consensus_queue.cc:237] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:52.233621 12257 raft_consensus.cc:2468] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:52.248405 12482 raft_consensus.cc:2468] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:52.251092 12529 tablet_bootstrap.cc:492] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:52.251950 12529 ts_tablet_manager.cc:1403] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.376s	user 0.015s	sys 0.003s
I20260709 12:28:52.253777 12620 consensus_queue.cc:1048] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:52.255697 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a81175bbecf4ba6b6be50b490564d9b" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:52.279585 12527 tablet_bootstrap.cc:492] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:52.280628 12527 ts_tablet_manager.cc:1403] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.356s	user 0.034s	sys 0.015s
I20260709 12:28:52.257644 12531 raft_consensus.cc:359] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:52.257090 12529 raft_consensus.cc:359] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:52.281243 12531 raft_consensus.cc:385] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.281446 12529 raft_consensus.cc:385] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.281746 12531 raft_consensus.cc:740] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.282219 12529 raft_consensus.cc:740] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.293731 12528 tablet_bootstrap.cc:492] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:52.294320 12528 ts_tablet_manager.cc:1403] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.519s	user 0.028s	sys 0.015s
I20260709 12:28:52.296737 12531 consensus_queue.cc:260] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:52.257920 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:52.300073 12529 consensus_queue.cc:260] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:52.301975 12334 raft_consensus.cc:3060] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.314034 12531 ts_tablet_manager.cc:1434] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.109s	user 0.003s	sys 0.010s
I20260709 12:28:52.290701 12447 leader_election.cc:304] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
W20260709 12:28:52.327582 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 43561.3us
I20260709 12:28:52.332912 12528 raft_consensus.cc:359] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.333662 12528 raft_consensus.cc:385] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.333950 12528 raft_consensus.cc:740] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.335048 12636 raft_consensus.cc:2804] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:52.328015 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a81175bbecf4ba6b6be50b490564d9b" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:52.336210 12333 raft_consensus.cc:3060] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.342693 12636 raft_consensus.cc:697] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:52.343690 12636 consensus_queue.cc:237] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:52.334559 12528 consensus_queue.cc:260] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.347926 12529 ts_tablet_manager.cc:1434] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.096s	user 0.009s	sys 0.003s
I20260709 12:28:52.348892 12527 raft_consensus.cc:359] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.349625 12527 raft_consensus.cc:385] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.350024 12527 raft_consensus.cc:740] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
W20260709 12:28:52.353163 12373 leader_election.cc:343] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.353127 12527 consensus_queue.cc:260] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.356463 12159 catalog_manager.cc:5697] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:52.359614 12749 consensus_queue.cc:1048] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.359956 12159 catalog_manager.cc:5697] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:52.364126 12158 catalog_manager.cc:5697] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:52.375348 12531 tablet_bootstrap.cc:492] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:52.376849 12528 ts_tablet_manager.cc:1434] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.001s
I20260709 12:28:52.377796 12527 ts_tablet_manager.cc:1434] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.097s	user 0.008s	sys 0.000s
I20260709 12:28:52.378451 12528 tablet_bootstrap.cc:492] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:52.379125 12527 tablet_bootstrap.cc:492] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:52.399998 12333 raft_consensus.cc:2468] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:52.400766 12529 tablet_bootstrap.cc:492] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:52.396660 12334 raft_consensus.cc:2468] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:52.423373 12528 tablet_bootstrap.cc:654] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.428331 12529 tablet_bootstrap.cc:654] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.441862 12531 tablet_bootstrap.cc:654] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.449252 12527 tablet_bootstrap.cc:654] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.456467 12373 leader_election.cc:304] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:28:52.458371 12812 raft_consensus.cc:2804] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:52.458887 12812 raft_consensus.cc:697] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:28:52.459751 12812 consensus_queue.cc:237] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:52.474555 12528 tablet_bootstrap.cc:492] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:52.475129 12528 ts_tablet_manager.cc:1403] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.097s	user 0.015s	sys 0.000s
I20260709 12:28:52.477957 12528 raft_consensus.cc:359] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.478739 12528 raft_consensus.cc:385] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.479043 12528 raft_consensus.cc:740] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.479909 12527 tablet_bootstrap.cc:492] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:52.479773 12528 consensus_queue.cc:260] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.480494 12527 ts_tablet_manager.cc:1403] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.102s	user 0.012s	sys 0.000s
I20260709 12:28:52.482224 12528 ts_tablet_manager.cc:1434] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:28:52.483493 12528 tablet_bootstrap.cc:492] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:52.484153 12527 raft_consensus.cc:359] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:52.485037 12527 raft_consensus.cc:385] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.497243 12529 tablet_bootstrap.cc:492] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:52.498193 12529 ts_tablet_manager.cc:1403] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.098s	user 0.011s	sys 0.004s
I20260709 12:28:52.485339 12527 raft_consensus.cc:740] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.508639 12527 consensus_queue.cc:260] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:52.510533 12527 ts_tablet_manager.cc:1434] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 12:28:52.511337 12527 tablet_bootstrap.cc:492] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:52.516664 12750 raft_consensus.cc:493] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.518340 12528 tablet_bootstrap.cc:654] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.517917 12750 raft_consensus.cc:515] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:52.524140 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eace44731c5747ca98999cf217b561a3" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:28:52.525619 12302 leader_election.cc:343] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.525213 12529 raft_consensus.cc:359] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.527205 12529 raft_consensus.cc:385] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.527666 12529 raft_consensus.cc:740] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.528393 12750 leader_election.cc:290] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:52.530604 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eace44731c5747ca98999cf217b561a3" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:52.531424 12481 raft_consensus.cc:2468] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:52.534303 12531 tablet_bootstrap.cc:492] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:52.534821 12531 ts_tablet_manager.cc:1403] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.160s	user 0.007s	sys 0.024s
I20260709 12:28:52.535166 12527 tablet_bootstrap.cc:654] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.537744 12531 raft_consensus.cc:359] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:52.538518 12531 raft_consensus.cc:385] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.538307 12529 consensus_queue.cc:260] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.538844 12531 raft_consensus.cc:740] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.554847 12812 raft_consensus.cc:493] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.555501 12812 raft_consensus.cc:515] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.542553 12299 leader_election.cc:304] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:52.566982 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:52.568764 12373 leader_election.cc:343] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.573693 12812 leader_election.cc:290] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:52.576198 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:52.577334 12334 raft_consensus.cc:2468] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:52.579278 12373 leader_election.cc:304] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:52.555150 12531 consensus_queue.cc:260] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:52.581804 12812 raft_consensus.cc:2804] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:52.585834 12812 raft_consensus.cc:493] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.586617 12812 raft_consensus.cc:3060] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.586169 12157 catalog_manager.cc:5697] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:52.590266 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 47828.7us
I20260709 12:28:52.590411 12828 raft_consensus.cc:2804] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:52.591095 12828 raft_consensus.cc:493] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.591532 12529 ts_tablet_manager.cc:1434] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.093s	user 0.008s	sys 0.001s
I20260709 12:28:52.591547 12828 raft_consensus.cc:3060] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.593027 12529 tablet_bootstrap.cc:492] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:52.591862 12620 raft_consensus.cc:493] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.594815 12620 raft_consensus.cc:515] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.600418 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:52.601486 12620 leader_election.cc:290] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
W20260709 12:28:52.604672 12227 leader_election.cc:343] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.605350 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:52.606189 12334 raft_consensus.cc:2468] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:52.583590 12531 ts_tablet_manager.cc:1434] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 12:28:52.583390 12806 consensus_queue.cc:1048] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:52.607889 12224 leader_election.cc:304] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:52.609584 12620 raft_consensus.cc:2804] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:52.610023 12620 raft_consensus.cc:493] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.610440 12620 raft_consensus.cc:3060] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:52.613994 12206 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:52.618641 12620 raft_consensus.cc:515] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.601351 12828 raft_consensus.cc:515] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:52.634955 12159 catalog_manager.cc:5697] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:52.637089 12828 leader_election.cc:290] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:52.638044 12528 tablet_bootstrap.cc:492] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:52.638612 12528 ts_tablet_manager.cc:1403] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.156s	user 0.018s	sys 0.034s
I20260709 12:28:52.624188 12527 tablet_bootstrap.cc:492] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:52.639279 12527 ts_tablet_manager.cc:1403] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.128s	user 0.034s	sys 0.020s
I20260709 12:28:52.639456 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eace44731c5747ca98999cf217b561a3" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:52.640142 12481 raft_consensus.cc:3060] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.643289 12527 raft_consensus.cc:359] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:52.644104 12527 raft_consensus.cc:385] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.644445 12527 raft_consensus.cc:740] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.626331 12529 tablet_bootstrap.cc:654] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.629954 12531 tablet_bootstrap.cc:492] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:52.634428 12812 raft_consensus.cc:515] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.645179 12527 consensus_queue.cc:260] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:52.649542 12763 raft_consensus.cc:493] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.650558 12763 raft_consensus.cc:515] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:52.656701 12481 raft_consensus.cc:2468] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:52.658069 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a03f6aed8fcb4ea2b45b6432634c00ae" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:52.661383 12528 raft_consensus.cc:359] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:52.662263 12528 raft_consensus.cc:385] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.662571 12528 raft_consensus.cc:740] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.663877 12527 ts_tablet_manager.cc:1434] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:28:52.665027 12527 tablet_bootstrap.cc:492] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:52.671638 12299 leader_election.cc:304] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:52.658046 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a03f6aed8fcb4ea2b45b6432634c00ae" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:52.673741 12632 raft_consensus.cc:2804] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:52.674280 12632 raft_consensus.cc:697] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:52.675209 12632 consensus_queue.cc:237] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:52.676963 12812 leader_election.cc:290] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:52.663951 12528 consensus_queue.cc:260] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
W20260709 12:28:52.677371 12373 leader_election.cc:343] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:52.678994 12373 leader_election.cc:343] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.679387 12373 leader_election.cc:304] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:28:52.682399 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:52.683077 12333 raft_consensus.cc:3060] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.686777 12261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:52.688525 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eace44731c5747ca98999cf217b561a3" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:52.689167 12409 raft_consensus.cc:3060] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.661556 12763 leader_election.cc:290] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:52.690436 12812 raft_consensus.cc:2749] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:52.693037 12620 leader_election.cc:290] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:52.693651 12636 consensus_queue.cc:1048] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
W20260709 12:28:52.701887 12373 leader_election.cc:343] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.696868 12806 consensus_queue.cc:1048] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 12:28:52.694293 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:52.723001 12527 tablet_bootstrap.cc:654] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.715394 12528 ts_tablet_manager.cc:1434] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.076s	user 0.004s	sys 0.004s
W20260709 12:28:52.727007 12227 leader_election.cc:343] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.728499 12531 tablet_bootstrap.cc:654] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.729516 12528 tablet_bootstrap.cc:492] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:52.694325 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:52.731482 12334 raft_consensus.cc:3060] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.718369 12409 raft_consensus.cc:2468] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:52.701256 12333 raft_consensus.cc:2468] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:52.754021 12373 leader_election.cc:304] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:52.756887 12812 raft_consensus.cc:2804] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:28:52.757771 12450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 43643.1us
I20260709 12:28:52.779758 12812 raft_consensus.cc:697] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:28:52.780432 12844 raft_consensus.cc:493] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.780848 12812 consensus_queue.cc:237] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.782070 12844 raft_consensus.cc:515] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:52.795413 12844 leader_election.cc:290] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:52.803638 12529 tablet_bootstrap.cc:492] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:52.809494 12529 ts_tablet_manager.cc:1403] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.217s	user 0.019s	sys 0.010s
I20260709 12:28:52.809739 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a81175bbecf4ba6b6be50b490564d9b" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:52.810570 12333 raft_consensus.cc:2393] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a800ae68e91f43a794e326badca3320a in current term 1: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:28:52.812775 12529 raft_consensus.cc:359] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:52.813237 12632 consensus_queue.cc:1048] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.815373 12844 consensus_queue.cc:1048] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.807456 12334 raft_consensus.cc:2468] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:52.807454 12404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a81175bbecf4ba6b6be50b490564d9b" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:52.813592 12529 raft_consensus.cc:385] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.822750 12529 raft_consensus.cc:740] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.823593 12529 consensus_queue.cc:260] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:52.824673 12224 leader_election.cc:304] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:52.824015 12806 raft_consensus.cc:493] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.825843 12806 raft_consensus.cc:515] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:52.825888 12679 raft_consensus.cc:2804] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:52.829398 12679 raft_consensus.cc:697] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:52.829648 12528 tablet_bootstrap.cc:654] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.830202 12529 ts_tablet_manager.cc:1434] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:28:52.830406 12679 consensus_queue.cc:237] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.833940 12529 tablet_bootstrap.cc:492] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:52.825088 12450 leader_election.cc:304] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:52.874760 12851 consensus_queue.cc:1048] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:52.865394 12827 raft_consensus.cc:493] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:52.901507 12450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 76495.8us
I20260709 12:28:52.877734 12157 catalog_manager.cc:5697] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:52.889186 12851 raft_consensus.cc:3060] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:52.919106 12847 log.cc:927] Time spent T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:28:52.885435 12827 raft_consensus.cc:515] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.889281 12632 consensus_queue.cc:1048] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 12:28:52.927661 12812 consensus_queue.cc:1048] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:52.930097 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a0424f5d64be0a7d4d8b5d38907c6" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:52.930943 12806 leader_election.cc:290] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
W20260709 12:28:52.936920 12299 leader_election.cc:343] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.915069 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2871c5312642a28d3d3d65a5941a3a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:52.945322 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2871c5312642a28d3d3d65a5941a3a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:52.946120 12257 raft_consensus.cc:2468] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:52.950909 12447 leader_election.cc:304] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
W20260709 12:28:52.951581 12450 leader_election.cc:343] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.952279 12846 raft_consensus.cc:2804] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:52.952713 12846 raft_consensus.cc:493] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.953068 12846 raft_consensus.cc:3060] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.953773 12527 tablet_bootstrap.cc:492] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:52.954289 12527 ts_tablet_manager.cc:1403] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.289s	user 0.016s	sys 0.030s
I20260709 12:28:52.962705 12159 catalog_manager.cc:5697] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:52.968189 12531 tablet_bootstrap.cc:492] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:52.968775 12531 ts_tablet_manager.cc:1403] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.361s	user 0.021s	sys 0.009s
I20260709 12:28:52.958353 12749 raft_consensus.cc:493] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.969924 12749 raft_consensus.cc:515] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:52.972843 12749 leader_election.cc:290] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:52.980199 12827 leader_election.cc:290] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:52.982870 12751 raft_consensus.cc:493] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.983840 12751 raft_consensus.cc:515] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:52.991026 12527 raft_consensus.cc:359] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.000720 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:53.011984 12527 raft_consensus.cc:385] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.998176 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a0424f5d64be0a7d4d8b5d38907c6" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:53.008755 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:53.012475 12527 raft_consensus.cc:740] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.014925 12529 tablet_bootstrap.cc:654] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.015923 12739 consensus_queue.cc:1048] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:53.018515 12527 consensus_queue.cc:260] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.031984 12528 tablet_bootstrap.cc:492] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:53.032581 12528 ts_tablet_manager.cc:1403] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.303s	user 0.016s	sys 0.008s
I20260709 12:28:53.032687 12751 leader_election.cc:290] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:53.035442 12528 raft_consensus.cc:359] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.036157 12528 raft_consensus.cc:385] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.036485 12528 raft_consensus.cc:740] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.036582 12846 raft_consensus.cc:515] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.032074 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6417cb26ca8d401f8e73d3512aeab50b" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:53.038734 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6417cb26ca8d401f8e73d3512aeab50b" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:53.037384 12528 consensus_queue.cc:260] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
W20260709 12:28:53.031992 12299 leader_election.cc:343] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:53.040735 12302 leader_election.cc:343] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.047583 12302 leader_election.cc:304] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
W20260709 12:28:53.048913 12299 leader_election.cc:343] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.049472 12299 leader_election.cc:304] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:28:53.050596 12528 ts_tablet_manager.cc:1434] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:28:53.051783 12528 tablet_bootstrap.cc:492] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:53.056661 12527 ts_tablet_manager.cc:1434] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.102s	user 0.009s	sys 0.000s
I20260709 12:28:53.058125 12527 tablet_bootstrap.cc:492] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:53.046200 12336 raft_consensus.cc:2468] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:53.066721 12531 raft_consensus.cc:359] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.069193 12531 raft_consensus.cc:385] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.069546 12531 raft_consensus.cc:740] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.055974 12749 raft_consensus.cc:2749] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:53.077682 12531 consensus_queue.cc:260] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.079764 12859 raft_consensus.cc:2749] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:28:53.074769 12227 leader_election.cc:343] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.081458 12531 ts_tablet_manager.cc:1434] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.112s	user 0.007s	sys 0.000s
I20260709 12:28:53.083388 12531 tablet_bootstrap.cc:492] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:53.087509 12846 leader_election.cc:290] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:53.090482 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2871c5312642a28d3d3d65a5941a3a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:53.091233 12257 raft_consensus.cc:3060] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.094200 12527 tablet_bootstrap.cc:654] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.103806 12158 catalog_manager.cc:5697] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:53.115513 12224 leader_election.cc:304] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:53.120332 12752 raft_consensus.cc:2804] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:53.120898 12752 raft_consensus.cc:493] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.123477 12257 raft_consensus.cc:2468] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:53.126663 12527 tablet_bootstrap.cc:492] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:53.127393 12527 ts_tablet_manager.cc:1403] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.069s	user 0.011s	sys 0.000s
I20260709 12:28:53.121376 12752 raft_consensus.cc:3060] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.141106 12528 tablet_bootstrap.cc:654] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.142931 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e2871c5312642a28d3d3d65a5941a3a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:53.151549 12851 raft_consensus.cc:2749] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:53.164376 12529 tablet_bootstrap.cc:492] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:53.165343 12529 ts_tablet_manager.cc:1403] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.332s	user 0.022s	sys 0.002s
I20260709 12:28:53.167213 12531 tablet_bootstrap.cc:654] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:53.161386 12450 leader_election.cc:343] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.183068 12811 consensus_queue.cc:1048] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:53.184576 12527 raft_consensus.cc:359] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.185374 12527 raft_consensus.cc:385] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.186269 12527 raft_consensus.cc:740] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.187906 12527 consensus_queue.cc:260] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.190420 12529 raft_consensus.cc:359] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.196029 12529 raft_consensus.cc:385] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.196336 12529 raft_consensus.cc:740] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.197134 12529 consensus_queue.cc:260] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
W20260709 12:28:53.201426 12224 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 86044.6us
I20260709 12:28:53.200991 12447 leader_election.cc:304] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:53.208185 12806 raft_consensus.cc:2804] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:53.208734 12806 raft_consensus.cc:697] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:53.209559 12806 consensus_queue.cc:237] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.194876 12750 consensus_queue.cc:1048] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:28:53.186414 12752 raft_consensus.cc:515] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.214157 12527 ts_tablet_manager.cc:1434] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.086s	user 0.010s	sys 0.001s
I20260709 12:28:53.222188 12531 tablet_bootstrap.cc:492] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:53.222860 12531 ts_tablet_manager.cc:1403] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.140s	user 0.012s	sys 0.009s
I20260709 12:28:53.224717 12752 leader_election.cc:290] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:53.225937 12531 raft_consensus.cc:359] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.228732 12531 raft_consensus.cc:385] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.229087 12531 raft_consensus.cc:740] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.230119 12531 consensus_queue.cc:260] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.233522 12531 ts_tablet_manager.cc:1434] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:28:53.244525 12531 tablet_bootstrap.cc:492] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:53.245224 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6417cb26ca8d401f8e73d3512aeab50b" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:53.245946 12333 raft_consensus.cc:3060] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.251190 12531 tablet_bootstrap.cc:654] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.258370 12529 ts_tablet_manager.cc:1434] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.093s	user 0.008s	sys 0.000s
I20260709 12:28:53.261854 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6417cb26ca8d401f8e73d3512aeab50b" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:53.262631 12410 raft_consensus.cc:3060] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.268699 12868 consensus_queue.cc:1048] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:53.284835 12333 raft_consensus.cc:2468] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:53.287083 12859 consensus_queue.cc:1048] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:53.288592 12410 raft_consensus.cc:2468] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:53.290158 12227 leader_election.cc:304] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:53.291329 12870 raft_consensus.cc:2804] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:53.291934 12870 raft_consensus.cc:697] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:53.292817 12870 consensus_queue.cc:237] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.307597 12528 tablet_bootstrap.cc:492] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:53.311616 12528 ts_tablet_manager.cc:1403] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.260s	user 0.020s	sys 0.005s
I20260709 12:28:53.315086 12528 raft_consensus.cc:359] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.318845 12816 raft_consensus.cc:493] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.321558 12529 tablet_bootstrap.cc:492] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:53.324295 12816 raft_consensus.cc:515] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.332643 12816 leader_election.cc:290] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:53.333392 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:53.334200 12482 raft_consensus.cc:2468] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:53.322319 12527 tablet_bootstrap.cc:492] T 2e2871c5312642a28d3d3d65a5941a3a P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:53.322840 12817 consensus_queue.cc:1048] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:53.335414 12299 leader_election.cc:304] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:53.340716 12749 raft_consensus.cc:2804] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:53.341454 12749 raft_consensus.cc:493] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.342149 12749 raft_consensus.cc:3060] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.341341 12528 raft_consensus.cc:385] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.343763 12528 raft_consensus.cc:740] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.346352 12528 consensus_queue.cc:260] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.347764 12259 raft_consensus.cc:3060] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.352523 12527 tablet_bootstrap.cc:654] T 2e2871c5312642a28d3d3d65a5941a3a P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.360368 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:53.361927 12528 ts_tablet_manager.cc:1434] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
W20260709 12:28:53.362135 12302 leader_election.cc:343] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.363111 12528 tablet_bootstrap.cc:492] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:53.362962 12812 consensus_queue.cc:1048] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:53.365109 12812 raft_consensus.cc:493] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.365621 12812 raft_consensus.cc:515] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.368079 12812 leader_election.cc:290] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:53.370914 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:53.372895 12531 tablet_bootstrap.cc:492] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:53.373410 12531 ts_tablet_manager.cc:1403] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.129s	user 0.009s	sys 0.016s
I20260709 12:28:53.376358 12531 raft_consensus.cc:359] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.377029 12531 raft_consensus.cc:385] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.377315 12531 raft_consensus.cc:740] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.380885 12409 raft_consensus.cc:3060] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.384361 12531 consensus_queue.cc:260] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.385874 12529 tablet_bootstrap.cc:654] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.388612 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:53.389629 12157 catalog_manager.cc:5697] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:53.393926 12373 leader_election.cc:304] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:28:53.396368 12531 ts_tablet_manager.cc:1434] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:28:53.397375 12531 tablet_bootstrap.cc:492] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:53.394407 12528 tablet_bootstrap.cc:654] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.408048 12817 raft_consensus.cc:2749] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:53.432556 12527 tablet_bootstrap.cc:492] T 2e2871c5312642a28d3d3d65a5941a3a P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:53.433351 12527 ts_tablet_manager.cc:1403] T 2e2871c5312642a28d3d3d65a5941a3a P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.111s	user 0.005s	sys 0.013s
I20260709 12:28:53.435261 12528 tablet_bootstrap.cc:492] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:53.438998 12529 tablet_bootstrap.cc:492] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:53.439517 12529 ts_tablet_manager.cc:1403] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.118s	user 0.016s	sys 0.003s
I20260709 12:28:53.441511 12528 ts_tablet_manager.cc:1403] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.079s	user 0.013s	sys 0.000s
I20260709 12:28:53.447017 12531 tablet_bootstrap.cc:654] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.449177 12155 catalog_manager.cc:5697] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:53.460454 12527 raft_consensus.cc:359] T 2e2871c5312642a28d3d3d65a5941a3a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.461485 12806 raft_consensus.cc:493] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.462052 12806 raft_consensus.cc:515] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.462888 12870 consensus_queue.cc:1048] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:53.464087 12806 leader_election.cc:290] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:53.466174 12527 raft_consensus.cc:385] T 2e2871c5312642a28d3d3d65a5941a3a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.466502 12527 raft_consensus.cc:740] T 2e2871c5312642a28d3d3d65a5941a3a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.466934 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:53.467164 12527 consensus_queue.cc:260] T 2e2871c5312642a28d3d3d65a5941a3a P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.467602 12336 raft_consensus.cc:2468] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:53.468869 12447 leader_election.cc:304] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:53.469911 12806 raft_consensus.cc:2804] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:53.470281 12806 raft_consensus.cc:493] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.470645 12806 raft_consensus.cc:3060] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.471258 12259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:53.478360 12259 raft_consensus.cc:2468] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:53.482654 12529 raft_consensus.cc:359] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.483546 12529 raft_consensus.cc:385] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.483846 12529 raft_consensus.cc:740] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.484548 12529 consensus_queue.cc:260] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.485707 12749 raft_consensus.cc:515] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.486840 12527 ts_tablet_manager.cc:1434] T 2e2871c5312642a28d3d3d65a5941a3a P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 12:28:53.489862 12531 tablet_bootstrap.cc:492] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:53.487392 12846 raft_consensus.cc:493] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.490403 12846 raft_consensus.cc:515] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.490895 12806 raft_consensus.cc:515] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.492702 12846 leader_election.cc:290] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:53.493007 12806 leader_election.cc:290] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:53.494036 12527 tablet_bootstrap.cc:492] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:53.495534 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263de4693a84afe93a508aff7f1dc29" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:53.496285 12333 raft_consensus.cc:2468] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:53.497962 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:53.498569 12336 raft_consensus.cc:3060] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.499229 12749 leader_election.cc:290] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:53.502754 12447 leader_election.cc:304] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:53.503895 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:53.490398 12531 ts_tablet_manager.cc:1403] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.093s	user 0.010s	sys 0.001s
W20260709 12:28:53.506305 12302 leader_election.cc:343] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.508385 12531 raft_consensus.cc:359] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.509150 12531 raft_consensus.cc:385] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.509425 12531 raft_consensus.cc:740] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.510455 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:53.511071 12481 raft_consensus.cc:3060] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.511880 12336 raft_consensus.cc:2468] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:53.512815 12258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263de4693a84afe93a508aff7f1dc29" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:53.511978 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
W20260709 12:28:53.516199 12447 leader_election.cc:343] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.516399 12257 raft_consensus.cc:3060] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.517247 12447 leader_election.cc:304] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:53.518394 12846 raft_consensus.cc:2804] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:53.518870 12846 raft_consensus.cc:493] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.519202 12846 raft_consensus.cc:3060] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.520560 12527 tablet_bootstrap.cc:654] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.520613 12851 raft_consensus.cc:2804] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:53.521327 12851 raft_consensus.cc:697] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:53.522179 12851 consensus_queue.cc:237] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.512688 12531 consensus_queue.cc:260] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.515046 12529 ts_tablet_manager.cc:1434] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.075s	user 0.007s	sys 0.000s
I20260709 12:28:53.521852 12528 raft_consensus.cc:359] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.527074 12529 tablet_bootstrap.cc:492] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:53.527462 12528 raft_consensus.cc:385] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.527808 12528 raft_consensus.cc:740] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.525380 12531 ts_tablet_manager.cc:1434] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 12:28:53.531765 12531 tablet_bootstrap.cc:492] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:53.534596 12481 raft_consensus.cc:2468] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:53.529496 12528 consensus_queue.cc:260] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.535676 12299 leader_election.cc:304] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:53.537921 12528 ts_tablet_manager.cc:1434] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 12:28:53.540629 12749 raft_consensus.cc:2804] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:53.543401 12749 raft_consensus.cc:697] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:53.544600 12529 tablet_bootstrap.cc:654] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.545022 12749 consensus_queue.cc:237] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.556866 12529 tablet_bootstrap.cc:492] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:53.557442 12529 ts_tablet_manager.cc:1403] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.004s
I20260709 12:28:53.559407 12528 tablet_bootstrap.cc:492] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:53.560155 12529 raft_consensus.cc:359] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.560865 12529 raft_consensus.cc:385] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.561116 12529 raft_consensus.cc:740] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.563287 12527 tablet_bootstrap.cc:492] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:53.563978 12527 ts_tablet_manager.cc:1403] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.070s	user 0.006s	sys 0.007s
I20260709 12:28:53.566733 12527 raft_consensus.cc:359] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.567432 12527 raft_consensus.cc:385] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.567720 12527 raft_consensus.cc:740] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.570556 12527 consensus_queue.cc:260] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.572872 12527 ts_tablet_manager.cc:1434] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:28:53.573863 12527 tablet_bootstrap.cc:492] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:53.542423 12257 raft_consensus.cc:2468] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:53.579310 12528 tablet_bootstrap.cc:654] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.579120 12529 consensus_queue.cc:260] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.581096 12529 ts_tablet_manager.cc:1434] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.001s
I20260709 12:28:53.567166 12531 tablet_bootstrap.cc:654] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.582099 12529 tablet_bootstrap.cc:492] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:53.586969 12157 catalog_manager.cc:5697] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:53.589847 12527 tablet_bootstrap.cc:654] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.596393 12529 tablet_bootstrap.cc:654] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.586112 12846 raft_consensus.cc:515] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.604715 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263de4693a84afe93a508aff7f1dc29" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
W20260709 12:28:53.606272 12447 leader_election.cc:343] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.612282 12531 tablet_bootstrap.cc:492] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:53.612496 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263de4693a84afe93a508aff7f1dc29" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:53.612857 12531 ts_tablet_manager.cc:1403] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.081s	user 0.013s	sys 0.002s
I20260709 12:28:53.613106 12336 raft_consensus.cc:3060] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.620944 12409 raft_consensus.cc:3060] T 2e2871c5312642a28d3d3d65a5941a3a P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.622440 12336 raft_consensus.cc:2468] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:53.623736 12447 leader_election.cc:304] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:53.623459 12159 catalog_manager.cc:5697] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:53.624662 12851 raft_consensus.cc:2804] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:53.625155 12851 raft_consensus.cc:697] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:53.626003 12851 consensus_queue.cc:237] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.631323 12851 consensus_queue.cc:1048] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:53.633175 12531 raft_consensus.cc:359] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.631119 12846 leader_election.cc:290] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:53.634266 12531 raft_consensus.cc:385] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.637620 12531 raft_consensus.cc:740] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.638024 12527 tablet_bootstrap.cc:492] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:53.638587 12527 ts_tablet_manager.cc:1403] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.065s	user 0.008s	sys 0.006s
I20260709 12:28:53.638638 12529 tablet_bootstrap.cc:492] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:53.639241 12529 ts_tablet_manager.cc:1403] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.057s	user 0.015s	sys 0.018s
I20260709 12:28:53.638965 12531 consensus_queue.cc:260] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.641230 12531 ts_tablet_manager.cc:1434] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.001s
I20260709 12:28:53.641781 12527 raft_consensus.cc:359] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.642585 12527 raft_consensus.cc:385] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.642933 12527 raft_consensus.cc:740] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.642669 12529 raft_consensus.cc:359] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.647347 12529 raft_consensus.cc:385] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.647564 12527 consensus_queue.cc:260] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.642640 12531 tablet_bootstrap.cc:492] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:53.649536 12527 ts_tablet_manager.cc:1434] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 12:28:53.647650 12529 raft_consensus.cc:740] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.651007 12527 tablet_bootstrap.cc:492] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:53.650837 12529 consensus_queue.cc:260] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.653398 12529 ts_tablet_manager.cc:1434] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:28:53.654526 12529 tablet_bootstrap.cc:492] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:53.656553 12531 tablet_bootstrap.cc:654] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.658816 12527 tablet_bootstrap.cc:654] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.661242 12529 tablet_bootstrap.cc:654] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.680771 12531 tablet_bootstrap.cc:492] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:53.681305 12531 ts_tablet_manager.cc:1403] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.039s	user 0.004s	sys 0.013s
I20260709 12:28:53.684222 12531 raft_consensus.cc:359] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.684978 12531 raft_consensus.cc:385] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.685253 12531 raft_consensus.cc:740] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.685941 12531 consensus_queue.cc:260] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.687445 12636 consensus_queue.cc:1048] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:53.691236 12531 ts_tablet_manager.cc:1434] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:28:53.692358 12531 tablet_bootstrap.cc:492] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:53.699388 12846 raft_consensus.cc:493] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.699952 12846 raft_consensus.cc:515] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.702178 12846 leader_election.cc:290] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:53.703958 12259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164179fdd9ee45ff80ea3530e210088a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:53.704063 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164179fdd9ee45ff80ea3530e210088a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
W20260709 12:28:53.705904 12447 leader_election.cc:343] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:53.706629 12447 leader_election.cc:343] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.707074 12447 leader_election.cc:304] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:28:53.707803 12851 raft_consensus.cc:493] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.708271 12851 raft_consensus.cc:515] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.709055 12528 tablet_bootstrap.cc:492] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:53.709606 12528 ts_tablet_manager.cc:1403] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.166s	user 0.017s	sys 0.003s
I20260709 12:28:53.710750 12846 raft_consensus.cc:2749] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:53.711213 12851 leader_election.cc:290] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:53.712716 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:53.713953 12531 tablet_bootstrap.cc:654] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:53.714841 12450 leader_election.cc:343] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.714869 12157 catalog_manager.cc:5697] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:53.717068 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:53.718051 12336 raft_consensus.cc:2468] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:53.718405 12528 raft_consensus.cc:359] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.719321 12528 raft_consensus.cc:385] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.719614 12528 raft_consensus.cc:740] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.720860 12528 consensus_queue.cc:260] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.725615 12528 ts_tablet_manager.cc:1434] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:28:53.726686 12528 tablet_bootstrap.cc:492] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:53.735034 12528 tablet_bootstrap.cc:654] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.735425 12447 leader_election.cc:304] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:53.736265 12851 raft_consensus.cc:493] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.736752 12851 raft_consensus.cc:515] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.741027 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a03f6aed8fcb4ea2b45b6432634c00ae" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:53.741813 12410 raft_consensus.cc:2468] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:53.751261 12851 leader_election.cc:290] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:53.752293 12846 raft_consensus.cc:2804] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:53.756466 12846 raft_consensus.cc:493] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.759346 12450 leader_election.cc:304] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:53.759240 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a03f6aed8fcb4ea2b45b6432634c00ae" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:53.760673 12846 raft_consensus.cc:3060] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.761387 12257 raft_consensus.cc:2468] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:53.760012 12528 tablet_bootstrap.cc:492] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:53.783931 12528 ts_tablet_manager.cc:1403] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.003s
I20260709 12:28:53.761669 12851 raft_consensus.cc:2804] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:53.787742 12763 raft_consensus.cc:493] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.787850 12528 raft_consensus.cc:359] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.788477 12763 raft_consensus.cc:515] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.789084 12528 raft_consensus.cc:385] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.789417 12528 raft_consensus.cc:740] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.790092 12528 consensus_queue.cc:260] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.790709 12763 leader_election.cc:290] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:53.788053 12851 raft_consensus.cc:493] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.792346 12851 raft_consensus.cc:3060] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.792649 12331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:53.791610 12528 ts_tablet_manager.cc:1434] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:28:53.802924 12528 tablet_bootstrap.cc:492] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:53.810235 12529 tablet_bootstrap.cc:492] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:53.810817 12529 ts_tablet_manager.cc:1403] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.157s	user 0.039s	sys 0.065s
I20260709 12:28:53.813727 12529 raft_consensus.cc:359] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.814445 12529 raft_consensus.cc:385] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.815001 12529 raft_consensus.cc:740] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.815498 12527 tablet_bootstrap.cc:492] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:53.816016 12527 ts_tablet_manager.cc:1403] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.165s	user 0.007s	sys 0.004s
I20260709 12:28:53.815666 12529 consensus_queue.cc:260] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:53.818838 12527 raft_consensus.cc:359] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.823554 12529 ts_tablet_manager.cc:1434] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:28:53.820231 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:53.822022 12851 raft_consensus.cc:515] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.824680 12481 raft_consensus.cc:2393] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2104c8a9b144a419ccb7c07e3f02596 in current term 1: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:28:53.824934 12529 tablet_bootstrap.cc:492] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:53.818923 12528 tablet_bootstrap.cc:654] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.826246 12373 leader_election.cc:304] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:28:53.828436 12812 raft_consensus.cc:3060] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.832026 12851 leader_election.cc:290] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:53.833313 12529 tablet_bootstrap.cc:654] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.836520 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a03f6aed8fcb4ea2b45b6432634c00ae" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:53.837467 12407 raft_consensus.cc:3060] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.841750 12679 consensus_queue.cc:1048] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:53.847077 12870 consensus_queue.cc:1048] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:53.855892 12812 raft_consensus.cc:2749] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:53.864938 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a03f6aed8fcb4ea2b45b6432634c00ae" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:53.866641 12257 raft_consensus.cc:3060] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.866742 12763 raft_consensus.cc:493] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.868348 12763 raft_consensus.cc:515] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.869835 12529 tablet_bootstrap.cc:492] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:53.870338 12529 ts_tablet_manager.cc:1403] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.004s
I20260709 12:28:53.870517 12763 leader_election.cc:290] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:53.872449 12259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a0424f5d64be0a7d4d8b5d38907c6" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:53.873131 12259 raft_consensus.cc:2468] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:53.873720 12529 raft_consensus.cc:359] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.874459 12529 raft_consensus.cc:385] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.874320 12373 leader_election.cc:304] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:53.874887 12529 raft_consensus.cc:740] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.875696 12763 raft_consensus.cc:2804] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:53.876184 12763 raft_consensus.cc:493] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.875869 12529 consensus_queue.cc:260] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.876636 12763 raft_consensus.cc:3060] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.878074 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a0424f5d64be0a7d4d8b5d38907c6" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:53.878854 12336 raft_consensus.cc:2468] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:53.879940 12529 ts_tablet_manager.cc:1434] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:28:53.867259 12527 raft_consensus.cc:385] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.880769 12527 raft_consensus.cc:740] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.883169 12531 tablet_bootstrap.cc:492] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:53.880978 12529 tablet_bootstrap.cc:492] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:53.883742 12531 ts_tablet_manager.cc:1403] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.192s	user 0.023s	sys 0.039s
I20260709 12:28:53.892076 12531 raft_consensus.cc:359] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.906111 12531 raft_consensus.cc:385] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.906471 12531 raft_consensus.cc:740] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.907184 12531 consensus_queue.cc:260] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.910811 12529 tablet_bootstrap.cc:654] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.917284 12531 ts_tablet_manager.cc:1434] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:28:53.918464 12531 tablet_bootstrap.cc:492] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:53.894148 12846 raft_consensus.cc:515] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:53.916644 12257 raft_consensus.cc:2468] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:53.890203 12527 consensus_queue.cc:260] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.930783 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:53.941919 12527 ts_tablet_manager.cc:1434] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.126s	user 0.007s	sys 0.005s
I20260709 12:28:53.944135 12447 leader_election.cc:304] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:53.937727 12846 leader_election.cc:290] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:53.945932 12636 raft_consensus.cc:2804] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:53.949443 12636 raft_consensus.cc:697] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:53.950317 12636 consensus_queue.cc:237] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:53.945211 12331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:53.953456 12331 raft_consensus.cc:3060] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:53.945386 12450 leader_election.cc:343] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.937119 12407 raft_consensus.cc:2468] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
W20260709 12:28:53.959303 12206 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:53.968498 12816 consensus_queue.cc:1048] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:53.972970 12528 tablet_bootstrap.cc:492] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:53.979238 12528 ts_tablet_manager.cc:1403] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.177s	user 0.027s	sys 0.048s
I20260709 12:28:53.951013 12763 raft_consensus.cc:515] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:53.982739 12531 tablet_bootstrap.cc:654] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.997062 12527 tablet_bootstrap.cc:492] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:54.040073 12763 leader_election.cc:290] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:54.041997 12529 tablet_bootstrap.cc:492] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:54.042572 12529 ts_tablet_manager.cc:1403] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.162s	user 0.033s	sys 0.018s
I20260709 12:28:54.044094 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a0424f5d64be0a7d4d8b5d38907c6" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:54.045086 12333 raft_consensus.cc:3060] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.050212 12749 raft_consensus.cc:493] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.053063 12749 raft_consensus.cc:515] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:54.052893 12636 consensus_queue.cc:1048] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:54.046700 12527 tablet_bootstrap.cc:654] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:54.052158 12158 catalog_manager.cc:5697] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:54.048179 12331 raft_consensus.cc:2468] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:54.058207 12259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:54.056106 12749 leader_election.cc:290] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:54.045495 12529 raft_consensus.cc:359] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:54.059566 12447 leader_election.cc:304] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:54.059981 12529 raft_consensus.cc:385] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.060349 12529 raft_consensus.cc:740] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:54.060861 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:54.061678 12846 consensus_queue.cc:1048] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:54.062266 12529 consensus_queue.cc:260] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:54.063092 12859 raft_consensus.cc:493] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.063235 12846 raft_consensus.cc:2804] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:54.063573 12859 raft_consensus.cc:515] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:54.064018 12846 raft_consensus.cc:697] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:54.066062 12739 consensus_queue.cc:1048] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:54.063680 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a0424f5d64be0a7d4d8b5d38907c6" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:54.067530 12257 raft_consensus.cc:3060] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.069253 12859 leader_election.cc:290] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:54.074734 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcca3076fc03410aa86e82c060c29335" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
W20260709 12:28:54.077430 12302 leader_election.cc:343] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.078490 12528 raft_consensus.cc:359] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:54.079288 12528 raft_consensus.cc:385] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.079571 12528 raft_consensus.cc:740] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:54.080253 12528 consensus_queue.cc:260] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
W20260709 12:28:54.082780 12299 leader_election.cc:343] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.086444 12299 leader_election.cc:304] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:54.095322 12529 ts_tablet_manager.cc:1434] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.052s	user 0.003s	sys 0.004s
I20260709 12:28:54.096942 12529 tablet_bootstrap.cc:492] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:54.108071 12528 ts_tablet_manager.cc:1434] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.128s	user 0.008s	sys 0.000s
W20260709 12:28:54.111724 12299 leader_election.cc:343] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.117617 12531 tablet_bootstrap.cc:492] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:54.118283 12531 ts_tablet_manager.cc:1403] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.200s	user 0.036s	sys 0.015s
I20260709 12:28:54.064990 12846 consensus_queue.cc:237] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.107437 12749 raft_consensus.cc:2749] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:54.136232 12529 tablet_bootstrap.cc:654] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:54.137033 12258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcca3076fc03410aa86e82c060c29335" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:54.159497 12259 raft_consensus.cc:3060] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.175577 12528 tablet_bootstrap.cc:492] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
W20260709 12:28:54.183415 12302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 64914.5us
I20260709 12:28:54.183017 12910 raft_consensus.cc:493] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.212066 12531 raft_consensus.cc:359] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:54.220932 12531 raft_consensus.cc:385] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.221292 12531 raft_consensus.cc:740] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:54.225723 12531 consensus_queue.cc:260] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
W20260709 12:28:54.232986 12299 leader_election.cc:343] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.233732 12299 leader_election.cc:304] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:28:54.236047 12527 tablet_bootstrap.cc:492] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:54.236580 12527 ts_tablet_manager.cc:1403] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.240s	user 0.033s	sys 0.015s
I20260709 12:28:54.241225 12531 ts_tablet_manager.cc:1434] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.123s	user 0.008s	sys 0.001s
I20260709 12:28:54.245471 12531 tablet_bootstrap.cc:492] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:54.236410 12859 raft_consensus.cc:493] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:54.318866 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 48487.4us
I20260709 12:28:54.221072 12910 raft_consensus.cc:515] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:54.257488 12333 raft_consensus.cc:2468] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
W20260709 12:28:54.339828 12224 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 99164.6us
I20260709 12:28:54.320546 12859 raft_consensus.cc:515] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:54.371054 12918 raft_consensus.cc:493] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.372269 12914 raft_consensus.cc:493] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.367947 12927 raft_consensus.cc:2749] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:54.374897 12527 raft_consensus.cc:359] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.397804 12527 raft_consensus.cc:385] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.398146 12527 raft_consensus.cc:740] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:54.399608 12527 consensus_queue.cc:260] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.381636 12928 raft_consensus.cc:493] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.415868 12527 ts_tablet_manager.cc:1434] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.179s	user 0.005s	sys 0.004s
I20260709 12:28:54.421025 12527 tablet_bootstrap.cc:492] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:54.419737 12910 leader_election.cc:290] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:54.397226 12918 raft_consensus.cc:515] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:54.396896 12914 raft_consensus.cc:515] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:54.477602 12529 tablet_bootstrap.cc:492] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:54.478188 12529 ts_tablet_manager.cc:1403] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.382s	user 0.027s	sys 0.046s
I20260709 12:28:54.421651 12928 raft_consensus.cc:515] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:54.505070 12928 leader_election.cc:290] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:54.519025 12531 tablet_bootstrap.cc:654] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:54.506186 12924 log.cc:927] Time spent T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
W20260709 12:28:54.490342 12450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 101725us
I20260709 12:28:54.531344 12941 raft_consensus.cc:493] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.531493 12918 leader_election.cc:290] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:54.531944 12941 raft_consensus.cc:515] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:54.534766 12941 leader_election.cc:290] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:54.535126 12528 tablet_bootstrap.cc:654] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:54.550729 12859 leader_election.cc:290] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:54.574079 12529 raft_consensus.cc:359] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.574810 12529 raft_consensus.cc:385] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.575122 12529 raft_consensus.cc:740] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:54.575788 12529 consensus_queue.cc:260] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.578251 12914 leader_election.cc:290] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:54.575135 12527 tablet_bootstrap.cc:654] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:54.582368 12529 ts_tablet_manager.cc:1434] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.104s	user 0.008s	sys 0.000s
I20260709 12:28:54.585961 12529 tablet_bootstrap.cc:492] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:54.594002 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad2a742adea4bf989def5e758a3a81c" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:54.603283 12257 consensus_meta.cc:287] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979: Time spent flushing consensus metadata: real 0.535s	user 0.014s	sys 0.000s
I20260709 12:28:54.603942 12257 raft_consensus.cc:2468] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:54.615204 12529 tablet_bootstrap.cc:654] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:54.617656 12679 raft_consensus.cc:493] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.618484 12679 raft_consensus.cc:515] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
W20260709 12:28:54.582834 12227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 106870us
I20260709 12:28:54.637138 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f39a7bfe4bff44b18d1007ced129ae09" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:54.638100 12410 raft_consensus.cc:2468] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:54.642282 12227 leader_election.cc:304] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:54.644017 12928 raft_consensus.cc:2804] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:54.644564 12928 raft_consensus.cc:493] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.645017 12928 raft_consensus.cc:3060] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.648029 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f39a7bfe4bff44b18d1007ced129ae09" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:54.642508 12373 leader_election.cc:304] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:54.652806 12531 tablet_bootstrap.cc:492] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:54.654160 12812 raft_consensus.cc:2804] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:54.654767 12812 raft_consensus.cc:697] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:28:54.653427 12531 ts_tablet_manager.cc:1403] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.411s	user 0.016s	sys 0.013s
I20260709 12:28:54.655650 12812 consensus_queue.cc:237] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:54.664819 12531 raft_consensus.cc:359] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.669481 12531 raft_consensus.cc:385] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.671597 12531 raft_consensus.cc:740] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
W20260709 12:28:54.674459 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 90030.7us
I20260709 12:28:54.679937 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164179fdd9ee45ff80ea3530e210088a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:54.680756 12481 raft_consensus.cc:2468] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:54.684939 12531 consensus_queue.cc:260] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.704048 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164179fdd9ee45ff80ea3530e210088a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:54.707633 12927 consensus_queue.cc:1048] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:54.699725 12299 leader_election.cc:304] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:54.721791 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:28:54.725062 12299 leader_election.cc:343] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.731534 12531 ts_tablet_manager.cc:1434] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.075s	user 0.010s	sys 0.000s
I20260709 12:28:54.735642 12531 tablet_bootstrap.cc:492] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
W20260709 12:28:54.751978 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 77647.7us
I20260709 12:28:54.724134 12934 raft_consensus.cc:2804] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:28:54.759497 12447 leader_election.cc:343] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.748823 12259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:54.773941 12302 leader_election.cc:343] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.782262 12527 tablet_bootstrap.cc:492] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:54.783062 12527 ts_tablet_manager.cc:1403] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.362s	user 0.011s	sys 0.009s
I20260709 12:28:54.761538 12934 raft_consensus.cc:493] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.775879 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f39a7bfe4bff44b18d1007ced129ae09" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:54.784046 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f39a7bfe4bff44b18d1007ced129ae09" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:54.795336 12484 raft_consensus.cc:2468] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:54.784400 12950 consensus_queue.cc:1048] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:28:54.792732 12528 tablet_bootstrap.cc:492] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:54.800613 12528 ts_tablet_manager.cc:1403] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.625s	user 0.015s	sys 0.012s
I20260709 12:28:54.795682 12928 raft_consensus.cc:515] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:54.813418 12529 tablet_bootstrap.cc:492] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:54.817619 12529 ts_tablet_manager.cc:1403] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.234s	user 0.032s	sys 0.030s
I20260709 12:28:54.797631 12934 raft_consensus.cc:3060] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.819875 12531 tablet_bootstrap.cc:654] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:54.809301 12679 leader_election.cc:290] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:54.834683 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263de4693a84afe93a508aff7f1dc29" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:54.836782 12928 leader_election.cc:290] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:54.840890 12528 raft_consensus.cc:359] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:54.841930 12528 raft_consensus.cc:385] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.842224 12528 raft_consensus.cc:740] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:54.842958 12528 consensus_queue.cc:260] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:54.843924 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f39a7bfe4bff44b18d1007ced129ae09" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:54.844619 12481 raft_consensus.cc:2468] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
W20260709 12:28:54.847836 12299 leader_election.cc:343] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.848316 12299 leader_election.cc:304] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:54.847994 12940 raft_consensus.cc:493] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.855590 12940 raft_consensus.cc:515] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.856235 12257 raft_consensus.cc:2393] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2104c8a9b144a419ccb7c07e3f02596 in current term 1: Already voted for candidate 1dcded10caa0493da818e67e13176979 in this term.
I20260709 12:28:54.856299 12636 raft_consensus.cc:493] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.858157 12940 leader_election.cc:290] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:54.858292 12636 raft_consensus.cc:515] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.860812 12636 leader_election.cc:290] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:54.861907 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263de4693a84afe93a508aff7f1dc29" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:54.865808 12404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad2a742adea4bf989def5e758a3a81c" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:54.866709 12529 raft_consensus.cc:359] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.875093 12529 raft_consensus.cc:385] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.878186 12529 raft_consensus.cc:740] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:54.879247 12529 consensus_queue.cc:260] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.881069 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f39a7bfe4bff44b18d1007ced129ae09" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:54.882036 12483 raft_consensus.cc:3060] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.900697 12806 consensus_queue.cc:1048] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:54.902261 12404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f39a7bfe4bff44b18d1007ced129ae09" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:54.903187 12404 raft_consensus.cc:3060] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.903525 12527 raft_consensus.cc:359] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.904032 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b0d4a9a29a4de9bc3a8ab529535d68" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:54.904469 12527 raft_consensus.cc:385] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.900909 12926 raft_consensus.cc:493] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.904870 12527 raft_consensus.cc:740] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:54.909018 12752 raft_consensus.cc:493] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.910006 12752 raft_consensus.cc:515] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:54.911098 12921 raft_consensus.cc:2749] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:54.905117 12926 raft_consensus.cc:515] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.913606 12752 leader_election.cc:290] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:54.914402 12926 leader_election.cc:290] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:54.914987 12332 raft_consensus.cc:2468] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:54.915541 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:54.915992 12331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:54.921965 12528 ts_tablet_manager.cc:1434] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.121s	user 0.009s	sys 0.000s
I20260709 12:28:54.930313 12477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b0d4a9a29a4de9bc3a8ab529535d68" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:54.931366 12477 raft_consensus.cc:2468] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:54.933779 12331 raft_consensus.cc:2468] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:54.934825 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b0d4a9a29a4de9bc3a8ab529535d68" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:54.937258 12948 consensus_queue.cc:1048] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:54.938067 12157 catalog_manager.cc:5697] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:54.939268 12450 leader_election.cc:343] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.939738 12450 leader_election.cc:304] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:54.940465 12948 raft_consensus.cc:2749] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:54.942013 12224 leader_election.cc:304] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:28:54.922569 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f39a7bfe4bff44b18d1007ced129ae09" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:54.918246 12527 consensus_queue.cc:260] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:54.944410 12528 tablet_bootstrap.cc:492] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:54.946295 12447 leader_election.cc:304] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:54.948290 12806 raft_consensus.cc:2804] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:54.945290 12299 leader_election.cc:304] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:54.933373 12483 raft_consensus.cc:2468] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:54.948773 12806 raft_consensus.cc:493] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.948868 12527 ts_tablet_manager.cc:1434] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.165s	user 0.008s	sys 0.001s
I20260709 12:28:54.950860 12529 ts_tablet_manager.cc:1434] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.133s	user 0.009s	sys 0.000s
I20260709 12:28:54.991165 12527 tablet_bootstrap.cc:492] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
W20260709 12:28:54.984112 12227 leader_election.cc:343] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.994050 12529 tablet_bootstrap.cc:492] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
W20260709 12:28:54.994910 12227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 44590.8us
I20260709 12:28:54.984182 12405 raft_consensus.cc:2468] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 0.
I20260709 12:28:54.997262 12929 raft_consensus.cc:2804] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:54.997781 12929 raft_consensus.cc:493] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.998159 12929 raft_consensus.cc:3060] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.999651 12944 raft_consensus.cc:2804] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.000448 12944 raft_consensus.cc:493] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.001410 12944 raft_consensus.cc:3060] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.009100 12527 tablet_bootstrap.cc:654] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:54.925115 12934 raft_consensus.cc:515] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:54.990653 12806 raft_consensus.cc:3060] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.997189 12531 tablet_bootstrap.cc:492] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:54.948874 12846 consensus_queue.cc:1048] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.060s
I20260709 12:28:55.015080 12531 ts_tablet_manager.cc:1403] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.282s	user 0.031s	sys 0.035s
I20260709 12:28:54.992976 12404 raft_consensus.cc:2468] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:55.021103 12224 leader_election.cc:304] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:28:55.023366 12928 raft_consensus.cc:2749] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:55.040167 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b0d4a9a29a4de9bc3a8ab529535d68" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:55.040913 12409 raft_consensus.cc:2468] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:55.039583 12929 raft_consensus.cc:515] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.038318 12934 leader_election.cc:290] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
W20260709 12:28:55.064904 12206 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:55.068318 12528 tablet_bootstrap.cc:654] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.059355 12926 raft_consensus.cc:493] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.072561 12926 raft_consensus.cc:515] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:55.074687 12926 leader_election.cc:290] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
W20260709 12:28:55.080466 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 233777us
I20260709 12:28:55.082098 12477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164179fdd9ee45ff80ea3530e210088a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:55.082861 12477 raft_consensus.cc:3060] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.099171 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:55.100889 12450 leader_election.cc:304] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:55.102952 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:55.105965 12480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:55.107450 12531 raft_consensus.cc:359] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.108295 12531 raft_consensus.cc:385] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.108582 12531 raft_consensus.cc:740] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.109349 12531 consensus_queue.cc:260] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.116252 12224 leader_election.cc:304] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:55.121500 12373 leader_election.cc:304] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
W20260709 12:28:55.118166 12299 leader_election.cc:343] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.124724 12959 raft_consensus.cc:2764] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:28:55.126649 12527 tablet_bootstrap.cc:492] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:55.126739 12679 raft_consensus.cc:2804] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.127615 12527 ts_tablet_manager.cc:1403] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.137s	user 0.016s	sys 0.002s
I20260709 12:28:55.128521 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:55.130461 12950 raft_consensus.cc:2749] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:55.127616 12679 raft_consensus.cc:697] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:55.143841 12679 consensus_queue.cc:237] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
W20260709 12:28:55.139180 12302 leader_election.cc:343] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.150153 12302 leader_election.cc:304] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
I20260709 12:28:55.151109 12749 raft_consensus.cc:2749] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:55.142799 12373 leader_election.cc:304] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:28:55.155946 12531 ts_tablet_manager.cc:1434] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.140s	user 0.006s	sys 0.001s
I20260709 12:28:55.157541 12531 tablet_bootstrap.cc:492] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:55.164929 12529 tablet_bootstrap.cc:654] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.135043 12527 raft_consensus.cc:359] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.163741 12940 raft_consensus.cc:2749] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:55.113193 12806 raft_consensus.cc:515] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:55.139600 12254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164179fdd9ee45ff80ea3530e210088a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:55.174589 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:55.173290 12929 leader_election.cc:290] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:55.177099 12957 consensus_queue.cc:1048] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:55.185590 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:55.186580 12528 tablet_bootstrap.cc:492] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:55.220646 12528 ts_tablet_manager.cc:1403] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.293s	user 0.017s	sys 0.002s
I20260709 12:28:55.203645 12527 raft_consensus.cc:385] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.221982 12527 raft_consensus.cc:740] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.224848 12527 consensus_queue.cc:260] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.216100 12404 raft_consensus.cc:3060] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.204802 12477 raft_consensus.cc:2468] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:55.221724 12334 raft_consensus.cc:3060] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:55.239969 12299 leader_election.cc:343] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.241108 12527 ts_tablet_manager.cc:1434] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.113s	user 0.008s	sys 0.001s
I20260709 12:28:55.246057 12299 leader_election.cc:304] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:55.250113 12749 raft_consensus.cc:2804] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.250691 12749 raft_consensus.cc:697] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:55.251626 12749 consensus_queue.cc:237] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.252446 12527 tablet_bootstrap.cc:492] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:55.205551 12944 raft_consensus.cc:515] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:55.264628 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b0d4a9a29a4de9bc3a8ab529535d68" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:55.269862 12806 leader_election.cc:290] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:55.273247 12528 raft_consensus.cc:359] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.274155 12528 raft_consensus.cc:385] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:28:55.261199 12227 leader_election.cc:343] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.274461 12528 raft_consensus.cc:740] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.275128 12528 consensus_queue.cc:260] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.276387 12477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b0d4a9a29a4de9bc3a8ab529535d68" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:55.276655 12332 raft_consensus.cc:2393] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a800ae68e91f43a794e326badca3320a in current term 1: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:28:55.273393 12944 leader_election.cc:290] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:55.278501 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b0d4a9a29a4de9bc3a8ab529535d68" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:55.287024 12410 raft_consensus.cc:3060] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.280360 12528 ts_tablet_manager.cc:1434] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.000s
I20260709 12:28:55.285485 12477 raft_consensus.cc:2393] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 in current term 1: Already voted for candidate a800ae68e91f43a794e326badca3320a in this term.
I20260709 12:28:55.318492 12528 tablet_bootstrap.cc:492] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:55.267683 12940 consensus_queue.cc:1048] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 12:28:55.309326 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b0d4a9a29a4de9bc3a8ab529535d68" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:55.313395 12529 tablet_bootstrap.cc:492] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:55.314911 12531 tablet_bootstrap.cc:654] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.328891 12529 ts_tablet_manager.cc:1403] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.337s	user 0.011s	sys 0.008s
I20260709 12:28:55.334975 12529 raft_consensus.cc:359] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.335865 12529 raft_consensus.cc:385] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.336264 12529 raft_consensus.cc:740] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.340201 12529 consensus_queue.cc:260] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.350473 12410 raft_consensus.cc:2468] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:28:55.351950 12302 leader_election.cc:304] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:28:55.357255 12859 raft_consensus.cc:2804] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.358035 12859 raft_consensus.cc:697] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:28:55.359826 12528 tablet_bootstrap.cc:654] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.359375 12859 consensus_queue.cc:237] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:55.364190 12679 raft_consensus.cc:493] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.364816 12679 raft_consensus.cc:515] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.366935 12914 consensus_queue.cc:1048] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:55.367436 12679 leader_election.cc:290] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:55.380647 12529 ts_tablet_manager.cc:1434] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.002s
I20260709 12:28:55.371821 12450 leader_election.cc:304] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:55.389019 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:55.390017 12405 raft_consensus.cc:2468] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:55.391992 12227 leader_election.cc:304] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:55.392939 12334 raft_consensus.cc:2468] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:55.393276 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:55.394326 12333 raft_consensus.cc:2468] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:55.394578 12928 raft_consensus.cc:2804] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.395259 12928 raft_consensus.cc:493] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.395802 12928 raft_consensus.cc:3060] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.401175 12948 raft_consensus.cc:2749] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:55.406071 12224 leader_election.cc:304] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:55.408697 12527 tablet_bootstrap.cc:654] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.419397 12529 tablet_bootstrap.cc:492] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4: Bootstrap starting.
I20260709 12:28:55.412734 12935 raft_consensus.cc:2804] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.428119 12935 raft_consensus.cc:697] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:55.441547 12935 consensus_queue.cc:237] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.451068 12531 tablet_bootstrap.cc:492] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:55.451633 12531 ts_tablet_manager.cc:1403] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.294s	user 0.020s	sys 0.012s
I20260709 12:28:55.453763 12528 tablet_bootstrap.cc:492] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:55.454303 12528 ts_tablet_manager.cc:1403] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.136s	user 0.017s	sys 0.012s
I20260709 12:28:55.446123 12914 raft_consensus.cc:493] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.454780 12914 raft_consensus.cc:515] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.459607 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcca3076fc03410aa86e82c060c29335" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:55.463577 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcca3076fc03410aa86e82c060c29335" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:55.464316 12333 raft_consensus.cc:2468] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:55.457057 12914 leader_election.cc:290] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:55.457057 12531 raft_consensus.cc:359] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:55.468032 12531 raft_consensus.cc:385] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.468292 12447 leader_election.cc:304] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:55.471407 12914 raft_consensus.cc:2804] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.471854 12914 raft_consensus.cc:493] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.472231 12914 raft_consensus.cc:3060] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.468371 12531 raft_consensus.cc:740] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.476200 12531 consensus_queue.cc:260] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:55.480111 12531 ts_tablet_manager.cc:1434] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 12:28:55.481199 12531 tablet_bootstrap.cc:492] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:55.457222 12528 raft_consensus.cc:359] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.494177 12528 raft_consensus.cc:385] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.494513 12528 raft_consensus.cc:740] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.495380 12528 consensus_queue.cc:260] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.497872 12257 raft_consensus.cc:2468] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:55.502206 12528 ts_tablet_manager.cc:1434] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.001s
I20260709 12:28:55.503233 12528 tablet_bootstrap.cc:492] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:55.510473 12528 tablet_bootstrap.cc:654] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.514552 12154 catalog_manager.cc:5697] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:55.536020 12914 raft_consensus.cc:515] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.545382 12914 leader_election.cc:290] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:55.568090 12255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcca3076fc03410aa86e82c060c29335" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:55.569195 12255 raft_consensus.cc:3060] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.572167 12331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcca3076fc03410aa86e82c060c29335" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:55.572836 12331 raft_consensus.cc:3060] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.577476 12527 tablet_bootstrap.cc:492] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:55.583069 12527 ts_tablet_manager.cc:1403] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.331s	user 0.014s	sys 0.004s
I20260709 12:28:55.587913 12527 raft_consensus.cc:359] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:55.588634 12527 raft_consensus.cc:385] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.588920 12527 raft_consensus.cc:740] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.589603 12527 consensus_queue.cc:260] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:55.589771 12935 consensus_queue.cc:1048] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:55.592319 12928 raft_consensus.cc:515] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.598752 12529 tablet_bootstrap.cc:654] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.608631 12928 leader_election.cc:290] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:55.609786 12404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:55.610492 12404 raft_consensus.cc:3060] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.610357 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:55.611177 12334 raft_consensus.cc:3060] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.615773 12531 tablet_bootstrap.cc:654] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.618276 12255 raft_consensus.cc:2468] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:55.627880 12447 leader_election.cc:304] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:55.628786 12914 raft_consensus.cc:2804] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.622936 12529 tablet_bootstrap.cc:492] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4: No bootstrap required, opened a new log
I20260709 12:28:55.629377 12914 raft_consensus.cc:697] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:55.629980 12529 ts_tablet_manager.cc:1403] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4: Time spent bootstrapping tablet: real 0.212s	user 0.014s	sys 0.000s
I20260709 12:28:55.630457 12914 consensus_queue.cc:237] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.624243 12334 raft_consensus.cc:2468] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:55.637702 12528 tablet_bootstrap.cc:492] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:55.638386 12528 ts_tablet_manager.cc:1403] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.135s	user 0.033s	sys 0.016s
I20260709 12:28:55.639853 12224 leader_election.cc:304] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:28:55.640878 12935 raft_consensus.cc:2804] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.641376 12935 raft_consensus.cc:697] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:55.642283 12935 consensus_queue.cc:237] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.632989 12529 raft_consensus.cc:359] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.644541 12529 raft_consensus.cc:385] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.644884 12529 raft_consensus.cc:740] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.645624 12529 consensus_queue.cc:260] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.646688 12527 ts_tablet_manager.cc:1434] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.000s
I20260709 12:28:55.645938 12528 raft_consensus.cc:359] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.654644 12528 raft_consensus.cc:385] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.655316 12528 raft_consensus.cc:740] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.648028 12957 raft_consensus.cc:493] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.656147 12957 raft_consensus.cc:515] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:55.656569 12528 consensus_queue.cc:260] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.658666 12957 leader_election.cc:290] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:55.660703 12255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad2a742adea4bf989def5e758a3a81c" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:55.662158 12528 ts_tablet_manager.cc:1434] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:28:55.663182 12531 tablet_bootstrap.cc:492] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:55.663345 12528 tablet_bootstrap.cc:492] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:55.663714 12531 ts_tablet_manager.cc:1403] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.183s	user 0.008s	sys 0.009s
W20260709 12:28:55.664180 12373 leader_election.cc:343] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.666944 12531 raft_consensus.cc:359] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:55.667716 12531 raft_consensus.cc:385] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.668006 12531 raft_consensus.cc:740] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.668730 12531 consensus_queue.cc:260] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:55.671272 12531 ts_tablet_manager.cc:1434] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:28:55.672314 12531 tablet_bootstrap.cc:492] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:55.647082 12928 consensus_queue.cc:1048] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:55.647732 12527 tablet_bootstrap.cc:492] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:55.655645 12529 ts_tablet_manager.cc:1434] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:28:55.681223 12528 tablet_bootstrap.cc:654] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.682039 12531 tablet_bootstrap.cc:654] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.682477 12527 tablet_bootstrap.cc:654] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.701609 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad2a742adea4bf989def5e758a3a81c" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:55.702401 12483 raft_consensus.cc:2468] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:55.707948 12528 tablet_bootstrap.cc:492] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:55.708647 12528 ts_tablet_manager.cc:1403] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.001s
I20260709 12:28:55.714751 12373 leader_election.cc:304] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:55.715651 12957 raft_consensus.cc:2804] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.716261 12957 raft_consensus.cc:493] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.722349 12957 raft_consensus.cc:3060] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.733032 12528 raft_consensus.cc:359] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:55.734067 12528 raft_consensus.cc:385] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.734470 12257 raft_consensus.cc:3060] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.734447 12528 raft_consensus.cc:740] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.755700 12528 consensus_queue.cc:260] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:55.760322 12944 consensus_queue.cc:1048] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:55.764034 12404 raft_consensus.cc:2468] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:55.764719 12531 tablet_bootstrap.cc:492] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:55.765393 12531 ts_tablet_manager.cc:1403] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.093s	user 0.018s	sys 0.017s
I20260709 12:28:55.750355 12957 raft_consensus.cc:515] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:55.774191 12806 raft_consensus.cc:493] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.774963 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad2a742adea4bf989def5e758a3a81c" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:55.775664 12483 raft_consensus.cc:3060] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.776548 12527 tablet_bootstrap.cc:492] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:55.777108 12527 ts_tablet_manager.cc:1403] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.130s	user 0.010s	sys 0.003s
I20260709 12:28:55.777279 12528 ts_tablet_manager.cc:1434] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.068s	user 0.011s	sys 0.000s
I20260709 12:28:55.778812 12528 tablet_bootstrap.cc:492] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:55.778731 12255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad2a742adea4bf989def5e758a3a81c" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:55.779448 12255 raft_consensus.cc:3060] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.780073 12527 raft_consensus.cc:359] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.780862 12527 raft_consensus.cc:385] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.781184 12527 raft_consensus.cc:740] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.781988 12527 consensus_queue.cc:260] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.784603 12157 catalog_manager.cc:5697] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:55.774837 12806 raft_consensus.cc:515] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:55.796108 12806 leader_election.cc:290] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:55.796778 12528 tablet_bootstrap.cc:654] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.797986 12254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:55.799979 12447 leader_election.cc:343] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.792989 12157 catalog_manager.cc:5697] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:55.807292 12531 raft_consensus.cc:359] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.808195 12531 raft_consensus.cc:385] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.785760 12957 leader_election.cc:290] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:55.808504 12531 raft_consensus.cc:740] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.816365 12483 raft_consensus.cc:2468] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:55.819254 12373 leader_election.cc:304] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:55.824715 12531 consensus_queue.cc:260] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.828099 12957 raft_consensus.cc:2804] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.829530 12531 ts_tablet_manager.cc:1434] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.000s
I20260709 12:28:55.830684 12531 tablet_bootstrap.cc:492] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a: Bootstrap starting.
I20260709 12:28:55.832834 12527 ts_tablet_manager.cc:1434] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.055s	user 0.004s	sys 0.004s
I20260709 12:28:55.822427 12331 raft_consensus.cc:2468] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:55.828961 12816 raft_consensus.cc:493] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.838451 12957 raft_consensus.cc:697] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:28:55.858405 12816 raft_consensus.cc:515] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:55.859089 12957 consensus_queue.cc:237] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:55.868440 12254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d9482906bb4ca4a34c22c21f178cab" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:55.875775 12299 leader_election.cc:343] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.879281 12404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d9482906bb4ca4a34c22c21f178cab" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:55.840821 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:28:55.888038 12450 leader_election.cc:343] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.888577 12450 leader_election.cc:304] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:55.889688 12806 raft_consensus.cc:2749] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:55.877429 12816 leader_election.cc:290] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
W20260709 12:28:55.900345 12302 leader_election.cc:343] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.900918 12302 leader_election.cc:304] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:55.902037 12859 raft_consensus.cc:2749] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:55.902532 12527 tablet_bootstrap.cc:492] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:55.905952 12531 tablet_bootstrap.cc:654] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.924808 12528 tablet_bootstrap.cc:492] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:55.919010 12158 catalog_manager.cc:5697] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:55.925424 12528 ts_tablet_manager.cc:1403] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.147s	user 0.041s	sys 0.063s
I20260709 12:28:55.930388 12255 raft_consensus.cc:2468] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:55.930672 12528 raft_consensus.cc:359] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:55.934518 12528 raft_consensus.cc:385] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.934984 12528 raft_consensus.cc:740] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.936193 12528 consensus_queue.cc:260] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:55.951071 12528 ts_tablet_manager.cc:1434] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.002s
I20260709 12:28:55.952504 12528 tablet_bootstrap.cc:492] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:55.951231 12158 catalog_manager.cc:5697] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:55.957029 12806 raft_consensus.cc:493] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.952983 12816 consensus_queue.cc:1048] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:28:55.956879 12944 consensus_queue.cc:1048] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:28:55.975987 12407 raft_consensus.cc:3060] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.978502 12531 tablet_bootstrap.cc:492] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a: No bootstrap required, opened a new log
I20260709 12:28:55.979126 12531 ts_tablet_manager.cc:1403] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a: Time spent bootstrapping tablet: real 0.149s	user 0.019s	sys 0.015s
I20260709 12:28:55.966768 12806 raft_consensus.cc:515] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:55.984866 12528 tablet_bootstrap.cc:654] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.989696 12806 leader_election.cc:290] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:55.991155 12257 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:55.994108 12447 leader_election.cc:343] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.997135 12527 tablet_bootstrap.cc:654] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.998970 12159 catalog_manager.cc:5697] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.003278 12531 raft_consensus.cc:359] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:56.004062 12531 raft_consensus.cc:385] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.004347 12531 raft_consensus.cc:740] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.004626 12331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:56.005036 12531 consensus_queue.cc:260] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:56.005441 12331 raft_consensus.cc:2468] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
I20260709 12:28:56.007339 12531 ts_tablet_manager.cc:1434] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 12:28:56.011759 12447 leader_election.cc:304] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:56.017745 12806 raft_consensus.cc:2804] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:56.022737 12806 raft_consensus.cc:493] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:56.023177 12806 raft_consensus.cc:3060] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.056129 12935 raft_consensus.cc:493] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.056949 12935 raft_consensus.cc:515] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:56.043148 12157 catalog_manager.cc:5697] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.093307 12935 leader_election.cc:290] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:56.105969 12477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:56.106734 12477 raft_consensus.cc:2468] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:56.113310 12224 leader_election.cc:304] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:56.136416 12404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:56.140806 12739 raft_consensus.cc:493] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.141427 12739 raft_consensus.cc:515] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:56.143604 12739 leader_election.cc:290] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
W20260709 12:28:56.145052 12227 leader_election.cc:343] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:56.126673 12929 raft_consensus.cc:2804] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:56.153272 12929 raft_consensus.cc:493] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:56.203449 12227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 57424.2us
W20260709 12:28:56.203864 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 82604.7us
I20260709 12:28:56.224727 12948 raft_consensus.cc:493] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.225324 12404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:56.225411 12948 raft_consensus.cc:515] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
W20260709 12:28:56.229157 12450 leader_election.cc:343] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:56.208304 12929 raft_consensus.cc:3060] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.186719 12870 consensus_queue.cc:1048] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 12:28:56.227632 12948 leader_election.cc:290] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:56.243935 12527 tablet_bootstrap.cc:492] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:56.244568 12527 ts_tablet_manager.cc:1403] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.342s	user 0.032s	sys 0.022s
I20260709 12:28:56.242408 13007 raft_consensus.cc:493] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.246227 13007 raft_consensus.cc:515] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:56.248536 13007 leader_election.cc:290] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:28:56.264887 12227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 60388us
W20260709 12:28:56.271116 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 65909.4us
W20260709 12:28:56.272558 12206 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:56.250592 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
W20260709 12:28:56.280200 12302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 149349us
I20260709 12:28:56.245103 12914 consensus_queue.cc:1048] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 12:28:56.244071 12806 raft_consensus.cc:515] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:56.286695 12333 raft_consensus.cc:2468] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 0.
W20260709 12:28:56.357127 12983 log.cc:927] Time spent T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:28:56.368440 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:56.369407 12334 raft_consensus.cc:2468] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:56.346014 12870 consensus_queue.cc:1048] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.060s
I20260709 12:28:56.410924 12154 catalog_manager.cc:5697] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:56.417302 12806 leader_election.cc:290] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:56.423462 12254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:56.441532 12935 consensus_queue.cc:1048] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:56.428196 12258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cc6fec74b6423bb49415fbcdd93e55" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:56.424202 12404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cc6fec74b6423bb49415fbcdd93e55" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:56.469702 12527 raft_consensus.cc:359] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:56.470582 12527 raft_consensus.cc:385] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.470920 12527 raft_consensus.cc:740] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.504200 12373 leader_election.cc:304] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:56.504379 12527 consensus_queue.cc:260] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:56.429061 12929 raft_consensus.cc:515] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
W20260709 12:28:56.518895 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 155539us
I20260709 12:28:56.518910 12527 ts_tablet_manager.cc:1434] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.274s	user 0.016s	sys 0.002s
W20260709 12:28:56.521356 12224 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 224486us
I20260709 12:28:56.522270 12527 tablet_bootstrap.cc:492] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:56.530014 12447 leader_election.cc:304] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:56.574178 12528 tablet_bootstrap.cc:492] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:56.575071 12528 ts_tablet_manager.cc:1403] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.623s	user 0.015s	sys 0.007s
W20260709 12:28:56.607443 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 56896.4us
I20260709 12:28:56.555814 13019 raft_consensus.cc:2804] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:56.606748 12961 raft_consensus.cc:2804] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:56.627399 13019 raft_consensus.cc:493] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:56.652490 13019 raft_consensus.cc:3060] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.652062 12528 raft_consensus.cc:359] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:56.653388 12528 raft_consensus.cc:385] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.654258 12528 raft_consensus.cc:740] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.656527 12528 consensus_queue.cc:260] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:56.662056 12528 ts_tablet_manager.cc:1434] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.085s	user 0.005s	sys 0.002s
I20260709 12:28:56.657879 12258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:56.695226 12527 tablet_bootstrap.cc:654] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:56.696843 12929 leader_election.cc:290] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:56.647603 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:56.704653 12327 raft_consensus.cc:3060] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.654112 13024 raft_consensus.cc:493] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.721980 13024 raft_consensus.cc:515] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:56.725347 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:56.726176 12407 raft_consensus.cc:3060] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.738612 13024 leader_election.cc:290] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:56.777199 12407 raft_consensus.cc:2468] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:56.778921 12227 leader_election.cc:304] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:56.780025 13013 raft_consensus.cc:2804] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:56.780939 13013 raft_consensus.cc:697] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:56.782037 13013 consensus_queue.cc:237] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:56.786931 13019 raft_consensus.cc:515] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:56.787878 12527 tablet_bootstrap.cc:492] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:56.788472 12527 ts_tablet_manager.cc:1403] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.266s	user 0.019s	sys 0.006s
I20260709 12:28:56.792001 12527 raft_consensus.cc:359] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:56.815883 12527 raft_consensus.cc:385] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.794000 12327 raft_consensus.cc:2468] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
W20260709 12:28:56.811439 12447 leader_election.cc:343] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:56.816241 12527 raft_consensus.cc:740] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.818269 12948 consensus_queue.cc:1048] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.818732 12527 consensus_queue.cc:260] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
W20260709 12:28:56.817476 12447 leader_election.cc:343] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:56.821378 13019 leader_election.cc:290] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:56.822090 12527 ts_tablet_manager.cc:1434] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 12:28:56.823124 12527 tablet_bootstrap.cc:492] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:56.831262 12447 leader_election.cc:304] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:56.832047 13019 raft_consensus.cc:2804] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:56.832702 13019 raft_consensus.cc:697] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:56.833506 13019 consensus_queue.cc:237] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:56.843952 12527 tablet_bootstrap.cc:654] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:56.851092 12258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:56.855370 13007 consensus_queue.cc:1048] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.864616 12528 tablet_bootstrap.cc:492] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:56.873427 12527 tablet_bootstrap.cc:492] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:56.874049 12527 ts_tablet_manager.cc:1403] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.000s
I20260709 12:28:56.875715 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:56.876461 12483 raft_consensus.cc:2468] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:56.876735 12527 raft_consensus.cc:359] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:56.877441 12527 raft_consensus.cc:385] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.877735 12527 raft_consensus.cc:740] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.878542 12527 consensus_queue.cc:260] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:56.880214 12527 ts_tablet_manager.cc:1434] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:28:56.881124 12527 tablet_bootstrap.cc:492] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596: Bootstrap starting.
I20260709 12:28:56.887763 12527 tablet_bootstrap.cc:654] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:56.892817 12961 raft_consensus.cc:493] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:28:56.893558 12961 raft_consensus.cc:3060] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.899902 12527 tablet_bootstrap.cc:492] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596: No bootstrap required, opened a new log
I20260709 12:28:56.900456 12527 ts_tablet_manager.cc:1403] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.001s
I20260709 12:28:56.903741 12527 raft_consensus.cc:359] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:56.904402 12527 raft_consensus.cc:385] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.904692 12527 raft_consensus.cc:740] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.904558 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:56.905249 12484 raft_consensus.cc:3060] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.905521 12527 consensus_queue.cc:260] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:56.907565 12527 ts_tablet_manager.cc:1434] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:28:56.911790 12484 raft_consensus.cc:2468] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:28:56.916213 12373 leader_election.cc:304] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:56.916332 13007 consensus_queue.cc:1048] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:56.917402 13037 raft_consensus.cc:2764] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:28:56.918165 12528 tablet_bootstrap.cc:654] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:56.919638 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:56.919853 13009 consensus_queue.cc:1048] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:56.920380 12410 raft_consensus.cc:3060] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:56.927727 12450 leader_election.cc:343] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:56.930361 12450 leader_election.cc:304] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:56.931185 13010 raft_consensus.cc:2749] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:56.916297 12961 raft_consensus.cc:515] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:56.936389 12961 leader_election.cc:290] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:28:56.983587 12224 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 139856us
I20260709 12:28:56.925987 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:56.999703 12328 raft_consensus.cc:3060] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.004580 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:57.005518 12928 consensus_queue.cc:1048] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:28:57.027314 13024 raft_consensus.cc:493] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.028671 13024 raft_consensus.cc:515] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
W20260709 12:28:57.033993 12227 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 185281us
I20260709 12:28:57.080883 12328 raft_consensus.cc:2468] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
W20260709 12:28:57.102466 12224 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 116016us
I20260709 12:28:57.115972 12260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:57.127924 12373 leader_election.cc:304] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:28:57.129976 13037 raft_consensus.cc:2749] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.131347 13055 raft_consensus.cc:493] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.132233 13055 raft_consensus.cc:515] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:57.144778 12157 catalog_manager.cc:5697] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:57.150753 12447 leader_election.cc:304] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:28:57.155694 13055 leader_election.cc:290] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:57.163436 12914 raft_consensus.cc:2804] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:57.164635 12914 raft_consensus.cc:697] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:28:57.166607 12914 consensus_queue.cc:237] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:28:57.185925 13024 leader_election.cc:290] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:57.196955 12528 tablet_bootstrap.cc:492] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:57.198184 12528 ts_tablet_manager.cc:1403] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.334s	user 0.016s	sys 0.002s
I20260709 12:28:57.199833 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cc6fec74b6423bb49415fbcdd93e55" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:57.200734 12405 raft_consensus.cc:2468] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:57.217344 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cc6fec74b6423bb49415fbcdd93e55" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:57.232249 12479 raft_consensus.cc:2468] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 0.
I20260709 12:28:57.235093 12253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cc6fec74b6423bb49415fbcdd93e55" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:57.235176 12227 leader_election.cc:304] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:57.236328 12253 raft_consensus.cc:2468] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:57.236429 12410 raft_consensus.cc:2468] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:28:57.237978 13053 raft_consensus.cc:2804] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:57.238446 13053 raft_consensus.cc:493] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:57.239135 13053 raft_consensus.cc:3060] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.242463 12373 leader_election.cc:304] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:57.243670 13022 raft_consensus.cc:2804] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:57.244131 13022 raft_consensus.cc:493] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:57.244556 13022 raft_consensus.cc:3060] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.272336 12410 raft_consensus.cc:1240] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Rejecting Update request from peer 1dcded10caa0493da818e67e13176979 for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:57.274546 12929 consensus_queue.cc:1059] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:57.275758 12929 raft_consensus.cc:3055] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:57.276201 12929 raft_consensus.cc:740] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:28:57.277500 12929 consensus_queue.cc:260] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:57.279305 12929 raft_consensus.cc:3060] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:57.294220 12528 raft_consensus.cc:359] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:57.300976 12528 raft_consensus.cc:385] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.291018 13024 raft_consensus.cc:493] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.302600 13024 raft_consensus.cc:515] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:57.305815 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cc6fec74b6423bb49415fbcdd93e55" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:57.301360 12528 raft_consensus.cc:740] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.313582 12479 raft_consensus.cc:2468] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:57.315752 13024 leader_election.cc:290] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:57.322045 12528 consensus_queue.cc:260] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
W20260709 12:28:57.326241 12447 consensus_peers.cc:597] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:57.347119 12528 ts_tablet_manager.cc:1434] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.148s	user 0.009s	sys 0.000s
I20260709 12:28:57.348740 12528 tablet_bootstrap.cc:492] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:57.349869 12253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d9482906bb4ca4a34c22c21f178cab" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:28:57.356041 12373 leader_election.cc:343] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:57.352980 13022 raft_consensus.cc:515] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:57.361270 13018 consensus_queue.cc:1048] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:28:57.379863 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d9482906bb4ca4a34c22c21f178cab" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:57.380700 12328 raft_consensus.cc:2468] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 0.
I20260709 12:28:57.390398 13022 leader_election.cc:290] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:28:57.391561 12253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cc6fec74b6423bb49415fbcdd93e55" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:57.392822 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cc6fec74b6423bb49415fbcdd93e55" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:57.393424 12482 raft_consensus.cc:3060] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.394942 12373 leader_election.cc:304] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:57.395634 13022 raft_consensus.cc:2804] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:57.396073 13022 raft_consensus.cc:493] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:57.396405 13022 raft_consensus.cc:3060] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.389240 12929 consensus_queue.cc:1048] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:57.402508 13053 raft_consensus.cc:515] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:57.403805 12253 raft_consensus.cc:2393] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2104c8a9b144a419ccb7c07e3f02596 in current term 1: Already voted for candidate 1dcded10caa0493da818e67e13176979 in this term.
I20260709 12:28:57.406261 12528 tablet_bootstrap.cc:654] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:57.406399 12403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cc6fec74b6423bb49415fbcdd93e55" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:57.407493 12482 raft_consensus.cc:2468] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:57.416466 12373 leader_election.cc:304] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:28:57.417518 13024 raft_consensus.cc:2804] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:57.418111 13024 raft_consensus.cc:697] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:28:57.419193 13024 consensus_queue.cc:237] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:57.428684 13019 consensus_queue.cc:1048] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:57.433171 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cc6fec74b6423bb49415fbcdd93e55" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:28:57.434274 12479 raft_consensus.cc:2393] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1dcded10caa0493da818e67e13176979 in current term 1: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:28:57.439827 13014 raft_consensus.cc:493] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.439896 12224 leader_election.cc:304] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
I20260709 12:28:57.440402 13014 raft_consensus.cc:515] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:57.443220 13014 leader_election.cc:290] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:57.441077 12929 raft_consensus.cc:2749] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:57.445618 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:28:57.451376 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:57.459030 12224 leader_election.cc:304] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:28:57.460798 12929 raft_consensus.cc:2749] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:57.407593 13053 leader_election.cc:290] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 1 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:28:57.523247 13022 raft_consensus.cc:515] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:57.531211 12528 tablet_bootstrap.cc:492] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979: No bootstrap required, opened a new log
I20260709 12:28:57.531950 12528 ts_tablet_manager.cc:1403] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.184s	user 0.015s	sys 0.003s
I20260709 12:28:57.535188 12528 raft_consensus.cc:359] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:57.537976 12253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d9482906bb4ca4a34c22c21f178cab" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:57.545653 12528 raft_consensus.cc:385] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.546093 12528 raft_consensus.cc:740] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.546976 12528 consensus_queue.cc:260] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:57.548036 12253 raft_consensus.cc:3060] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.559332 12253 raft_consensus.cc:2468] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:57.565186 12373 leader_election.cc:304] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:57.567260 13024 raft_consensus.cc:2804] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:57.567842 13024 raft_consensus.cc:697] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:28:57.568926 13024 consensus_queue.cc:237] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:57.577911 12528 ts_tablet_manager.cc:1434] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.004s
I20260709 12:28:57.580349 13022 leader_election.cc:290] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 1 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:57.581522 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d9482906bb4ca4a34c22c21f178cab" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:57.582298 12327 raft_consensus.cc:3060] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:57.590273 12206 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:57.592221 12157 catalog_manager.cc:5697] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:57.600522 12327 raft_consensus.cc:2468] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:28:57.614840 12159 catalog_manager.cc:5697] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } }
I20260709 12:28:57.627377 12157 catalog_manager.cc:5697] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:57.630376 12914 consensus_queue.cc:1048] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:57.645255 12154 catalog_manager.cc:5697] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 0 to 1, leader changed from <none> to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 1 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:57.672320 13019 consensus_queue.cc:1048] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:57.798761 12253 raft_consensus.cc:3060] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.998001 13022 consensus_queue.cc:1048] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:57.999183 13024 consensus_queue.cc:1048] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:58.007972 12940 consensus_queue.cc:1048] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:58.029644 12940 consensus_queue.cc:1048] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:58.686496 13055 raft_consensus.cc:493] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.687105 13055 raft_consensus.cc:515] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:58.687544 13075 raft_consensus.cc:493] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.688126 13075 raft_consensus.cc:515] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:58.689544 13055 leader_election.cc:290] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:58.691215 12254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:58.691409 13075 leader_election.cc:290] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:58.691908 12254 raft_consensus.cc:2468] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:28:58.692153 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:58.692759 12410 raft_consensus.cc:2468] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:28:58.693429 12373 leader_election.cc:304] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:58.694402 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:58.694361 12227 leader_election.cc:304] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:28:58.695442 13075 raft_consensus.cc:2804] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:28:58.695889 13075 raft_consensus.cc:493] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:58.696286 13075 raft_consensus.cc:3060] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:58.696650 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:28:58.698133 12929 raft_consensus.cc:2804] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:28:58.702900 12929 raft_consensus.cc:493] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:58.703398 12929 raft_consensus.cc:3060] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:58.707603 13075 raft_consensus.cc:515] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:58.711119 13075 leader_election.cc:290] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:28:58.715730 12253 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:28:58.716150 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:58.720206 12929 raft_consensus.cc:515] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:58.722347 12253 raft_consensus.cc:2393] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2104c8a9b144a419ccb7c07e3f02596 in current term 3: Already voted for candidate 1dcded10caa0493da818e67e13176979 in this term.
I20260709 12:28:58.723030 12929 leader_election.cc:290] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:58.723959 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:28:58.724089 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:28:58.724794 12410 raft_consensus.cc:2393] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1dcded10caa0493da818e67e13176979 in current term 3: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:28:58.724874 12373 leader_election.cc:304] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:28:58.725647 13075 raft_consensus.cc:2749] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:28:58.726200 12224 leader_election.cc:304] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:28:58.726912 12929 raft_consensus.cc:2749] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:28:58.818058 13084 raft_consensus.cc:493] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:28:58.818666 13084 raft_consensus.cc:515] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:58.820649 13084 leader_election.cc:290] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:28:58.821344 12254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:28:58.821612 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:28:58.822216 12254 raft_consensus.cc:2368] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 2. Current term is 3.
I20260709 12:28:58.822456 12410 raft_consensus.cc:2368] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 2. Current term is 3.
I20260709 12:28:58.823411 12299 leader_election.cc:400] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927) with higher term. Message: Invalid argument: T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 2. Current term is 3.
I20260709 12:28:58.823719 12299 leader_election.cc:403] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:58.824250 12302 leader_election.cc:400] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Vote denied by peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155) with higher term. Message: Invalid argument: T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 2. Current term is 3.
I20260709 12:28:58.824241 13084 raft_consensus.cc:3060] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:28:58.831115 13084 raft_consensus.cc:2749] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927) with higher term. Message: Invalid argument: T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 2. Current term is 3.
I20260709 12:28:58.986160  3431 tablet_server.cc:179] TabletServer@127.3.89.193:0 shutting down...
W20260709 12:28:58.992449 12447 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:58.996528 12447 consensus_peers.cc:597] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:59.016578 12447 consensus_peers.cc:597] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.034024  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:59.035060  3431 tablet_replica.cc:333] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.035741  3431 raft_consensus.cc:2243] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.036603  3431 raft_consensus.cc:2272] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.039103  3431 tablet_replica.cc:333] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.039763  3431 raft_consensus.cc:2243] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.040287  3431 raft_consensus.cc:2272] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.042961  3431 tablet_replica.cc:333] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.043934  3431 raft_consensus.cc:2243] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.044569  3431 raft_consensus.cc:2272] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:59.046525 12447 consensus_peers.cc:597] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.048867  3431 tablet_replica.cc:333] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.049862  3431 raft_consensus.cc:2243] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.050871  3431 raft_consensus.cc:2272] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:59.051033 12447 consensus_peers.cc:597] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.053495  3431 tablet_replica.cc:333] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.054550  3431 raft_consensus.cc:2243] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.055112  3431 raft_consensus.cc:2272] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:59.057080 12373 consensus_peers.cc:597] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.058008  3431 tablet_replica.cc:333] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.059072  3431 raft_consensus.cc:2243] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.060243  3431 raft_consensus.cc:2272] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.063587  3431 tablet_replica.cc:333] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
W20260709 12:28:59.064081 12447 consensus_peers.cc:597] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.064392  3431 raft_consensus.cc:2243] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.065035  3431 raft_consensus.cc:2272] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.068874  3431 tablet_replica.cc:333] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.069976  3431 raft_consensus.cc:2243] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.070513  3431 raft_consensus.cc:2272] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.074255  3431 tablet_replica.cc:333] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
W20260709 12:28:59.074326 12447 consensus_peers.cc:597] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.075177  3431 raft_consensus.cc:2243] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.075773  3431 raft_consensus.cc:2272] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.078918  3431 tablet_replica.cc:333] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.079779  3431 raft_consensus.cc:2243] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.080296  3431 raft_consensus.cc:2272] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:59.083022 12373 consensus_peers.cc:597] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.083627  3431 tablet_replica.cc:333] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.084697  3431 raft_consensus.cc:2243] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.085208  3431 raft_consensus.cc:2272] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.088244  3431 tablet_replica.cc:333] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.089071  3431 raft_consensus.cc:2243] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.089707  3431 raft_consensus.cc:2272] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.092654  3431 tablet_replica.cc:333] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.093508  3431 raft_consensus.cc:2243] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.094653  3431 raft_consensus.cc:2272] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.097815  3431 tablet_replica.cc:333] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.098706  3431 raft_consensus.cc:2243] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.099848  3431 raft_consensus.cc:2272] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.103039  3431 tablet_replica.cc:333] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: stopping tablet replica
W20260709 12:28:59.103291 12373 consensus_peers.cc:597] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.103760  3431 raft_consensus.cc:2243] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.104705  3431 raft_consensus.cc:2272] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.108006  3431 tablet_replica.cc:333] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.108692  3431 raft_consensus.cc:2243] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.109172  3431 raft_consensus.cc:2272] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.111747  3431 tablet_replica.cc:333] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.112563  3431 raft_consensus.cc:2243] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:59.112668 12447 consensus_peers.cc:597] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.113022  3431 raft_consensus.cc:2272] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.115389  3431 tablet_replica.cc:333] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.116098  3431 raft_consensus.cc:2243] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.116591  3431 raft_consensus.cc:2272] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.119444  3431 tablet_replica.cc:333] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.120237  3431 raft_consensus.cc:2243] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.120911  3431 raft_consensus.cc:2272] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.123611  3431 tablet_replica.cc:333] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.124415  3431 raft_consensus.cc:2243] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.125588  3431 raft_consensus.cc:2272] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.128060  3431 tablet_replica.cc:333] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.128572  3431 raft_consensus.cc:2243] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.129693  3431 raft_consensus.cc:2272] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.132534  3431 tablet_replica.cc:333] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.133486  3431 raft_consensus.cc:2243] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.134140  3431 raft_consensus.cc:2272] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:59.135119 12299 consensus_peers.cc:597] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.136526  3431 tablet_replica.cc:333] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.137187  3431 raft_consensus.cc:2243] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.137727  3431 raft_consensus.cc:2272] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.140040  3431 tablet_replica.cc:333] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.140698  3431 raft_consensus.cc:2243] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.141610  3431 raft_consensus.cc:2272] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:59.142410 12447 consensus_peers.cc:597] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.144222  3431 tablet_replica.cc:333] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.144996  3431 raft_consensus.cc:2243] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.145475  3431 raft_consensus.cc:2272] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.147710  3431 tablet_replica.cc:333] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.148366  3431 raft_consensus.cc:2243] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.148856  3431 raft_consensus.cc:2272] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.150852  3431 tablet_replica.cc:333] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.151458  3431 raft_consensus.cc:2243] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.151947  3431 raft_consensus.cc:2272] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.153934  3431 tablet_replica.cc:333] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.154493  3431 raft_consensus.cc:2243] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.155264  3431 raft_consensus.cc:2272] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.157840  3431 tablet_replica.cc:333] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.158365  3431 raft_consensus.cc:2243] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.159132  3431 raft_consensus.cc:2272] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.161093  3431 tablet_replica.cc:333] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.161651  3431 raft_consensus.cc:2243] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.162249  3431 raft_consensus.cc:2272] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.165313  3431 tablet_replica.cc:333] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.166122  3431 raft_consensus.cc:2243] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.166617  3431 raft_consensus.cc:2272] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:59.166723 12373 consensus_peers.cc:597] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.168880  3431 tablet_replica.cc:333] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.169440  3431 raft_consensus.cc:2243] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.170261  3431 raft_consensus.cc:2272] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.172293  3431 tablet_replica.cc:333] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.172780  3431 raft_consensus.cc:2243] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.173180  3431 raft_consensus.cc:2272] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.175806  3431 tablet_replica.cc:333] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.176586  3431 raft_consensus.cc:2243] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.177867  3431 raft_consensus.cc:2272] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:59.180387 12447 consensus_peers.cc:597] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:59.180881 12373 consensus_peers.cc:597] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.180903  3431 tablet_replica.cc:333] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.181883  3431 raft_consensus.cc:2243] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.182341  3431 raft_consensus.cc:2272] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.184317  3431 tablet_replica.cc:333] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.184872  3431 raft_consensus.cc:2243] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.185745  3431 raft_consensus.cc:2272] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.187826  3431 tablet_replica.cc:333] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.188397  3431 raft_consensus.cc:2243] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.188910  3431 raft_consensus.cc:2272] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.191250  3431 tablet_replica.cc:333] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.191859  3431 raft_consensus.cc:2243] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.192368  3431 raft_consensus.cc:2272] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.194729  3431 tablet_replica.cc:333] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.195475  3431 raft_consensus.cc:2243] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.196516  3431 raft_consensus.cc:2272] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.199168  3431 tablet_replica.cc:333] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.199849  3431 raft_consensus.cc:2243] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.200342  3431 raft_consensus.cc:2272] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.202497  3431 tablet_replica.cc:333] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.203214  3431 raft_consensus.cc:2243] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.203712  3431 raft_consensus.cc:2272] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.206367  3431 tablet_replica.cc:333] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.207118  3431 raft_consensus.cc:2243] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.208104  3431 raft_consensus.cc:2272] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.210302  3431 tablet_replica.cc:333] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.210829  3431 raft_consensus.cc:2243] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.211277  3431 raft_consensus.cc:2272] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.213967  3431 tablet_replica.cc:333] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.214648  3431 raft_consensus.cc:2243] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.215643  3431 raft_consensus.cc:2272] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.218315  3431 tablet_replica.cc:333] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979: stopping tablet replica
I20260709 12:28:59.218820  3431 raft_consensus.cc:2243] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.219312  3431 raft_consensus.cc:2272] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:59.228664 12299 consensus_peers.cc:597] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:59.245165 12373 consensus_peers.cc:597] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:59.247170 12447 consensus_peers.cc:597] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:59.249172 12447 consensus_peers.cc:597] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:59.264859 12373 consensus_peers.cc:597] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:59.270309 12299 consensus_peers.cc:597] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:59.300792 12373 consensus_peers.cc:597] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.302973  3431 tablet_server.cc:196] TabletServer@127.3.89.193:0 shutdown complete.
W20260709 12:28:59.344772 12299 consensus_peers.cc:597] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.388916  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:59.396701 13091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:59.396696 13092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:59.397383 13094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:59.398156  3431 server_base.cc:1061] running on GCE node
I20260709 12:28:59.401500  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:59.401844  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:59.402060  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600139402036 us; error 0 us; skew 500 ppm
I20260709 12:28:59.402966  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:59.406937  3431 webserver.cc:533] Webserver started at http://127.3.89.193:34689/ using document root <none> and password file <none>
I20260709 12:28:59.407475  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:59.407696  3431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:28:59.411475  3431 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:28:59.420828 13099 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
W20260709 12:28:59.420773 12299 consensus_peers.cc:597] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.422111  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:28:59.422513  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-0-root
uuid: "1dcded10caa0493da818e67e13176979"
format_stamp: "Formatted at 2026-07-09 12:28:45 on dist-test-slave-3hrd"
I20260709 12:28:59.423082  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599794910850-3431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:59.454144  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:59.456172  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260709 12:28:59.457109 12299 consensus_peers.cc:597] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.458401  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:59.465768 13107 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 12:28:59.504004 12299 consensus_peers.cc:597] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:59.517866 12299 consensus_peers.cc:597] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:59.526116 12373 consensus_peers.cc:597] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.786947  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:28:59.787343  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.323s	user 0.011s	sys 0.001s
I20260709 12:28:59.787591  3431 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:28:59.792605 13107 tablet_bootstrap.cc:492] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:59.810056 13107 tablet_bootstrap.cc:492] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:59.810910 13107 tablet_bootstrap.cc:492] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:28:59.811489 13107 ts_tablet_manager.cc:1403] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:28:59.814085 13107 raft_consensus.cc:359] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:59.814867 13107 raft_consensus.cc:740] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:59.815560 13107 consensus_queue.cc:260] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:59.818118 13107 ts_tablet_manager.cc:1434] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:28:59.818833 13107 tablet_bootstrap.cc:492] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:59.835331 13107 tablet_bootstrap.cc:492] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:59.836330 13107 tablet_bootstrap.cc:492] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:28:59.836990 13107 ts_tablet_manager.cc:1403] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.004s
I20260709 12:28:59.839566 13107 raft_consensus.cc:359] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:59.840284 13107 raft_consensus.cc:740] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:59.840937 13107 consensus_queue.cc:260] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:59.842885 13107 ts_tablet_manager.cc:1434] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 12:28:59.843704 13107 tablet_bootstrap.cc:492] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:59.860129 13107 tablet_bootstrap.cc:492] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:59.861086 13107 tablet_bootstrap.cc:492] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:28:59.861747 13107 ts_tablet_manager.cc:1403] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.004s
I20260709 12:28:59.864094 13107 raft_consensus.cc:359] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:59.864744 13107 raft_consensus.cc:740] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:59.865404 13107 consensus_queue.cc:260] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:59.867522 13107 ts_tablet_manager.cc:1434] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:28:59.868348 13107 tablet_bootstrap.cc:492] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:59.884482 13107 tablet_bootstrap.cc:492] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:59.885321 13107 tablet_bootstrap.cc:492] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:28:59.885949 13107 ts_tablet_manager.cc:1403] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 12:28:59.887989 13107 raft_consensus.cc:359] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:59.888624 13107 raft_consensus.cc:740] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:59.889308 13107 consensus_queue.cc:260] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:28:59.891404 13107 ts_tablet_manager.cc:1434] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:28:59.892313 13107 tablet_bootstrap.cc:492] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:59.911132 13107 tablet_bootstrap.cc:492] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:59.912250 13107 tablet_bootstrap.cc:492] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:28:59.913067 13107 ts_tablet_manager.cc:1403] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 12:28:59.915778 13107 raft_consensus.cc:359] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:59.916590 13107 raft_consensus.cc:740] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:59.917541 13107 consensus_queue.cc:260] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:59.919919 13107 ts_tablet_manager.cc:1434] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.005s
I20260709 12:28:59.921187 13107 tablet_bootstrap.cc:492] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:59.948127 13107 tablet_bootstrap.cc:492] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:59.949137 13107 tablet_bootstrap.cc:492] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:28:59.949836 13107 ts_tablet_manager.cc:1403] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.009s
I20260709 12:28:59.952553 13107 raft_consensus.cc:359] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:59.953290 13107 raft_consensus.cc:740] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:59.954089 13107 consensus_queue.cc:260] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:28:59.956424 13107 ts_tablet_manager.cc:1434] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.004s
I20260709 12:28:59.957270 13107 tablet_bootstrap.cc:492] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:28:59.976528 13107 tablet_bootstrap.cc:492] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:59.977600 13107 tablet_bootstrap.cc:492] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:28:59.978298 13107 ts_tablet_manager.cc:1403] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.003s
I20260709 12:28:59.981040 13107 raft_consensus.cc:359] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:59.981812 13107 raft_consensus.cc:740] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:28:59.982643 13107 consensus_queue.cc:260] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:28:59.987337 13107 ts_tablet_manager.cc:1434] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:28:59.988169 13107 tablet_bootstrap.cc:492] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:00.004770 13107 tablet_bootstrap.cc:492] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:00.005720 13107 tablet_bootstrap.cc:492] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:00.006356 13107 ts_tablet_manager.cc:1403] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 12:29:00.008770 13107 raft_consensus.cc:359] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.009445 13107 raft_consensus.cc:740] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:00.010162 13107 consensus_queue.cc:260] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.012079 13107 ts_tablet_manager.cc:1434] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:29:00.012866 13107 tablet_bootstrap.cc:492] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:00.027280 13107 tablet_bootstrap.cc:492] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:00.028174 13107 tablet_bootstrap.cc:492] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:00.028775 13107 ts_tablet_manager.cc:1403] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.004s
I20260709 12:29:00.031010 13107 raft_consensus.cc:359] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.032361 13107 raft_consensus.cc:740] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:00.032997 13107 consensus_queue.cc:260] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.034929 13107 ts_tablet_manager.cc:1434] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:29:00.035765 13107 tablet_bootstrap.cc:492] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:00.049542 13107 tablet_bootstrap.cc:492] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:00.050504 13107 tablet_bootstrap.cc:492] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:00.051112 13107 ts_tablet_manager.cc:1403] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 12:29:00.053670 13107 raft_consensus.cc:359] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.054404 13107 raft_consensus.cc:740] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:00.055169 13107 consensus_queue.cc:260] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.057085 13107 ts_tablet_manager.cc:1434] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.004s
I20260709 12:29:00.057861 13107 tablet_bootstrap.cc:492] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:00.071803  3431 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:29:00.072184  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.285s	user 0.273s	sys 0.000s
I20260709 12:29:00.075271 13107 tablet_bootstrap.cc:492] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:00.076314 13107 tablet_bootstrap.cc:492] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:00.077008 13107 ts_tablet_manager.cc:1403] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.004s
I20260709 12:29:00.079576 13107 raft_consensus.cc:359] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.080276 13107 raft_consensus.cc:740] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:00.081112 13107 consensus_queue.cc:260] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.083348 13107 ts_tablet_manager.cc:1434] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:29:00.084486 13107 tablet_bootstrap.cc:492] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:00.102861 13107 tablet_bootstrap.cc:492] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:00.103843 13107 tablet_bootstrap.cc:492] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:00.104473 13107 ts_tablet_manager.cc:1403] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.008s
I20260709 12:29:00.107059 13107 raft_consensus.cc:359] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.107743 13107 raft_consensus.cc:740] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:00.108281 13107 consensus_queue.cc:260] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.110392 13107 ts_tablet_manager.cc:1434] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:29:00.111123 13107 tablet_bootstrap.cc:492] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:00.123910 13081 raft_consensus.cc:493] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.124713 13081 raft_consensus.cc:515] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.131668 13107 tablet_bootstrap.cc:492] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:00.132637 13107 tablet_bootstrap.cc:492] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:00.133237 13107 ts_tablet_manager.cc:1403] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.022s	user 0.006s	sys 0.011s
I20260709 12:29:00.136478 13107 raft_consensus.cc:359] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.137406 13107 raft_consensus.cc:740] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:00.138146 13107 consensus_queue.cc:260] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.140761 13107 ts_tablet_manager.cc:1434] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:00.141811 13107 tablet_bootstrap.cc:492] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
W20260709 12:29:00.142333 12373 leader_election.cc:336] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:00.143091 13081 leader_election.cc:290] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.150614 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f39a7bfe4bff44b18d1007ced129ae09" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:00.153198 12373 leader_election.cc:304] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:00.154315 13081 raft_consensus.cc:2749] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:00.161103 13107 tablet_bootstrap.cc:492] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:00.162303 13107 tablet_bootstrap.cc:492] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:00.162942 13107 ts_tablet_manager.cc:1403] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 12:29:00.165863 12859 raft_consensus.cc:493] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.166402 12859 raft_consensus.cc:515] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.168298 12859 leader_election.cc:290] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:00.169559 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:00.174605 13107 raft_consensus.cc:359] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.177872 13107 raft_consensus.cc:740] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
W20260709 12:29:00.178293 12299 leader_election.cc:336] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:00.178743 12299 leader_election.cc:304] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:00.179750 13084 raft_consensus.cc:2749] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:00.180591 13107 consensus_queue.cc:260] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.182945 13107 ts_tablet_manager.cc:1434] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:29:00.183614 13107 tablet_bootstrap.cc:492] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:00.192317 13019 raft_consensus.cc:493] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.194433 13019 raft_consensus.cc:515] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.202380 13019 leader_election.cc:290] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:00.205801 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
W20260709 12:29:00.211848 12447 leader_election.cc:336] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:00.212335 12447 leader_election.cc:304] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:00.212999 13019 raft_consensus.cc:2749] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:00.214597 13081 raft_consensus.cc:493] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.215163 13081 raft_consensus.cc:515] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.218031 13109 raft_consensus.cc:493] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.218580 13109 raft_consensus.cc:515] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.220743 13109 leader_election.cc:290] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.221005 13107 tablet_bootstrap.cc:492] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:00.221804 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:00.222362 13107 tablet_bootstrap.cc:492] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:00.223243 13107 ts_tablet_manager.cc:1403] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.003s
I20260709 12:29:00.223474 13081 leader_election.cc:290] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.226289 13107 raft_consensus.cc:359] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.226953 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:00.226970 13107 raft_consensus.cc:740] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:00.228142 13107 consensus_queue.cc:260] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
W20260709 12:29:00.228897 12373 leader_election.cc:336] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:00.229661 12373 leader_election.cc:304] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:00.230345 13081 raft_consensus.cc:2749] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:00.231045 13107 ts_tablet_manager.cc:1434] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:00.232147 13107 tablet_bootstrap.cc:492] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
W20260709 12:29:00.233264 12373 leader_election.cc:336] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:00.233685 12373 leader_election.cc:304] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:00.234282 13081 raft_consensus.cc:2749] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:00.242389  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:37927
I20260709 12:29:00.242486 13186 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.193:37927 every 8 connection(s)
I20260709 12:29:00.246628 13081 raft_consensus.cc:493] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.247195 13081 raft_consensus.cc:515] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.249320 13081 leader_election.cc:290] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:00.249855 13107 tablet_bootstrap.cc:492] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:00.250852 13107 tablet_bootstrap.cc:492] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:00.251463 13107 ts_tablet_manager.cc:1403] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.006s
I20260709 12:29:00.253450 13081 raft_consensus.cc:493] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.254016 13081 raft_consensus.cc:515] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.256240 13081 leader_election.cc:290] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.254015 13107 raft_consensus.cc:359] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.257426 13107 raft_consensus.cc:740] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:00.257977 13107 consensus_queue.cc:260] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.259084 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6417cb26ca8d401f8e73d3512aeab50b" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:00.259665 13107 ts_tablet_manager.cc:1434] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.001s
I20260709 12:29:00.260318 13107 tablet_bootstrap.cc:492] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:00.261080 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:00.274513 13081 raft_consensus.cc:493] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.275122 13081 raft_consensus.cc:515] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.281529 13081 leader_election.cc:290] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:00.283599 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d8f5c63c391477fad5c9155f5022b07" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:00.294206 13107 tablet_bootstrap.cc:492] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:00.295313 13107 tablet_bootstrap.cc:492] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:00.295993 13107 ts_tablet_manager.cc:1403] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.036s	user 0.021s	sys 0.006s
I20260709 12:29:00.296253 13160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6417cb26ca8d401f8e73d3512aeab50b" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:29:00.298013 12373 leader_election.cc:343] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:00.298493 12373 leader_election.cc:304] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:00.298775 13157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d8f5c63c391477fad5c9155f5022b07" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.298916 13107 raft_consensus.cc:359] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.299646 13107 raft_consensus.cc:740] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:00.300083 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.300766 13159 raft_consensus.cc:2468] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:29:00.300716 13107 consensus_queue.cc:260] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.297132 13081 raft_consensus.cc:493] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.299659 13109 raft_consensus.cc:2749] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:00.301680 13081 raft_consensus.cc:515] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.302881 13107 ts_tablet_manager.cc:1434] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:29:00.304422 13107 tablet_bootstrap.cc:492] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:00.304608 13081 leader_election.cc:290] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:00.302873 13088 raft_consensus.cc:493] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.308555 13088 raft_consensus.cc:515] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
W20260709 12:29:00.310293 12373 leader_election.cc:343] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:00.311324 12373 leader_election.cc:304] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:00.311935 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:00.312412 12373 leader_election.cc:304] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:29:00.313112 13109 raft_consensus.cc:2749] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:00.313517 13081 raft_consensus.cc:2804] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:00.310848 13088 leader_election.cc:290] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:00.313992 13081 raft_consensus.cc:493] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.314399 13081 raft_consensus.cc:3060] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.314648 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:00.315162 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:29:00.316960 12373 leader_election.cc:343] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:00.317633 12373 leader_election.cc:304] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:00.318493 13109 raft_consensus.cc:2749] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:00.322137 13188 heartbeater.cc:344] Connected to a master server at 127.3.89.254:45675
I20260709 12:29:00.322608 13188 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:00.323581 13188 heartbeater.cc:507] Master 127.3.89.254:45675 requested a full tablet report, sending...
I20260709 12:29:00.325984 13081 raft_consensus.cc:515] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.327184 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.328298 13081 leader_election.cc:290] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.328109 13159 raft_consensus.cc:2468] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:29:00.330120 12859 raft_consensus.cc:493] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.330725 13157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:00.330715 12859 raft_consensus.cc:515] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.331473 13157 raft_consensus.cc:3060] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.333487 12447 leader_election.cc:304] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:00.334182 12859 leader_election.cc:290] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:00.334717 13088 raft_consensus.cc:2804] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:00.335093 13088 raft_consensus.cc:493] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.328871 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:00.335427 13088 raft_consensus.cc:3060] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.335368 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:00.336727 12405 raft_consensus.cc:2468] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:00.337951 12302 leader_election.cc:304] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:00.338757 12859 raft_consensus.cc:2804] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:00.339103 12859 raft_consensus.cc:493] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.339439 12859 raft_consensus.cc:3060] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.339524 13107 tablet_bootstrap.cc:492] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:00.341825 13107 tablet_bootstrap.cc:492] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:00.343112 13084 raft_consensus.cc:493] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.345067 13088 raft_consensus.cc:515] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.345630 13084 raft_consensus.cc:515] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.346017 13107 ts_tablet_manager.cc:1403] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.015s
I20260709 12:29:00.347797 13084 leader_election.cc:290] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:00.347945 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:00.348665 13159 raft_consensus.cc:3060] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.351152 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:00.351332 13157 raft_consensus.cc:2468] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:29:00.352622 12373 leader_election.cc:304] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:29:00.352807 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6417cb26ca8d401f8e73d3512aeab50b" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:00.353530 12405 raft_consensus.cc:2468] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:00.354189 13107 raft_consensus.cc:359] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.354911 12302 leader_election.cc:304] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:00.355792 13084 raft_consensus.cc:2804] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:00.355928 13109 raft_consensus.cc:2804] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:00.356314 13084 raft_consensus.cc:493] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.356510 13109 raft_consensus.cc:697] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:00.356846 13084 raft_consensus.cc:3060] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.357534 13109 consensus_queue.cc:237] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.359619 13019 raft_consensus.cc:493] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.360188 13019 raft_consensus.cc:515] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.362001 13111 raft_consensus.cc:493] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.362547 13111 raft_consensus.cc:515] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.364753 13110 raft_consensus.cc:493] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.364948 13111 leader_election.cc:290] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:00.365358 13110 raft_consensus.cc:515] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.365950 13159 raft_consensus.cc:2468] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2.
I20260709 12:29:00.350843 13088 leader_election.cc:290] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:00.367589 13110 leader_election.cc:290] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.368391 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:00.369052 12327 raft_consensus.cc:2468] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:29:00.367714 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.370124 13159 raft_consensus.cc:2468] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:29:00.370954 12373 leader_election.cc:304] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:00.371940 13111 raft_consensus.cc:2804] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:00.372314 13111 raft_consensus.cc:493] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.372691 13111 raft_consensus.cc:3060] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.374233 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:00.374944 12479 raft_consensus.cc:2468] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:00.375186 13156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6417cb26ca8d401f8e73d3512aeab50b" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.376331 12914 raft_consensus.cc:493] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.376845 13156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.376883 12914 raft_consensus.cc:515] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.377820 13156 raft_consensus.cc:2468] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:00.380244 12914 leader_election.cc:290] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
W20260709 12:29:00.381395 12299 leader_election.cc:343] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:00.382118 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:00.382580 12299 leader_election.cc:304] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:29:00.382056 12859 raft_consensus.cc:515] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.382975 12410 raft_consensus.cc:2468] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:29:00.384277 12450 leader_election.cc:304] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:00.385354 13019 leader_election.cc:290] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.354921 13107 raft_consensus.cc:740] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:00.386061 13019 raft_consensus.cc:2804] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:00.381129 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.386251 13156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:00.376784 13084 raft_consensus.cc:515] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.387905 13019 raft_consensus.cc:493] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.386700 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:00.388511 13019 raft_consensus.cc:3060] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.388813 12410 raft_consensus.cc:3060] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.391840 12403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6417cb26ca8d401f8e73d3512aeab50b" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:00.392457 13156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.392735 12403 raft_consensus.cc:3060] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.394145 13156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4bdda56dfcd42de9ca75e3e5a9b6c73" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.395790 12447 leader_election.cc:304] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:00.396613 13157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6417cb26ca8d401f8e73d3512aeab50b" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:00.396783 13110 raft_consensus.cc:2804] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:00.397267 13110 raft_consensus.cc:493] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
W20260709 12:29:00.392904 12299 leader_election.cc:343] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:00.397758 13110 raft_consensus.cc:3060] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:00.398317 12299 leader_election.cc:343] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:00.386405 13107 consensus_queue.cc:260] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.401482 13084 leader_election.cc:290] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:00.386458 13111 raft_consensus.cc:515] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.403239 12914 raft_consensus.cc:2804] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:00.403818 12914 raft_consensus.cc:697] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:00.404448 12403 raft_consensus.cc:2468] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
I20260709 12:29:00.405983 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4bdda56dfcd42de9ca75e3e5a9b6c73" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:00.405999 12302 leader_election.cc:304] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:29:00.408358 13084 raft_consensus.cc:2804] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:00.409090 13107 ts_tablet_manager.cc:1434] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.001s
W20260709 12:29:00.410146 12299 leader_election.cc:343] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:00.411290 13107 tablet_bootstrap.cc:492] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:00.411218 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:00.412024 12327 raft_consensus.cc:3060] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.415575 12159 ts_manager.cc:194] Re-registered known tserver with Master: 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.404829 12914 consensus_queue.cc:237] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.408265 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:00.419478 13159 raft_consensus.cc:3060] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.411114 12410 raft_consensus.cc:2468] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
I20260709 12:29:00.425621 12859 leader_election.cc:290] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:00.422477 13111 leader_election.cc:290] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:29:00.455430 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 59711.9us
W20260709 12:29:00.456431 12447 leader_election.cc:343] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:00.457448 12447 leader_election.cc:343] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:00.457937 12447 leader_election.cc:304] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:00.455605 12327 raft_consensus.cc:2468] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:29:00.470206 12859 raft_consensus.cc:493] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.470764 12859 raft_consensus.cc:515] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.467044 13110 raft_consensus.cc:515] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.498673 12859 leader_election.cc:290] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.496949 13111 raft_consensus.cc:493] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.501549 13084 raft_consensus.cc:697] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:00.504570 13111 raft_consensus.cc:515] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.507350 13111 leader_election.cc:290] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:00.509402 13084 consensus_queue.cc:237] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.512902 12914 raft_consensus.cc:2749] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:00.512637 13111 raft_consensus.cc:493] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.516598 13019 raft_consensus.cc:515] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.558748 13157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.559376 13019 leader_election.cc:290] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.559589 13157 raft_consensus.cc:2468] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:29:00.561484 13019 raft_consensus.cc:493] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.582937 13019 raft_consensus.cc:515] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.544324 13111 raft_consensus.cc:515] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.585974 12154 catalog_manager.cc:5697] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:00.592283 13019 leader_election.cc:290] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.595528 12914 raft_consensus.cc:493] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.599148 13109 raft_consensus.cc:493] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.599749 13109 raft_consensus.cc:515] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.600162 13110 leader_election.cc:290] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.601378 13199 raft_consensus.cc:493] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.601728 12859 raft_consensus.cc:493] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.602245 12859 raft_consensus.cc:515] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.602218 13199 raft_consensus.cc:515] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.603384 13207 raft_consensus.cc:493] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.603910 13207 raft_consensus.cc:515] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.604364 12859 leader_election.cc:290] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:00.598052 12914 raft_consensus.cc:515] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.605016 13199 leader_election.cc:290] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.605229 13109 leader_election.cc:290] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.605854 13160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.606500 12914 leader_election.cc:290] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:00.591360 12403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:00.611593 13019 raft_consensus.cc:493] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.601629 13110 raft_consensus.cc:493] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.618407 13019 raft_consensus.cc:515] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.628665 13107 tablet_bootstrap.cc:492] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:00.624917 12302 leader_election.cc:304] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:29:00.630059 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:00.630447 13019 leader_election.cc:290] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.633493 13109 raft_consensus.cc:493] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.634094 13109 raft_consensus.cc:515] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.636078 13109 leader_election.cc:290] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.637521 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:00.641283 12405 raft_consensus.cc:2468] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:29:00.641669 13111 leader_election.cc:290] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.644038 13204 raft_consensus.cc:493] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.618894 12403 raft_consensus.cc:3060] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.638459 13160 raft_consensus.cc:2393] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 in current term 2: Already voted for candidate a800ae68e91f43a794e326badca3320a in this term.
I20260709 12:29:00.652837 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf8c3b9ee65415785e06fb8c085ada7" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:00.653633 12482 raft_consensus.cc:2468] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:29:00.640301 13199 raft_consensus.cc:2804] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:00.658308 13107 tablet_bootstrap.cc:492] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:00.659348 13107 ts_tablet_manager.cc:1403] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.248s	user 0.022s	sys 0.014s
I20260709 12:29:00.624888 13110 raft_consensus.cc:515] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.661885 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:00.669219 12479 raft_consensus.cc:3060] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.672616 13110 leader_election.cc:290] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:00.647233 13204 raft_consensus.cc:515] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.674172 13154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:00.683467 12450 leader_election.cc:304] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:00.688102 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:00.688446 13212 raft_consensus.cc:493] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.689031 12405 raft_consensus.cc:2468] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:29:00.689311 13212 raft_consensus.cc:515] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.691391 13107 raft_consensus.cc:359] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.692196 13107 raft_consensus.cc:740] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:00.692232 13212 leader_election.cc:290] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:00.693078 13107 consensus_queue.cc:260] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.694625 13211 raft_consensus.cc:2804] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:00.695102 13211 raft_consensus.cc:493] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.695494 13211 raft_consensus.cc:3060] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.696214 13159 raft_consensus.cc:2468] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:29:00.696522 12373 leader_election.cc:304] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:00.697290 13111 raft_consensus.cc:2804] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:00.698689 13111 raft_consensus.cc:697] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:00.699355 13160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.684204 13154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.699940 13111 consensus_queue.cc:237] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.701217 13210 raft_consensus.cc:493] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:00.702015 13210 raft_consensus.cc:515] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:00.708202 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f39a7bfe4bff44b18d1007ced129ae09" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:00.709103 12405 raft_consensus.cc:2468] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:29:00.709736 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ae8840033347bea42f3c0ceb4d316a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:00.710634 12484 raft_consensus.cc:2468] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:00.682478 13199 raft_consensus.cc:697] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:00.718847 13199 consensus_queue.cc:237] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:00.722357 13210 leader_election.cc:290] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:00.731621 13107 ts_tablet_manager.cc:1434] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.072s	user 0.009s	sys 0.000s
I20260709 12:29:00.697451 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ae8840033347bea42f3c0ceb4d316a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:00.736508 13107 tablet_bootstrap.cc:492] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:00.755043 13211 raft_consensus.cc:515] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:00.759753 13211 leader_election.cc:290] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
W20260709 12:29:01.370170 13221 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.025s	sys 0.026s
W20260709 12:29:01.403415 13221 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.026s	sys 0.026s
I20260709 12:29:01.943284 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:01.947072 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d8f5c63c391477fad5c9155f5022b07" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:01.947852 12405 raft_consensus.cc:2468] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:01.949536 13223 raft_consensus.cc:493] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.950093 13223 raft_consensus.cc:515] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:01.952716 13223 leader_election.cc:290] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:01.955281 13224 raft_consensus.cc:493] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:01.956013 13150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58646 (request call id 9) took 1289 ms. Trace:
I20260709 12:29:01.955811 13224 raft_consensus.cc:515] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:01.956493 13150 rpcz_store.cc:276] 0709 12:29:00.667038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.667336 (+   298us) service_pool.cc:224] Handling call
0709 12:29:01.955608 (+1288272us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b580056bb98. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff348ffc0 00007efff347a929 00007efff347a3d2 00007effec9f1bd4
0709 12:29:01.955921 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30928768}
I20260709 12:29:01.957650 13223 raft_consensus.cc:493] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:01.958249 13223 raft_consensus.cc:515] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:01.960712 13223 leader_election.cc:290] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:01.965193 13227 raft_consensus.cc:493] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
W20260709 12:29:01.966152 12154 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.356s	user 0.013s	sys 0.008s
I20260709 12:29:01.966107 13227 raft_consensus.cc:515] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
W20260709 12:29:01.967589 12154 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.357s	user 0.014s	sys 0.008s
I20260709 12:29:01.968708 13223 raft_consensus.cc:493] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:01.969086 13227 leader_election.cc:290] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:01.969388 13223 raft_consensus.cc:515] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:01.971928 13223 leader_election.cc:290] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:01.958622 13224 leader_election.cc:290] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:01.974056 13019 raft_consensus.cc:493] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:01.974633 13019 raft_consensus.cc:515] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:01.976609 13019 leader_election.cc:290] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:01.977421 13019 raft_consensus.cc:493] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:01.977926 13019 raft_consensus.cc:515] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:01.978624 13214 raft_consensus.cc:493] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:01.978853 13212 raft_consensus.cc:493] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:01.979207 13214 raft_consensus.cc:515] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:01.979833 13019 leader_election.cc:290] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:01.979580 13212 raft_consensus.cc:515] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:01.981056 13019 raft_consensus.cc:493] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:01.981534 13019 raft_consensus.cc:515] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:01.982147 13214 leader_election.cc:290] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:01.983317 13019 leader_election.cc:290] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:01.983687 13019 raft_consensus.cc:493] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:01.983979 13019 raft_consensus.cc:515] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:01.984277 13110 raft_consensus.cc:493] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2104c8a9b144a419ccb7c07e3f02596)
I20260709 12:29:01.984989 13110 raft_consensus.cc:515] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:01.985659 13019 leader_election.cc:290] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:01.987455 13212 leader_election.cc:290] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:01.988581 13228 raft_consensus.cc:493] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:01.989166 13228 raft_consensus.cc:515] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:01.990662 13223 raft_consensus.cc:493] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:01.991271 13223 raft_consensus.cc:515] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:01.991838 13228 leader_election.cc:290] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:01.994007 13228 raft_consensus.cc:493] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2104c8a9b144a419ccb7c07e3f02596)
I20260709 12:29:01.994515 13228 raft_consensus.cc:515] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:01.996688 13228 leader_election.cc:290] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:01.998481 13227 raft_consensus.cc:493] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:01.998721 13226 raft_consensus.cc:493] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:01.999007 13227 raft_consensus.cc:515] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:01.999450 13226 raft_consensus.cc:515] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:02.000072 13110 leader_election.cc:290] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.001065 13227 leader_election.cc:290] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:02.002017 13223 leader_election.cc:290] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:02.002076 13226 leader_election.cc:290] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.002696 12450 leader_election.cc:304] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:02.003544 12450 leader_election.cc:304] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:02.004442 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:02.004591 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2a2e0c681f24264a729f89d08042fd9" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.005108 12410 raft_consensus.cc:3060] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.005230 13019 raft_consensus.cc:2804] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.005724 13019 raft_consensus.cc:493] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:02.006132 13019 raft_consensus.cc:3060] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.002939 13226 raft_consensus.cc:493] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:02.007404 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.007828 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d163b2f24c845d8b11b35d222659673" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.008965 12404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.009989 13232 raft_consensus.cc:2804] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.010433 13232 raft_consensus.cc:493] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:02.010823 13232 raft_consensus.cc:3060] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.008780 13226 raft_consensus.cc:515] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:02.012913 13226 leader_election.cc:290] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.013509 12373 leader_election.cc:304] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:02.020131 13019 raft_consensus.cc:515] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:02.020186 13151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ae8840033347bea42f3c0ceb4d316a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.028333 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b0d4a9a29a4de9bc3a8ab529535d68" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.028843 13151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58654 (request call id 12) took 1362 ms. Trace:
I20260709 12:29:02.022567 13232 raft_consensus.cc:515] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:02.023128 12302 leader_election.cc:304] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:02.030359 13110 raft_consensus.cc:2804] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.030819 13110 raft_consensus.cc:493] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:02.030999 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:02.031235 13110 raft_consensus.cc:3060] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.023778 13230 raft_consensus.cc:493] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.031638 12408 raft_consensus.cc:3060] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.032044 13230 raft_consensus.cc:515] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.034381 13230 leader_election.cc:290] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.023528 13212 raft_consensus.cc:493] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.026114 13225 raft_consensus.cc:2804] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.027352 13231 raft_consensus.cc:493] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.031252 13019 leader_election.cc:290] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.025475 12410 raft_consensus.cc:2468] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2.
I20260709 12:29:02.206549 13212 raft_consensus.cc:515] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.206813 13231 raft_consensus.cc:515] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.206563 13225 raft_consensus.cc:493] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:02.208360 13225 raft_consensus.cc:3060] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.037503 12404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f01aed6d25447b0be01b95f4518c8ef" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.209089 13231 leader_election.cc:290] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.210093 13231 raft_consensus.cc:493] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.210541 13231 raft_consensus.cc:515] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.212580 13231 leader_election.cc:290] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.029261 13151 rpcz_store.cc:276] 0709 12:29:00.666369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.666754 (+   385us) service_pool.cc:224] Handling call
0709 12:29:02.019830 (+1353076us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b580056bb98. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:29:02.028777 (+  8947us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30264704}
I20260709 12:29:02.214110 13151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.215116 12450 leader_election.cc:304] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:02.215624 13212 leader_election.cc:290] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.216002 13019 raft_consensus.cc:2804] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.039927 12408 raft_consensus.cc:2468] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2.
I20260709 12:29:02.040999 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.041682 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.042026 12331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.218986 12332 raft_consensus.cc:2468] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 3.
I20260709 12:29:02.043710 13232 leader_election.cc:290] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.044567 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.046005 13227 raft_consensus.cc:493] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:02.047039 13224 raft_consensus.cc:493] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.222116 13225 raft_consensus.cc:515] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:02.222398 13224 raft_consensus.cc:515] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:02.225071 13224 leader_election.cc:290] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:02.048161 13110 raft_consensus.cc:515] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.048373 13153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:02.226152 13153 raft_consensus.cc:3060] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.227677 13110 leader_election.cc:290] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.048988 13081 raft_consensus.cc:493] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:02.053068 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f39a7bfe4bff44b18d1007ced129ae09" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:02.055570 13214 raft_consensus.cc:493] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2104c8a9b144a419ccb7c07e3f02596)
I20260709 12:29:02.057302 13234 raft_consensus.cc:493] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.059391 13228 raft_consensus.cc:493] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2104c8a9b144a419ccb7c07e3f02596)
I20260709 12:29:02.060220 13226 raft_consensus.cc:493] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:02.060114 13088 raft_consensus.cc:493] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:02.062422 13230 raft_consensus.cc:493] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.066140 12477 raft_consensus.cc:3060] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.070972 12478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
W20260709 12:29:02.072544 12447 leader_election.cc:343] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:02.076422 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.076967 13157 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58654 (request call id 9) took 1449 ms. Trace:
W20260709 12:29:02.078599 12403 consensus_meta.cc:287] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596: Time spent flushing consensus metadata: real 1.428s	user 0.013s	sys 0.011s
I20260709 12:29:02.083817 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a81175bbecf4ba6b6be50b490564d9b" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.091322 12329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.091589 13084 raft_consensus.cc:493] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.099324 13229 raft_consensus.cc:493] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.100354 13237 raft_consensus.cc:493] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.101958 12476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.106366 13238 raft_consensus.cc:493] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.107535 13240 raft_consensus.cc:493] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.117434 13241 raft_consensus.cc:493] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.122700 13242 raft_consensus.cc:493] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.123793 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f01aed6d25447b0be01b95f4518c8ef" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.124183 13239 raft_consensus.cc:493] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.125932 12373 leader_election.cc:304] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:02.126360 12480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.142545 13207 leader_election.cc:290] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.143993 13245 raft_consensus.cc:493] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.145524 12402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.149011 13150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.149955 12914 raft_consensus.cc:493] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:02.150558 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f39a7bfe4bff44b18d1007ced129ae09" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.151968 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.153192 13154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.154943 12330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.158187 13244 raft_consensus.cc:493] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.159552 13243 raft_consensus.cc:493] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2104c8a9b144a419ccb7c07e3f02596)
I20260709 12:29:02.159808 13246 raft_consensus.cc:493] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2104c8a9b144a419ccb7c07e3f02596)
I20260709 12:29:02.159983 12475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.160379 13155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58630 (request call id 10) took 1589 ms. Trace:
I20260709 12:29:02.163412 13248 raft_consensus.cc:493] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.166216 13233 raft_consensus.cc:493] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:02.166878 13247 raft_consensus.cc:493] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.167850 13249 raft_consensus.cc:493] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.169498 13160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:02.173868 13204 leader_election.cc:290] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.179020 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4bdda56dfcd42de9ca75e3e5a9b6c73" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.181589 12158 catalog_manager.cc:5697] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:02.182446 13251 raft_consensus.cc:493] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2104c8a9b144a419ccb7c07e3f02596)
I20260709 12:29:02.181614 13250 raft_consensus.cc:493] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.184056 13252 raft_consensus.cc:493] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.187295 13200 raft_consensus.cc:493] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.248361 13019 raft_consensus.cc:697] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:02.194816 13253 raft_consensus.cc:493] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:02.196862 13235 raft_consensus.cc:493] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.197628 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.197939 13116 raft_consensus.cc:493] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.199249 13223 raft_consensus.cc:493] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.257903 12157 catalog_manager.cc:5697] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 2 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:02.410991 12482 raft_consensus.cc:2393] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2104c8a9b144a419ccb7c07e3f02596 in current term 2: Already voted for candidate a800ae68e91f43a794e326badca3320a in this term.
I20260709 12:29:02.225263 13225 leader_election.cc:290] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.231419 13228 raft_consensus.cc:515] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:02.230505 12407 raft_consensus.cc:3060] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.237929 12478 raft_consensus.cc:2468] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:29:02.232766 13152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d8f5c63c391477fad5c9155f5022b07" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.243774 13237 raft_consensus.cc:515] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:02.232035 13230 raft_consensus.cc:515] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:02.222139 13227 raft_consensus.cc:515] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.209136 12410 raft_consensus.cc:1275] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:02.381650 12154 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57026 (request call id 99) took 1743 ms. Trace:
I20260709 12:29:02.251494 13248 raft_consensus.cc:515] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.258683 13233 raft_consensus.cc:515] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:02.246964 13232 raft_consensus.cc:493] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
W20260709 12:29:02.237926 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 165403us
I20260709 12:29:02.248974 12327 raft_consensus.cc:2468] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
W20260709 12:29:02.246400 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 120572us
W20260709 12:29:02.463951 12327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56372 (request call id 127) took 1801 ms (client timeout 1534 ms). Trace:
W20260709 12:29:02.464363 12327 rpcz_store.cc:269] 0709 12:29:00.662605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.664173 (+  1568us) service_pool.cc:224] Handling call
0709 12:29:02.463806 (+1799633us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:02.250474 13243 raft_consensus.cc:515] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.247632 12914 raft_consensus.cc:515] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:02.377276 12483 raft_consensus.cc:2468] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
W20260709 12:29:02.412995 12302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 195996us
I20260709 12:29:02.469798 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f01aed6d25447b0be01b95f4518c8ef" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.414580 13107 tablet_bootstrap.cc:700] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: Time spent opening tablet: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:29:02.415804 13019 consensus_queue.cc:237] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:02.229506 12328 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56384 (request call id 197) took 1603 ms. Trace:
I20260709 12:29:02.474908 12330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7916cfbec26b4115a9251411482bed41" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.475062 12328 rpcz_store.cc:276] 0709 12:29:00.625976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:02.201119 (+1575143us) service_pool.cc:224] Handling call
0709 12:29:02.201469 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:02.226169 (+ 24700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:02.226219 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:02.226273 (+    54us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:29:02.226314 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:02.229218 (+  2904us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b5400971154. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b6e01 00007effee4b583e 00007efff347a419 00007effec9f1bd4 00007effec9f1b62 00007effec9f1a91 00007effec9f1a0c 00007effec9f0cc2 00007effec7f148c 00007effec7f08c6
0709 12:29:02.229274 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:02.229439 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56236416}
W20260709 12:29:02.246112 12479 consensus_meta.cc:287] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a: Time spent flushing consensus metadata: real 1.576s	user 0.019s	sys 0.008s
I20260709 12:29:02.476588 12479 raft_consensus.cc:2468] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
W20260709 12:29:02.477144 12479 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59418 (request call id 271) took 1832 ms (client timeout 1899 ms). Trace:
W20260709 12:29:02.235803 13111 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.527s	user 0.020s	sys 0.014s
W20260709 12:29:02.477860 13111 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.769s	user 0.022s	sys 0.022s
I20260709 12:29:02.479936 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.480121 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe640756185b42d7874674a29af747da" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.480675 12483 raft_consensus.cc:2468] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:02.481716 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d163b2f24c845d8b11b35d222659673" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.446559 13237 leader_election.cc:290] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.483351 13199 consensus_queue.cc:1048] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:02.484118 12402 raft_consensus.cc:1275] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:02.231195 13234 raft_consensus.cc:515] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:02.442031 12154 rpcz_store.cc:276] 0709 12:29:00.517366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.517679 (+   313us) service_pool.cc:224] Handling call
0709 12:29:01.968044 (+1450365us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:01.968079 (+    35us) write_op.cc:454] Released schema lock
0709 12:29:02.260935 (+292856us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:00.749917 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:00.750598 (+   681us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:00.751040 (+   442us) write_op.cc:435] Acquired schema lock
0709 12:29:00.752055 (+  1015us) tablet.cc:662] Decoding operations
0709 12:29:00.752424 (+   369us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:00.752486 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:00.752515 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:29:00.752697 (+   182us) tablet.cc:701] Row locks acquired
0709 12:29:00.752714 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:29:00.752875 (+   161us) write_op.cc:270] Start()
0709 12:29:00.752981 (+   106us) write_op.cc:276] Timestamp: P: 1783600140752823 usec, L: 0
0709 12:29:00.752999 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:29:00.753748 (+   749us) log.cc:844] Serialized 537 byte log entry
0709 12:29:00.770667 (+ 16919us) op_driver.cc:464] REPLICATION: finished
0709 12:29:01.061688 (+291021us) write_op.cc:301] APPLY: starting
0709 12:29:01.061800 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:01.061932 (+   132us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:01.402203 (+340271us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:01.404457 (+  2254us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:01.404692 (+   235us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:29:01.404761 (+    69us) write_op.cc:312] APPLY: finished
0709 12:29:01.405003 (+   242us) log.cc:844] Serialized 22 byte log entry
0709 12:29:01.405956 (+   953us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:01.406067 (+   111us) write_op.cc:454] Released schema lock
0709 12:29:01.406120 (+    53us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":217728,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":290019,"apply.run_cpu_time_us":2279,"apply.run_wall_time_us":344865,"num_ops":1,"prepare.queue_time_us":136105,"prepare.run_cpu_time_us":7718,"prepare.run_wall_time_us":15756,"raft.queue_time_us":2519,"raft.run_cpu_time_us":53412,"raft.run_wall_time_us":637238,"replication_time_us":17492,"thread_start_us":2007008,"threads_started":4,"wal-append.queue_time_us":11669,"wal-append.run_cpu_time_us":3600,"wal-append.run_wall_time_us":1643680}]]}
I20260709 12:29:02.486862 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d8f5c63c391477fad5c9155f5022b07" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:02.478678 12407 raft_consensus.cc:2468] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2.
I20260709 12:29:02.487600 12405 raft_consensus.cc:3060] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.247393 13150 raft_consensus.cc:2393] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a800ae68e91f43a794e326badca3320a in current term 2: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:29:02.488963 13150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164179fdd9ee45ff80ea3530e210088a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.490010 13150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcca3076fc03410aa86e82c060c29335" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.490675 12914 leader_election.cc:290] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.491659 13150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.492830 13150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.244477 13240 raft_consensus.cc:515] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.402899 13200 raft_consensus.cc:515] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.495334 13240 leader_election.cc:290] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:02.495864 13240 raft_consensus.cc:493] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.496557 13200 leader_election.cc:290] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.496623 13154 raft_consensus.cc:2468] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:29:02.415221 13253 raft_consensus.cc:515] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:02.243254 13084 raft_consensus.cc:515] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.499224 13154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e72262c8e2b24a8fa97e1cacbb60b49a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.500296 13154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.248656 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.242115 12477 raft_consensus.cc:1275] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:02.231673 13226 raft_consensus.cc:515] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.502838 13226 leader_election.cc:290] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:02.259734 13249 raft_consensus.cc:515] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.231980 13088 raft_consensus.cc:515] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:02.440552 12299 leader_election.cc:304] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
W20260709 12:29:02.508656 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 68173.7us
I20260709 12:29:02.509894 12299 leader_election.cc:304] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:02.445708 13255 raft_consensus.cc:493] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.242120 12403 raft_consensus.cc:2468] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2.
I20260709 12:29:02.250049 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:29:02.449340 12447 leader_election.cc:343] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:02.518978 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 69700.7us
W20260709 12:29:02.519699 12447 leader_election.cc:343] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:02.230693 12450 leader_election.cc:304] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:02.244313 13238 raft_consensus.cc:515] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:02.244046 12476 raft_consensus.cc:2468] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:29:02.243530 13229 raft_consensus.cc:515] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.451593 12478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcca3076fc03410aa86e82c060c29335" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
W20260709 12:29:02.524886 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 58508.7us
I20260709 12:29:02.443921 13223 raft_consensus.cc:515] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:02.453701 13227 leader_election.cc:290] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:02.244729 13241 raft_consensus.cc:515] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.246026 13239 raft_consensus.cc:515] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.244993 13242 raft_consensus.cc:515] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.230109 13081 raft_consensus.cc:515] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.413946 13116 raft_consensus.cc:515] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.409896 13235 raft_consensus.cc:515] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:02.241920 13157 rpcz_store.cc:276] 0709 12:29:00.646573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.647097 (+   524us) service_pool.cc:224] Handling call
0709 12:29:02.076672 (+1429575us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b580056bb98. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff348ffc0 00007efff347a929 00007efff347a3d2 00007effec9f1bd4
0709 12:29:02.076914 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5373696}
I20260709 12:29:02.536233 13256 consensus_queue.cc:1048] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:29:02.536831 13157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
W20260709 12:29:02.523824 12403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55938 (request call id 153) took 1954 ms (client timeout 1802 ms). Trace:
I20260709 12:29:02.537782 13256 raft_consensus.cc:3060] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.538015 13157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.538702 13157 raft_consensus.cc:2468] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
I20260709 12:29:02.539582 13157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4bdda56dfcd42de9ca75e3e5a9b6c73" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.539867 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.526005 12159 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48222
I20260709 12:29:02.456532 13248 leader_election.cc:290] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:02.457686 13232 raft_consensus.cc:515] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:02.459296 13230 leader_election.cc:290] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:02.459715 13228 leader_election.cc:290] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.462872 12475 raft_consensus.cc:2468] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:29:02.550837 13258 raft_consensus.cc:493] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:02.537797 12403 rpcz_store.cc:269] 0709 12:29:00.565150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.572309 (+  7159us) service_pool.cc:224] Handling call
0709 12:29:02.517502 (+1945193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:02.551915 13232 leader_election.cc:290] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.552635 13229 leader_election.cc:290] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.554342 13249 leader_election.cc:290] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:02.556161 12328 raft_consensus.cc:1275] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:02.523533 13255 raft_consensus.cc:515] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:02.470451 12482 raft_consensus.cc:2468] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:29:02.473048 12480 raft_consensus.cc:2468] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:29:02.473870 13160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf8c3b9ee65415785e06fb8c085ada7" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.402987 13252 raft_consensus.cc:515] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.532325 13233 leader_election.cc:290] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.378053 13251 raft_consensus.cc:515] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:02.500897 13084 leader_election.cc:290] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.250283 13224 raft_consensus.cc:2804] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:02.417503 13153 raft_consensus.cc:2468] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
I20260709 12:29:02.500968 13240 raft_consensus.cc:515] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.230926 13214 raft_consensus.cc:515] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:02.539604 13116 leader_election.cc:290] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.487601 13234 leader_election.cc:290] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:02.500897 13154 raft_consensus.cc:2468] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:02.557696 12330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d163b2f24c845d8b11b35d222659673" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.576745 12330 raft_consensus.cc:2468] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:29:02.576982 13214 leader_election.cc:290] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:02.259158 13247 raft_consensus.cc:515] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:02.578996 13240 leader_election.cc:290] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
W20260709 12:29:02.524094 12450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 293463us
I20260709 12:29:02.250731 13246 raft_consensus.cc:515] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.524325 13019 raft_consensus.cc:2804] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.250227 13244 raft_consensus.cc:515] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.378298 13250 raft_consensus.cc:515] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.587903 12159 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48222 (request call id 1) took 2178 ms. Trace:
I20260709 12:29:02.515843 13110 raft_consensus.cc:2804] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.258699 13245 raft_consensus.cc:515] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.567879 13224 raft_consensus.cc:493] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
W20260709 12:29:02.477530 12479 rpcz_store.cc:269] 0709 12:29:00.644700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.646440 (+  1740us) service_pool.cc:224] Handling call
0709 12:29:02.477064 (+1830624us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:02.523774 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.251230 13155 rpcz_store.cc:276] 0709 12:29:00.590036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:02.125423 (+1535387us) service_pool.cc:224] Handling call
0709 12:29:02.125770 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:02.159282 (+ 33512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:02.159320 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:02.159398 (+    78us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:29:02.159444 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:02.160106 (+   662us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:02.160221 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22152,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":119,"thread_start_us":33081,"threads_started":1}
I20260709 12:29:02.524663 12481 raft_consensus.cc:2468] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:29:02.537359 13239 leader_election.cc:290] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:02.549844 13159 raft_consensus.cc:2468] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:29:02.558535 12373 leader_election.cc:304] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
W20260709 12:29:02.466428 13152 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58654 (request call id 11) took 1800 ms (client timeout 1563 ms). Trace:
I20260709 12:29:02.566139 13088 leader_election.cc:290] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.566883 13252 leader_election.cc:290] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:29:02.521433 12447 leader_election.cc:343] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:02.608467 13227 raft_consensus.cc:2804] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:02.608566 13238 leader_election.cc:290] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
W20260709 12:29:02.608851 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 87440.5us
W20260709 12:29:02.609935 12447 leader_election.cc:343] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:02.610349 13245 leader_election.cc:290] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:29:02.610809 12447 leader_election.cc:343] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:02.611723 12447 leader_election.cc:304] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
W20260709 12:29:02.612845 12447 leader_election.cc:343] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:02.575179 13153 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58654 (request call id 10) took 1915 ms (client timeout 1899 ms). Trace:
I20260709 12:29:02.549122 13081 leader_election.cc:290] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:02.586444 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.586743 13247 leader_election.cc:290] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:02.551339 13258 raft_consensus.cc:515] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
W20260709 12:29:02.589133 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 79330.3us
I20260709 12:29:02.613507 12447 leader_election.cc:304] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:02.622745 13233 raft_consensus.cc:2804] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.595057 13151 raft_consensus.cc:2468] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 3.
I20260709 12:29:02.623378 13258 leader_election.cc:290] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
W20260709 12:29:02.623891 12447 leader_election.cc:343] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:02.624219 13155 raft_consensus.cc:1275] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:29:02.624670 12447 leader_election.cc:343] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:02.625531 12299 leader_election.cc:343] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:02.582727 13019 raft_consensus.cc:697] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:02.613842 13088 raft_consensus.cc:2749] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:02.605146 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:29:02.626596 12299 leader_election.cc:343] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:02.626844 13019 consensus_queue.cc:237] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:02.627507 12299 leader_election.cc:304] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:29:02.627972 13250 leader_election.cc:290] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.628129 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.629321 13235 leader_election.cc:290] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:02.613233 12405 raft_consensus.cc:2468] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
W20260709 12:29:02.625828 13153 rpcz_store.cc:269] 0709 12:29:00.665493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.665863 (+   370us) service_pool.cc:224] Handling call
0709 12:29:02.048178 (+1382315us) spinlock_profiling.cc:243] Waited 9.28 ms on lock 0x7b580056bb98. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:29:02.572362 (+524184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21333248}
I20260709 12:29:02.630980 13241 leader_election.cc:290] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:29:02.606492 13152 rpcz_store.cc:269] 0709 12:29:00.666010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.666336 (+   326us) service_pool.cc:224] Handling call
0709 12:29:02.232462 (+1566126us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b580056bb98. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effeb251db1 00007effeb391b71 00007effeb391a22 00007efff42aa8d9 00007efff3404ee0 00007efff34ce0ab 00007efff34ce131 00007efff347c0a0 00007efff347b984 00007effec9f66d4 00007effec9f6662
0709 12:29:02.466352 (+233890us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31176960}
I20260709 12:29:02.634341 13223 leader_election.cc:290] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.603487 12327 raft_consensus.cc:2468] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:29:02.602288 13150 raft_consensus.cc:2468] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:02.598486 13224 raft_consensus.cc:3060] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:02.591706 13110 raft_consensus.cc:493] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:02.638593 13110 raft_consensus.cc:3060] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.644265 13246 leader_election.cc:290] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.645304 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f39a7bfe4bff44b18d1007ced129ae09" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:02.645612 13224 raft_consensus.cc:515] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.646698 12299 leader_election.cc:304] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:29:02.614564 13255 leader_election.cc:290] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:02.518723 13253 leader_election.cc:290] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.591388 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eace44731c5747ca98999cf217b561a3" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.647480 13231 raft_consensus.cc:2804] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:02.647626 13212 raft_consensus.cc:2804] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.648013 13231 raft_consensus.cc:493] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:02.595672 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cc6fec74b6423bb49415fbcdd93e55" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.648154 13212 raft_consensus.cc:493] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:02.648422 13231 raft_consensus.cc:3060] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:02.648574 13212 raft_consensus.cc:3060] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.608978 13227 raft_consensus.cc:493] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:02.647567 12299 leader_election.cc:304] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:29:02.649120 13227 raft_consensus.cc:3060] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:29:02.649857 12299 leader_election.cc:343] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:02.650928 13214 raft_consensus.cc:2804] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.575503 13243 leader_election.cc:290] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.623239 13233 raft_consensus.cc:493] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:02.651455 13214 raft_consensus.cc:697] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:02.651619 12299 leader_election.cc:304] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596
W20260709 12:29:02.605201 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 46708.8us
I20260709 12:29:02.652385 13214 consensus_queue.cc:237] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.653489 13199 raft_consensus.cc:2804] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.653970 13199 raft_consensus.cc:493] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2104c8a9b144a419ccb7c07e3f02596)
I20260709 12:29:02.654390 13199 raft_consensus.cc:3060] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.588295 12159 rpcz_store.cc:276] 0709 12:29:00.409707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.409898 (+   191us) service_pool.cc:224] Handling call
0709 12:29:02.587817 (+2177919us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70982,"reported_tablets":45}
I20260709 12:29:02.654065 12406 raft_consensus.cc:2393] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 in current term 3: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:29:02.658866 13225 consensus_queue.cc:1048] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:02.651777 13233 raft_consensus.cc:3060] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:02.661686 12299 consensus_peers.cc:597] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:02.663192 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ae8840033347bea42f3c0ceb4d316a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.623385 13251 leader_election.cc:290] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.666440 13188 heartbeater.cc:499] Master 127.3.89.254:45675 was elected leader, sending a full tablet report...
I20260709 12:29:02.594455 12403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf8c3b9ee65415785e06fb8c085ada7" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:29:02.668480 12450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 78385.4us
I20260709 12:29:02.616852 13242 leader_election.cc:290] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.678406 12373 leader_election.cc:304] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:02.692456 12373 leader_election.cc:304] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:02.694833 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.698804 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2a2e0c681f24264a729f89d08042fd9" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.685180 13224 leader_election.cc:290] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:02.706341 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.706693 13237 raft_consensus.cc:2804] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.707056 12410 raft_consensus.cc:2468] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:02.707121 13237 raft_consensus.cc:493] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:02.707564 13237 raft_consensus.cc:3060] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.701613 13111 raft_consensus.cc:2804] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.708992 13111 raft_consensus.cc:493] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:02.709367 13111 raft_consensus.cc:3060] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.710386 13256 raft_consensus.cc:2749] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:02.710955 13107 tablet_bootstrap.cc:492] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:02.711920 13107 tablet_bootstrap.cc:492] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:02.712677 13107 ts_tablet_manager.cc:1403] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 1.977s	user 0.022s	sys 0.021s
I20260709 12:29:02.713371 13233 raft_consensus.cc:515] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.714107 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a81175bbecf4ba6b6be50b490564d9b" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.715574 13233 leader_election.cc:290] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:02.715283 13231 raft_consensus.cc:515] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.722031 13111 raft_consensus.cc:515] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:02.618554 13244 leader_election.cc:290] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.725350 13107 raft_consensus.cc:359] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.726433 13107 raft_consensus.cc:740] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:02.727562 12403 raft_consensus.cc:2393] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a800ae68e91f43a794e326badca3320a in current term 2: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:29:02.730453 13107 consensus_queue.cc:260] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.753535 12329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.757910 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.758742 12334 raft_consensus.cc:2393] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2104c8a9b144a419ccb7c07e3f02596 in current term 3: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:29:02.759588 13110 raft_consensus.cc:515] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.760830 12327 raft_consensus.cc:2393] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a800ae68e91f43a794e326badca3320a in current term 2: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:29:02.767012 13227 raft_consensus.cc:515] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.735468 12330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.774709 13237 raft_consensus.cc:515] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:02.775758 13155 raft_consensus.cc:1275] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:02.777822 13157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6cc6fec74b6423bb49415fbcdd93e55" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.778699 13199 raft_consensus.cc:515] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:02.780821 13199 leader_election.cc:290] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.781489 13212 raft_consensus.cc:515] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.782672 12336 raft_consensus.cc:2393] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a800ae68e91f43a794e326badca3320a in current term 2: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:29:02.783589 13212 leader_election.cc:290] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.782706 13237 leader_election.cc:290] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.788172 12404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13100e49825f4fcfa78791b73b8d7555" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.788542 12403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a03f6aed8fcb4ea2b45b6432634c00ae" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.734642 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd19eddb6c2e45b7acefc5c295d92928" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.728922 12450 leader_election.cc:304] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
W20260709 12:29:02.791193 12450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 62550.2us
I20260709 12:29:02.801697 12302 leader_election.cc:304] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:02.808152 13107 ts_tablet_manager.cc:1434] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.095s	user 0.009s	sys 0.000s
I20260709 12:29:02.823271 13107 tablet_bootstrap.cc:492] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:02.823518 12302 leader_election.cc:304] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:02.825452 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d9482906bb4ca4a34c22c21f178cab" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.789728 12404 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f0d8651a6144ca9fd21ccc0f57edd9" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.774431 13110 leader_election.cc:290] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:02.803357 13232 raft_consensus.cc:2804] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.824148 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.862715 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67e469fc10a43bf9660c48b5ae1335a" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.864374 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.865731 12447 leader_election.cc:304] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:02.866998 12480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd891675417c4265b77695b4227d980f" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.868268 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d163b2f24c845d8b11b35d222659673" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.869400 12477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf8c3b9ee65415785e06fb8c085ada7" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:02.870019 12477 raft_consensus.cc:3060] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.829659 13212 raft_consensus.cc:2804] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.871912 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:02.874118 12914 raft_consensus.cc:2804] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.874698 12914 raft_consensus.cc:493] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:02.875471 12914 raft_consensus.cc:3060] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:02.874482 12447 leader_election.cc:343] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:02.880177 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.882493 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76b0d4a9a29a4de9bc3a8ab529535d68" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.882768 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a81175bbecf4ba6b6be50b490564d9b" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.884914 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff454851114445da8be05eda167efc4" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.886031 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e72262c8e2b24a8fa97e1cacbb60b49a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.887066 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.887555 13111 leader_election.cc:290] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:02.888309 12330 raft_consensus.cc:1275] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:02.830928 12403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7916cfbec26b4115a9251411482bed41" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.890937 13227 leader_election.cc:290] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:02.891419 12407 raft_consensus.cc:1240] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Rejecting Update request from peer 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:02.893941 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b789be887a492fb0e51b08f2475a97" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.884958 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.830746 13270 raft_consensus.cc:2804] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.877810 13231 leader_election.cc:290] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:02.898057 13270 raft_consensus.cc:493] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:02.898531 13270 raft_consensus.cc:3060] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.887209 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eace44731c5747ca98999cf217b561a3" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.906822 12914 raft_consensus.cc:515] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:02.912329 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.888339 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.923816 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83d73b1bb20e4e79819072554a1ee076" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:02.924702 12410 raft_consensus.cc:2376] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2. Re-sending same reply.
I20260709 12:29:02.928435 13155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf8c3b9ee65415785e06fb8c085ada7" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:02.930310 12450 leader_election.cc:304] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:29:02.864969 13232 raft_consensus.cc:697] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:02.871325 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "164179fdd9ee45ff80ea3530e210088a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.934957 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe640756185b42d7874674a29af747da" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.974368 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
W20260709 12:29:02.930346 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 238063us
I20260709 12:29:02.880239 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.907366 13157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff454851114445da8be05eda167efc4" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.974005 13231 consensus_queue.cc:1059] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:29:02.889477 12329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f01aed6d25447b0be01b95f4518c8ef" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:02.981355 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.907110 13212 raft_consensus.cc:697] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:02.986969 12914 leader_election.cc:290] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:02.931033 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:02.977711 13155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf8c3b9ee65415785e06fb8c085ada7" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:02.993784 12299 leader_election.cc:304] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
I20260709 12:29:02.993901 12336 raft_consensus.cc:1240] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Rejecting Update request from peer a2104c8a9b144a419ccb7c07e3f02596 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:02.974309 13233 raft_consensus.cc:2804] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:03.002038 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ae8840033347bea42f3c0ceb4d316a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:03.002681 12484 raft_consensus.cc:3060] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.999578 12373 leader_election.cc:304] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:02.980563 12403 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f0d8651a6144ca9fd21ccc0f57edd9" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:02.997167 12475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a81175bbecf4ba6b6be50b490564d9b" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:03.007614 13152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cf8c3b9ee65415785e06fb8c085ada7" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.008687 13152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:03.009284 12478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:03.010162 13233 raft_consensus.cc:697] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:03.010689 12402 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:03.010936 13156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd19eddb6c2e45b7acefc5c295d92928" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.011530 12402 raft_consensus.cc:2393] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1dcded10caa0493da818e67e13176979 in current term 4: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:29:03.011369 13233 consensus_queue.cc:237] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:03.012516 12330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:03.013715 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f0d8651a6144ca9fd21ccc0f57edd9" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.014881 13150 raft_consensus.cc:1275] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:02.996562 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:03.015926 13150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.016413 12406 raft_consensus.cc:2393] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 in current term 3: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:29:03.016832 13154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd891675417c4265b77695b4227d980f" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.017053 13150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4bdda56dfcd42de9ca75e3e5a9b6c73" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:03.018070 13266 raft_consensus.cc:2749] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.003611 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:03.001967 13235 consensus_queue.cc:1048] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:29:03.019520 13266 raft_consensus.cc:3055] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:02.995002 12480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:03.020010 13266 raft_consensus.cc:740] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:03.009325 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05dfc5c361a94fe5af3e9a5462a0b12f" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:03.006839 13151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7916cfbec26b4115a9251411482bed41" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.013157 13153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67e469fc10a43bf9660c48b5ae1335a" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.021134 13266 consensus_queue.cc:260] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.009248 13152 raft_consensus.cc:3060] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:03.022791 13266 raft_consensus.cc:3060] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.022567 13232 consensus_queue.cc:237] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:03.019639 12333 raft_consensus.cc:2393] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2104c8a9b144a419ccb7c07e3f02596 in current term 3: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:29:03.010277 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:03.024489 12482 raft_consensus.cc:2393] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 in current term 2: Already voted for candidate a800ae68e91f43a794e326badca3320a in this term.
I20260709 12:29:03.025005 13153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.026296 13160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b789be887a492fb0e51b08f2475a97" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.027206 13151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.027915 12299 leader_election.cc:304] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.028013 13151 raft_consensus.cc:2468] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:03.028890 13110 raft_consensus.cc:2749] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.029136 13212 consensus_queue.cc:237] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
W20260709 12:29:03.030512 13183 debug-util.cc:398] Leaking SignalData structure 0x7b08002986c0 after lost signal to thread 3434
I20260709 12:29:03.027956 13223 raft_consensus.cc:2804] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:03.031428 13223 raft_consensus.cc:493] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:03.031883 13223 raft_consensus.cc:3060] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:03.031900 13183 debug-util.cc:398] Leaking SignalData structure 0x7b080038b7a0 after lost signal to thread 12193
I20260709 12:29:03.040009 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263de4693a84afe93a508aff7f1dc29" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:03.004853 12373 leader_election.cc:304] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
W20260709 12:29:03.051668 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 48099.1us
I20260709 12:29:03.051780 13227 raft_consensus.cc:2804] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:03.052399 13227 raft_consensus.cc:493] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:03.052809 13227 raft_consensus.cc:3060] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:03.055567 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 139242us
W20260709 12:29:03.056821 12447 leader_election.cc:343] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:03.057324 12447 leader_election.cc:304] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
W20260709 12:29:03.073606 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 45765.9us
I20260709 12:29:03.074522 12299 leader_election.cc:304] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.026607 13154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d8f5c63c391477fad5c9155f5022b07" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:03.025854 12401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:29:03.122411 12450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 47669.4us
W20260709 12:29:03.140805 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 74710.2us
W20260709 12:29:03.174274 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 99832.9us
I20260709 12:29:03.176483 12373 leader_election.cc:304] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
W20260709 12:29:03.122413 12447 leader_election.cc:343] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:03.181201 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 105313us
W20260709 12:29:03.187736 12447 leader_election.cc:343] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:03.199342 12447 leader_election.cc:304] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.193212 12373 leader_election.cc:304] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:03.124317 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:03.206888 12447 leader_election.cc:304] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.127990 13156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ae8840033347bea42f3c0ceb4d316a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:03.185657 13231 raft_consensus.cc:2749] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.170084 13251 raft_consensus.cc:2749] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.210244 13019 raft_consensus.cc:2749] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:29:03.200172 13183 debug-util.cc:398] Leaking SignalData structure 0x7b0800155260 after lost signal to thread 12361
I20260709 12:29:03.203816 12373 leader_election.cc:304] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:03.201036 13111 raft_consensus.cc:2804] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:03.215716 13111 raft_consensus.cc:493] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:03.216101 12373 leader_election.cc:304] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:03.217250 12373 leader_election.cc:304] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:03.219094 13223 raft_consensus.cc:515] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.221757 13235 raft_consensus.cc:2804] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:03.222208 13235 raft_consensus.cc:493] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:03.222592 13235 raft_consensus.cc:3060] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.226326 13155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.227744 13155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0c8b89ad96240b588abe04f13ea2f9d" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:03.228866 13155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:03.229873 13152 raft_consensus.cc:2468] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 3.
I20260709 12:29:03.230875 13153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:03.231535 13153 raft_consensus.cc:3060] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:03.229441 13227 raft_consensus.cc:515] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.234762 12401 raft_consensus.cc:1240] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Rejecting Update request from peer a800ae68e91f43a794e326badca3320a for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:03.235708 12404 raft_consensus.cc:1275] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.236254 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.236950 13204 consensus_queue.cc:1059] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.238108 13227 leader_election.cc:290] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:03.240480 13253 raft_consensus.cc:3055] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:03.240943 13253 raft_consensus.cc:740] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:03.239240 13227 raft_consensus.cc:2804] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:03.241830 13253 consensus_queue.cc:260] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.236950 13159 raft_consensus.cc:2468] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:03.244481 13081 raft_consensus.cc:2804] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:03.242463 13227 raft_consensus.cc:493] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:03.245121 13081 raft_consensus.cc:493] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:03.234742 13154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:03.182112 13225 consensus_queue.cc:1048] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:29:03.212832 12327 raft_consensus.cc:3060] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:03.277927 13151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:03.208451 13237 raft_consensus.cc:2804] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:03.249118 13227 raft_consensus.cc:3060] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.261457 12405 raft_consensus.cc:1275] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.261938 13253 raft_consensus.cc:3060] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.248832 13081 raft_consensus.cc:3060] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.294317 13228 consensus_queue.cc:1048] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:03.216173 13111 raft_consensus.cc:3060] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.268864 13157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1263de4693a84afe93a508aff7f1dc29" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
W20260709 12:29:03.270689 13183 debug-util.cc:398] Leaking SignalData structure 0x7b08003d18e0 after lost signal to thread 12509
I20260709 12:29:03.273424 13156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:03.276675 13088 consensus_queue.cc:1048] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:03.283296 12484 raft_consensus.cc:2468] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
I20260709 12:29:03.082973 13226 consensus_queue.cc:1048] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.129s
I20260709 12:29:03.284297 13237 raft_consensus.cc:493] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:03.251400 13223 leader_election.cc:290] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:03.127682 13270 raft_consensus.cc:515] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.279721 13154 raft_consensus.cc:3060] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.303396 12409 raft_consensus.cc:1240] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Rejecting Update request from peer a800ae68e91f43a794e326badca3320a for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:03.304186 13156 raft_consensus.cc:3060] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.308092 13237 raft_consensus.cc:3060] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:03.356711 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 161676us
W20260709 12:29:03.351788 13183 debug-util.cc:398] Leaking SignalData structure 0x7b080026ede0 after lost signal to thread 13186
W20260709 12:29:03.358302 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 141105us
I20260709 12:29:03.339284 13088 consensus_queue.cc:1059] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:29:03.356796 12477 raft_consensus.cc:2468] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:29:03.382038 12914 raft_consensus.cc:2749] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.385664 13270 leader_election.cc:290] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:03.402169 13153 raft_consensus.cc:2468] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 4.
I20260709 12:29:03.404181 13226 raft_consensus.cc:3055] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:03.404812 13226 raft_consensus.cc:740] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:03.407035 13237 raft_consensus.cc:515] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.409056 12157 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57042 (request call id 90) took 2776 ms. Trace:
I20260709 12:29:03.414057 12158 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:57018 (request call id 93) took 2807 ms. Trace:
I20260709 12:29:03.409646 12157 rpcz_store.cc:276] 0709 12:29:00.633039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.633271 (+   232us) service_pool.cc:224] Handling call
0709 12:29:02.476194 (+1842923us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b54008936a8. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff1e20161 00007efff1eaa3af 00007efff1eabb11 00007efff43abfa1 00007efff43ac7ff 00007efff426ba1b 00007efff437390e 00007efff28342a4 00007efff2834232 00007efff2834161 00007efff28340dc
0709 12:29:03.408971 (+932777us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:02.935614 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:03.033885 (+ 98271us) spinlock_profiling.cc:243] Waited 7.16 ms on lock 0x7b5c0012638c. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007efff46a8331 00007efff46a8221 00007efff46a2f6c 00007efff1f00d69 00007efff1eec0a6 00007efff1eebd58 00007efff1ef09f1 00007efff1ef09a9 00007efff1ef0939 00007efff1ef0901 00007efff1eefb2d
0709 12:29:03.033983 (+    98us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:03.034025 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:29:03.034054 (+    29us) tablet.cc:662] Decoding operations
0709 12:29:03.043645 (+  9591us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:03.043718 (+    73us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:03.043762 (+    44us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:29:03.043985 (+   223us) tablet.cc:701] Row locks acquired
0709 12:29:03.044001 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:29:03.044145 (+   144us) write_op.cc:270] Start()
0709 12:29:03.044400 (+   255us) write_op.cc:276] Timestamp: P: 1783600143044103 usec, L: 0
0709 12:29:03.044434 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 12:29:03.237030 (+192596us) log.cc:844] Serialized 537 byte log entry
0709 12:29:03.238203 (+  1173us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b5400893854. stack: 00007effeb397343 00007effeb397240 00007effead261d4 00007efff4a64304 00007efff4a64299 00007effee41f571 00007effee4b1e28 00007efff1eec581 00007efff1eebd58 00007efff1ef09f1 00007efff1ef09a9 00007efff1ef0939 00007efff1ef0901 00007efff1eefb2d 0000000000388634 0000000000388479
0709 12:29:03.239777 (+  1574us) op_driver.cc:464] REPLICATION: finished
0709 12:29:03.395770 (+155993us) write_op.cc:301] APPLY: starting
0709 12:29:03.395842 (+    72us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:03.395938 (+    96us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:03.395961 (+    23us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:03.396170 (+   209us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:03.396317 (+   147us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:29:03.396373 (+    56us) write_op.cc:312] APPLY: finished
0709 12:29:03.396536 (+   163us) log.cc:844] Serialized 22 byte log entry
0709 12:29:03.396686 (+   150us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:03.396762 (+    76us) write_op.cc:454] Released schema lock
0709 12:29:03.396808 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:03.397139 (+   331us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:03.397167 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1588546,"reported_tablets":1,"spinlock_wait_cycles":9257216,"child_traces":[["op",{"apply.queue_time_us":155786,"apply.run_cpu_time_us":1340,"apply.run_wall_time_us":1340,"num_ops":1,"prepare.queue_time_us":430012,"prepare.run_cpu_time_us":14899,"prepare.run_wall_time_us":318992,"replication_time_us":195196,"spinlock_wait_cycles":381381248,"thread_start_us":112546,"threads_started":1}]]}
I20260709 12:29:03.418431 13237 leader_election.cc:290] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:03.384042 12299 leader_election.cc:304] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
W20260709 12:29:03.430615 12373 leader_election.cc:343] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:03.431397 12373 leader_election.cc:343] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:03.433061 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 49109.7us
I20260709 12:29:03.430768 12159 catalog_manager.cc:5697] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } }
I20260709 12:29:03.445998 12159 catalog_manager.cc:5697] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:03.446631 12299 leader_election.cc:304] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.413036 13227 raft_consensus.cc:515] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
W20260709 12:29:03.451197 12373 leader_election.cc:343] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:03.452062 12447 leader_election.cc:304] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
W20260709 12:29:03.454339 12373 leader_election.cc:343] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:03.454901 12373 leader_election.cc:304] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
W20260709 12:29:03.456084 12373 leader_election.cc:343] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:03.456840 12373 leader_election.cc:304] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:03.459576 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b789be887a492fb0e51b08f2475a97" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.432350 13266 raft_consensus.cc:2749] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:29:03.461653 12373 leader_election.cc:343] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:03.461933 12299 leader_election.cc:304] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
W20260709 12:29:03.462328 12373 leader_election.cc:343] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:03.462925 12373 leader_election.cc:304] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:03.407073 13081 raft_consensus.cc:515] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.460489 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.463941 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.464330 12331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.464668 12332 raft_consensus.cc:2393] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1dcded10caa0493da818e67e13176979 in current term 2: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:29:03.460700 13228 raft_consensus.cc:2749] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.465342 13081 leader_election.cc:290] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:03.458832 13289 raft_consensus.cc:2749] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.469631 13281 raft_consensus.cc:2804] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:03.470257 13281 raft_consensus.cc:697] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 3 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:03.471210 13281 consensus_queue.cc:237] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:03.467940 13225 raft_consensus.cc:2749] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.471227 12405 raft_consensus.cc:1240] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Rejecting Update request from peer 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:03.474233 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.414505 12158 rpcz_store.cc:276] 0709 12:29:00.606660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.606859 (+   199us) service_pool.cc:224] Handling call
0709 12:29:03.413972 (+2807113us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:02.720797 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:02.721372 (+   575us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:02.721408 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:29:02.721429 (+    21us) tablet.cc:662] Decoding operations
0709 12:29:02.721809 (+   380us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:02.721861 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:02.721899 (+    38us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:29:02.722090 (+   191us) tablet.cc:701] Row locks acquired
0709 12:29:02.722110 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:29:02.722248 (+   138us) write_op.cc:270] Start()
0709 12:29:02.722354 (+   106us) write_op.cc:276] Timestamp: P: 1783600142722205 usec, L: 0
0709 12:29:02.722374 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:29:02.723035 (+   661us) log.cc:844] Serialized 537 byte log entry
0709 12:29:03.035044 (+312009us) op_driver.cc:464] REPLICATION: finished
0709 12:29:03.353050 (+318006us) write_op.cc:301] APPLY: starting
0709 12:29:03.357848 (+  4798us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:03.362840 (+  4992us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:03.394048 (+ 31208us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:03.394302 (+   254us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:03.394484 (+   182us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:29:03.394529 (+    45us) write_op.cc:312] APPLY: finished
0709 12:29:03.394709 (+   180us) log.cc:844] Serialized 22 byte log entry
0709 12:29:03.394885 (+   176us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:03.394994 (+   109us) write_op.cc:454] Released schema lock
0709 12:29:03.395044 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:03.395414 (+   370us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:03.395441 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1571666,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":309377,"apply.run_cpu_time_us":1862,"apply.run_wall_time_us":46107,"num_ops":1,"prepare.queue_time_us":319242,"prepare.run_cpu_time_us":17575,"prepare.run_wall_time_us":145734,"raft.queue_time_us":166374,"raft.run_cpu_time_us":78875,"raft.run_wall_time_us":286994,"replication_time_us":312513,"thread_start_us":587048,"threads_started":4,"wal-append.queue_time_us":143898}]]}
I20260709 12:29:03.475003 12328 raft_consensus.cc:2468] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 1.
I20260709 12:29:03.423781 13111 raft_consensus.cc:515] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.476564 13235 raft_consensus.cc:515] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.477898 13111 leader_election.cc:290] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:03.478677 12407 raft_consensus.cc:1240] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Rejecting Update request from peer 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:03.478883 12373 leader_election.cc:304] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:03.479152 12410 raft_consensus.cc:1240] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Rejecting Update request from peer 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:03.453269 13212 raft_consensus.cc:2749] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.497542 12447 leader_election.cc:304] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.485870 13110 raft_consensus.cc:2804] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:03.498872 13110 raft_consensus.cc:697] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:03.499441 12447 leader_election.cc:304] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.499737 13110 consensus_queue.cc:237] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:03.500608 12447 leader_election.cc:304] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.501365 13295 raft_consensus.cc:2749] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.501926 13019 raft_consensus.cc:2749] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.502866 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13100e49825f4fcfa78791b73b8d7555" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.491933 13154 raft_consensus.cc:2468] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
I20260709 12:29:03.438665 13226 consensus_queue.cc:260] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.504395 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d9482906bb4ca4a34c22c21f178cab" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.504829 13226 raft_consensus.cc:3060] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:03.505882 13232 consensus_queue.cc:1175] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4e02e5f693ce49a3b161ad90848be7d4" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 12:29:03.464630 12335 raft_consensus.cc:2468] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:03.454211 13223 consensus_queue.cc:1048] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:29:03.483719 13156 raft_consensus.cc:2468] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2.
I20260709 12:29:03.501403 13228 raft_consensus.cc:2749] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.487510 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:03.487556 12477 raft_consensus.cc:1275] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.519834 13157 raft_consensus.cc:1240] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Rejecting Update request from peer 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:03.511538 12373 leader_election.cc:304] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.487617 13289 raft_consensus.cc:2804] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:03.517177 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:03.516968 13225 consensus_queue.cc:1059] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:03.517901 12331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.509104 12299 leader_election.cc:304] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.522387 12299 leader_election.cc:304] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.523550 12299 leader_election.cc:304] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.524026 12407 raft_consensus.cc:1275] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:29:03.524655 12299 leader_election.cc:343] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:03.527050 12373 leader_election.cc:304] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.528612 12299 leader_election.cc:304] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.529661 12299 leader_election.cc:304] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.538061 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:03.538441 13156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4bdda56dfcd42de9ca75e3e5a9b6c73" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.539012 12332 raft_consensus.cc:2393] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a800ae68e91f43a794e326badca3320a in current term 2: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:29:03.525674 13228 consensus_queue.cc:1048] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:29:03.547412 13225 raft_consensus.cc:3055] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:03.547871 13225 raft_consensus.cc:740] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:03.548697 13225 consensus_queue.cc:260] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:03.549631 13111 raft_consensus.cc:2749] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:29:03.554103 12299 leader_election.cc:343] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:03.551932 13289 raft_consensus.cc:697] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 4 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:03.553236 13290 raft_consensus.cc:2749] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.555536 13288 raft_consensus.cc:2749] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:29:03.555696 13289 consensus_queue.cc:237] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:03.556643 13292 raft_consensus.cc:2804] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:03.556811 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f9d02452bc41bbab1eb4cb7e1ccf44" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.555269 12299 leader_election.cc:304] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.559661 12373 leader_election.cc:304] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.558646 13107 tablet_bootstrap.cc:492] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:03.567937 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.557108 13292 raft_consensus.cc:493] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:03.527552 13266 raft_consensus.cc:2749] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.557196 12331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.558686 12299 leader_election.cc:304] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
W20260709 12:29:03.571660 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 44417.4us
I20260709 12:29:03.511379 12327 raft_consensus.cc:2468] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 4.
I20260709 12:29:03.597620 13107 tablet_bootstrap.cc:492] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:03.598637 13107 ts_tablet_manager.cc:1403] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.776s	user 0.026s	sys 0.015s
I20260709 12:29:03.568250 13084 raft_consensus.cc:2804] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:03.566426 13237 raft_consensus.cc:2749] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.567476 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7916cfbec26b4115a9251411482bed41" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.567910 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
W20260709 12:29:03.604713 12299 consensus_peers.cc:597] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:03.611690 12299 leader_election.cc:343] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:03.600191 12447 leader_election.cc:304] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:03.612130 12299 leader_election.cc:304] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.568989 13227 leader_election.cc:290] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:03.608208 13229 raft_consensus.cc:2749] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.616622 13231 consensus_queue.cc:1059] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:03.609493 13225 raft_consensus.cc:3060] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.607892 13235 leader_election.cc:290] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:03.599757 13228 raft_consensus.cc:493] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2104c8a9b144a419ccb7c07e3f02596)
I20260709 12:29:03.621433 13228 raft_consensus.cc:515] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:03.622354 12447 leader_election.cc:304] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.623857 12447 leader_election.cc:304] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:03.623901 13228 leader_election.cc:290] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:03.625752 13019 raft_consensus.cc:2749] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.582497 13111 raft_consensus.cc:2749] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.626065 12447 leader_election.cc:304] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.607267 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05dfc5c361a94fe5af3e9a5462a0b12f" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.578397 13292 raft_consensus.cc:3060] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.558244 13200 raft_consensus.cc:2749] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.603578 13084 raft_consensus.cc:493] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:03.626680 12409 raft_consensus.cc:1275] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.627353 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89bcc99e094d1bbaf83979c2b78081" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:03.629232 13229 raft_consensus.cc:2749] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.631639 13107 raft_consensus.cc:359] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:03.625475 13295 raft_consensus.cc:2749] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.633095 12447 leader_election.cc:304] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.634907 13107 raft_consensus.cc:740] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:03.639029 12447 leader_election.cc:304] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.640066 12914 raft_consensus.cc:2749] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.641022 12299 leader_election.cc:304] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.641353 12447 leader_election.cc:304] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.642374 13019 raft_consensus.cc:2749] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.644448 12447 leader_election.cc:304] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:03.644804 13107 consensus_queue.cc:260] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:03.654273 13290 consensus_queue.cc:1048] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:03.655654 12407 raft_consensus.cc:1240] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Rejecting Update request from peer 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:03.655947 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:03.657720 13156 raft_consensus.cc:1275] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.659114 13100 leader_election.cc:304] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:29:03.661310 12409 raft_consensus.cc:1240] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Rejecting Update request from peer 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:03.662154 13290 consensus_queue.cc:1048] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.662621 13107 ts_tablet_manager.cc:1434] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.001s
I20260709 12:29:03.634951 13084 raft_consensus.cc:3060] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.664685 13019 raft_consensus.cc:2749] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.633767 13253 raft_consensus.cc:2749] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.678833 13107 tablet_bootstrap.cc:492] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:03.682847 13264 raft_consensus.cc:3055] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:03.662882 13231 raft_consensus.cc:2804] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:03.683341 13264 raft_consensus.cc:740] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:03.683878 13231 raft_consensus.cc:697] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:03.684450 13264 consensus_queue.cc:260] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.684834 13231 consensus_queue.cc:237] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:03.686100 12331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d163b2f24c845d8b11b35d222659673" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:03.670503 12159 catalog_manager.cc:5697] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 2 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.690537 12159 catalog_manager.cc:5697] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 2 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.692652 12159 catalog_manager.cc:5697] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 2 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.695945 12159 catalog_manager.cc:5697] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 2 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.647248 13226 raft_consensus.cc:2749] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.701079 13273 raft_consensus.cc:2804] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:03.707948 13273 raft_consensus.cc:493] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:03.708354 13273 raft_consensus.cc:3060] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.637007 13232 raft_consensus.cc:2749] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.711297 12484 raft_consensus.cc:1240] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Rejecting Update request from peer 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:03.634087 13233 consensus_queue.cc:1048] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 12:29:03.662822 13204 consensus_queue.cc:1048] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:29:03.730525 12331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:03.710254 13264 raft_consensus.cc:3060] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.725142 13084 raft_consensus.cc:515] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:03.740343 13084 leader_election.cc:290] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
W20260709 12:29:03.709502 13305 log.cc:927] Time spent T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4: Append to log took a long time: real 0.061s	user 0.002s	sys 0.000s
I20260709 12:29:03.646543 13255 consensus_queue.cc:1059] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.086s
I20260709 12:29:03.742406 13255 raft_consensus.cc:3055] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:03.742830 13255 raft_consensus.cc:740] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:03.743722 13255 consensus_queue.cc:260] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.745100 13255 raft_consensus.cc:3060] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.635386 13212 consensus_queue.cc:1059] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
W20260709 12:29:03.746901 12299 consensus_peers.cc:597] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:03.747494 13154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:03.747511 13199 raft_consensus.cc:3055] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:03.748219 13199 raft_consensus.cc:740] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:03.752892 13292 raft_consensus.cc:515] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
W20260709 12:29:03.768056 12450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 121910us
I20260709 12:29:03.665709 13266 consensus_queue.cc:1059] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.095s
I20260709 12:29:03.769325 12302 leader_election.cc:304] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.786181 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
W20260709 12:29:03.790076 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 146757us
I20260709 12:29:03.792567 12477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e94567d0d49d4bf0882ca910a8757cbc" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:03.797906 12299 leader_election.cc:304] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.749663 13199 consensus_queue.cc:260] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
W20260709 12:29:03.801067 13100 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 142208us
W20260709 12:29:03.804749 12373 leader_election.cc:343] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:03.806527 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4bdda56dfcd42de9ca75e3e5a9b6c73" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:03.808530 12484 raft_consensus.cc:3060] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.790459 13229 raft_consensus.cc:2749] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.779794 13230 consensus_queue.cc:1059] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 12:29:03.816869 12480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:03.818491 12331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:03.811849 12475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:03.801898 13084 consensus_queue.cc:1059] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 12:29:03.819842 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:03.806145 13266 raft_consensus.cc:2749] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.831955 12328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:03.836112 12333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:03.837718 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f01aed6d25447b0be01b95f4518c8ef" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:03.812677 13231 consensus_queue.cc:1059] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:29:03.842935 13231 raft_consensus.cc:3055] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:03.843353 13231 raft_consensus.cc:740] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:03.844522 13110 raft_consensus.cc:3055] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:03.844322 13231 consensus_queue.cc:260] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.844973 13110 raft_consensus.cc:740] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:03.845956 13110 consensus_queue.cc:260] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:03.847795 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d163b2f24c845d8b11b35d222659673" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:03.856618 13107 tablet_bootstrap.cc:492] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:03.859740 13288 raft_consensus.cc:3055] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:03.860257 13288 raft_consensus.cc:740] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:03.864991 13110 raft_consensus.cc:3060] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.868111 13288 consensus_queue.cc:260] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:03.870009 12477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:03.870880 13288 raft_consensus.cc:3060] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.873883 12410 raft_consensus.cc:1275] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.874961 13292 leader_election.cc:290] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:03.875458 12407 raft_consensus.cc:1240] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Rejecting Update request from peer 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:03.860951 13273 raft_consensus.cc:515] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:03.878077 13273 leader_election.cc:290] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:03.853354 13100 leader_election.cc:304] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
W20260709 12:29:03.879379 13100 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 63798.8us
I20260709 12:29:03.880187 13100 leader_election.cc:304] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
W20260709 12:29:03.880935 13306 log.cc:927] Time spent T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979: Append to log took a long time: real 0.229s	user 0.003s	sys 0.000s
I20260709 12:29:03.886837 13157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:03.889170 12299 leader_election.cc:304] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.889751 13311 raft_consensus.cc:2749] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.890594 13212 raft_consensus.cc:2749] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.897615 12484 raft_consensus.cc:2468] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:29:03.900653 12482 raft_consensus.cc:2468] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:29:03.904445 13255 consensus_queue.cc:1048] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.907795 13107 tablet_bootstrap.cc:492] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:03.861171 13231 raft_consensus.cc:3060] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.908661 13107 ts_tablet_manager.cc:1403] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.230s	user 0.010s	sys 0.027s
I20260709 12:29:03.915516 12476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33683dafd0e7419e8f52bdf98e28e33c" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:03.918088 12480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:03.918251 12478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f01aed6d25447b0be01b95f4518c8ef" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:03.890625 13311 raft_consensus.cc:2749] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.897543 13199 raft_consensus.cc:3060] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.922096 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:03.929879 12373 leader_election.cc:304] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
W20260709 12:29:03.935199 12447 consensus_peers.cc:597] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:03.936698 12334 raft_consensus.cc:1240] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Rejecting Update request from peer a800ae68e91f43a794e326badca3320a for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:03.936723 12373 leader_election.cc:304] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:03.938557 12336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5ae8840033347bea42f3c0ceb4d316a" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:03.938954 12334 raft_consensus.cc:1240] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Rejecting Update request from peer a800ae68e91f43a794e326badca3320a for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:03.881408 13100 leader_election.cc:304] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:29:03.939638 13289 raft_consensus.cc:2804] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:03.943028 12154 catalog_manager.cc:5697] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 2 to 3, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:03.945888 13289 raft_consensus.cc:697] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:03.947166 12373 leader_election.cc:304] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
W20260709 12:29:03.951433 12447 consensus_peers.cc:597] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:03.951267 12154 catalog_manager.cc:5697] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 1 to 4, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 4 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:03.952590 12447 leader_election.cc:304] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:03.955965 12447 leader_election.cc:304] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:03.945529 12475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:03.949280 13311 raft_consensus.cc:2804] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:03.949295 13225 raft_consensus.cc:2749] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.964284 12373 leader_election.cc:304] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
W20260709 12:29:03.945485 13100 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 64159.9us
I20260709 12:29:03.968815 13100 leader_election.cc:304] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.969753 13273 raft_consensus.cc:2749] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:03.970271 13100 leader_election.cc:304] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.971853 13233 consensus_queue.cc:1059] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.972131 13100 leader_election.cc:304] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:03.973057 12333 raft_consensus.cc:2468] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:29:03.973230 13226 consensus_queue.cc:1059] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:29:03.965704 13311 raft_consensus.cc:493] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:03.975018 13311 raft_consensus.cc:3060] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:03.976738 13081 raft_consensus.cc:2804] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:03.978039 13289 consensus_queue.cc:237] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:03.978713 13273 raft_consensus.cc:2749] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:03.959239 13253 raft_consensus.cc:2749] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.979501 13273 raft_consensus.cc:2749] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.965622 12373 leader_election.cc:304] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:03.966684 12329 raft_consensus.cc:1240] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Rejecting Update request from peer a800ae68e91f43a794e326badca3320a for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:03.967914 13150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b89bcc99e094d1bbaf83979c2b78081" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:03.958232 13295 raft_consensus.cc:2804] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:03.986558 13107 raft_consensus.cc:359] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:03.988688 13107 raft_consensus.cc:740] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:03.962803 13303 raft_consensus.cc:2749] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.992522 13225 raft_consensus.cc:2804] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:04.022507 13295 raft_consensus.cc:697] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:04.023114 13225 raft_consensus.cc:697] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:04.024096 13225 consensus_queue.cc:237] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:04.024175 13295 consensus_queue.cc:237] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:03.987651 13081 raft_consensus.cc:697] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:04.026532 13081 consensus_queue.cc:237] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:04.033290 12335 raft_consensus.cc:2468] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
W20260709 12:29:04.042641 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 76113.6us
I20260709 12:29:04.043731 12447 leader_election.cc:304] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
W20260709 12:29:04.046602 12299 consensus_peers.cc:597] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:04.054275 13228 raft_consensus.cc:493] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:04.054934 13228 raft_consensus.cc:515] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:04.055476 12477 raft_consensus.cc:2468] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:29:04.044148 12475 raft_consensus.cc:1275] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.056751 13314 raft_consensus.cc:2749] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.057157 13228 leader_election.cc:290] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:04.059691 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:03.991019 13107 consensus_queue.cc:260] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:03.960690 12328 raft_consensus.cc:2468] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:29:04.073016 13107 ts_tablet_manager.cc:1434] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.163s	user 0.008s	sys 0.000s
I20260709 12:29:04.068523 13253 consensus_queue.cc:1059] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:29:04.075641 12373 leader_election.cc:304] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.076236 13253 raft_consensus.cc:3055] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:04.076668 13253 raft_consensus.cc:740] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:04.077185 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "485001bca51c4145939dc8bd38b1169a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:04.077452 12477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:04.078161 12483 raft_consensus.cc:2393] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1dcded10caa0493da818e67e13176979 in current term 2: Already voted for candidate a800ae68e91f43a794e326badca3320a in this term.
I20260709 12:29:04.078837 12475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:04.079200 13303 raft_consensus.cc:2749] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.080240 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:04.080775 12373 leader_election.cc:304] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.081662 13237 raft_consensus.cc:2749] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.082456 12373 leader_election.cc:304] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:04.085952 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:04.088069 13107 tablet_bootstrap.cc:492] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
W20260709 12:29:04.090305 12447 consensus_peers.cc:597] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:04.091035 12447 consensus_peers.cc:597] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:04.093021 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:04.095000 13084 consensus_queue.cc:1048] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:04.077361 12329 raft_consensus.cc:1275] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 4 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:29:04.083089 13253 consensus_queue.cc:260] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:04.097157 13088 raft_consensus.cc:3055] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:29:04.098610 12447 leader_election.cc:343] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:04.099264 12447 leader_election.cc:304] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:04.100452 12373 leader_election.cc:304] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.083323 13237 raft_consensus.cc:2804] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:04.104879 12329 raft_consensus.cc:1275] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:04.105973 13157 raft_consensus.cc:1240] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Rejecting Update request from peer a800ae68e91f43a794e326badca3320a for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:04.106269 13253 raft_consensus.cc:3060] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.106784 13233 raft_consensus.cc:2749] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.097676 13088 raft_consensus.cc:740] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:04.109721 13088 consensus_queue.cc:260] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
W20260709 12:29:04.092522 13100 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 120475us
I20260709 12:29:04.110957 13251 raft_consensus.cc:3055] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:04.111159 13100 leader_election.cc:304] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:04.111465 13251 raft_consensus.cc:740] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
W20260709 12:29:04.133260 12447 consensus_peers.cc:597] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:04.139561 12447 consensus_peers.cc:597] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:04.103492 13303 raft_consensus.cc:2804] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:04.127522 13088 raft_consensus.cc:3060] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.157308 12373 leader_election.cc:304] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.130263 12477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:04.130977 13237 raft_consensus.cc:697] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:04.161653 12373 leader_election.cc:304] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.130288 13227 consensus_queue.cc:1048] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:29:04.150938 13323 raft_consensus.cc:2749] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.142072 13233 consensus_queue.cc:1059] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:29:04.175390 13251 consensus_queue.cc:260] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:04.177095 13251 raft_consensus.cc:3060] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.178452 12481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b789be887a492fb0e51b08f2475a97" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
W20260709 12:29:04.156446 13100 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 45409.1us
I20260709 12:29:04.182215 13100 leader_election.cc:304] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:04.182812 13237 consensus_queue.cc:237] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:04.160319 13303 raft_consensus.cc:697] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:04.185531 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:04.186368 13303 consensus_queue.cc:237] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:04.187958 13100 leader_election.cc:304] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.156409 13225 consensus_queue.cc:1048] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
W20260709 12:29:04.195950 13325 log.cc:927] Time spent T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596: Append to log took a long time: real 0.062s	user 0.002s	sys 0.003s
I20260709 12:29:04.169749 13235 raft_consensus.cc:2804] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:04.203943 13235 raft_consensus.cc:697] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:04.173146 13304 raft_consensus.cc:2804] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:04.205371 13273 raft_consensus.cc:2749] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.205334 13235 consensus_queue.cc:237] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:04.206295 13273 raft_consensus.cc:2749] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.207180 13304 raft_consensus.cc:697] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:04.208742 12477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:04.211676 12475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81f0d8651a6144ca9fd21ccc0f57edd9" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:04.213469 13304 consensus_queue.cc:237] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:04.213546 12159 catalog_manager.cc:5697] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } }
W20260709 12:29:04.219810 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 104965us
W20260709 12:29:04.220623 12299 consensus_peers.cc:597] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:04.221921 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 48556.1us
I20260709 12:29:04.226692 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a03f6aed8fcb4ea2b45b6432634c00ae" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:04.231854 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74e672f5fa0b4b68ad22e6fd8002d68d" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:04.255575 13154 raft_consensus.cc:1275] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.256516 13232 raft_consensus.cc:3055] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:04.256994 13232 raft_consensus.cc:740] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:04.257938 12335 raft_consensus.cc:2393] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a800ae68e91f43a794e326badca3320a in current term 2: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:29:04.261548 13232 consensus_queue.cc:260] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
W20260709 12:29:04.234606 13100 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 46732.4us
W20260709 12:29:04.275570 12299 consensus_peers.cc:597] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:04.277534 13311 raft_consensus.cc:515] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:04.278182 13232 raft_consensus.cc:3060] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.278865 13151 raft_consensus.cc:1240] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Rejecting Update request from peer a800ae68e91f43a794e326badca3320a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:29:04.281744 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 52183.8us
I20260709 12:29:04.282935 13311 leader_election.cc:290] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:04.283278 12447 leader_election.cc:304] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:04.285830 13100 leader_election.cc:304] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.287665 12914 raft_consensus.cc:3060] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.294649 12475 raft_consensus.cc:1275] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.295841 13100 leader_election.cc:304] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.296695 13323 raft_consensus.cc:2749] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.298032 13100 leader_election.cc:304] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.294564 13253 consensus_queue.cc:1059] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:29:04.299837 13212 consensus_queue.cc:1048] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.300276 13253 raft_consensus.cc:3055] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:04.300889 13253 raft_consensus.cc:740] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:04.302335 13311 raft_consensus.cc:2749] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.309257 13107 tablet_bootstrap.cc:492] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:04.310869 13323 raft_consensus.cc:2749] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:04.311503 13253 consensus_queue.cc:260] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:04.332617 12914 raft_consensus.cc:2749] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:29:04.334895 12447 consensus_peers.cc:597] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:04.336072 13107 tablet_bootstrap.cc:492] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:04.337177 13107 ts_tablet_manager.cc:1403] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.249s	user 0.017s	sys 0.012s
W20260709 12:29:04.342500 13336 log.cc:927] Time spent T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:29:04.327840 13270 consensus_queue.cc:1048] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 12:29:04.366946 13253 raft_consensus.cc:3060] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:04.381121 13107 raft_consensus.cc:359] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:04.385232 13107 raft_consensus.cc:740] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:04.386761 13107 consensus_queue.cc:260] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
W20260709 12:29:04.385982 12447 consensus_peers.cc:597] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:04.392899 12447 consensus_peers.cc:597] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:04.395095 13107 ts_tablet_manager.cc:1434] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.057s	user 0.005s	sys 0.000s
I20260709 12:29:04.403863 13107 tablet_bootstrap.cc:492] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:04.353309 13100 leader_election.cc:304] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.410265 13311 raft_consensus.cc:2749] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:29:04.424243 13100 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 81054.6us
I20260709 12:29:04.425619 13100 leader_election.cc:304] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.426949 13311 raft_consensus.cc:2749] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:29:04.438611 12299 consensus_peers.cc:597] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:04.451745 12159 catalog_manager.cc:5697] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:04.456892 13100 leader_election.cc:304] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.458034 13311 raft_consensus.cc:2749] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.458214 12159 catalog_manager.cc:5697] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:04.462045 12159 catalog_manager.cc:5697] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:04.466912 12329 raft_consensus.cc:1275] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.473763 13253 consensus_queue.cc:1048] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:04.485355 12328 raft_consensus.cc:3060] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.493999 13100 leader_election.cc:304] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.496270 13323 raft_consensus.cc:2749] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:29:04.500557 12299 consensus_peers.cc:597] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:04.504202 12328 raft_consensus.cc:1275] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.505203 12483 raft_consensus.cc:1275] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.505899 13159 raft_consensus.cc:1275] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:04.512275 13237 consensus_queue.cc:1048] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.513621 13303 consensus_queue.cc:1048] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.525533 13159 raft_consensus.cc:1275] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 4 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:29:04.509716 12154 catalog_manager.cc:5697] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 1 to 2, leader changed from 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194) to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 2 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:04.532989 13159 raft_consensus.cc:1275] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.541162 12475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:04.542309 12475 raft_consensus.cc:2410] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1dcded10caa0493da818e67e13176979 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:29:04.542105 13204 consensus_queue.cc:1048] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:04.546087 12329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:04.547685 12475 raft_consensus.cc:1275] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:29:04.549268 12373 consensus_peers.cc:597] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:04.552954 12158 catalog_manager.cc:5697] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 1 to 2, leader changed from a800ae68e91f43a794e326badca3320a (127.3.89.196) to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:04.549755 13303 consensus_queue.cc:1048] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:04.550431 13237 consensus_queue.cc:1048] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.553884 13225 consensus_queue.cc:1048] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.556208 12158 catalog_manager.cc:5697] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:04.562255 13212 raft_consensus.cc:493] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:04.559484 12158 catalog_manager.cc:5697] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 1 to 2, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:04.562808 13212 raft_consensus.cc:515] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:04.549147 13100 leader_election.cc:304] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.565270 13311 raft_consensus.cc:2749] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.565639 13212 leader_election.cc:290] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:04.566965 12478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:04.567950 12478 raft_consensus.cc:2393] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 in current term 2: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:29:04.574505 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:04.577280 12302 leader_election.cc:304] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.580677 13255 raft_consensus.cc:3060] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.579537 12158 catalog_manager.cc:5697] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 1 to 2, leader changed from a800ae68e91f43a794e326badca3320a (127.3.89.196) to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:04.588960 13255 raft_consensus.cc:2749] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.594502 13100 leader_election.cc:304] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:29:04.622272 13311 raft_consensus.cc:2749] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:29:04.624018 12329 raft_consensus.cc:1275] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.582713 12158 catalog_manager.cc:5697] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 1 to 2, leader changed from 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194) to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:04.609445 12483 raft_consensus.cc:1275] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.628746 12158 catalog_manager.cc:5697] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 1 to 2, leader changed from 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194) to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:04.630527 12158 catalog_manager.cc:5697] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 1 to 2, leader changed from 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194) to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:04.635771 13237 consensus_queue.cc:1048] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:29:04.639536 12373 consensus_peers.cc:597] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:04.651400 12373 consensus_peers.cc:597] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:04.673496 13107 tablet_bootstrap.cc:492] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:04.677683 12483 raft_consensus.cc:1275] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.678082 13107 tablet_bootstrap.cc:492] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:04.678831 13107 ts_tablet_manager.cc:1403] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.275s	user 0.019s	sys 0.025s
I20260709 12:29:04.680809 12483 raft_consensus.cc:1275] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.685145 12475 raft_consensus.cc:3060] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.685976 12483 raft_consensus.cc:3060] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.693046 13107 raft_consensus.cc:359] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:04.693912 13107 raft_consensus.cc:740] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:04.696743 13304 consensus_queue.cc:1048] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.699218 13237 consensus_queue.cc:1048] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.699242 13303 consensus_queue.cc:1048] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:04.703549 12335 raft_consensus.cc:1275] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.703853 12329 raft_consensus.cc:1275] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.704970 13107 consensus_queue.cc:260] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:04.707491 13107 ts_tablet_manager.cc:1434] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 12:29:04.709964 13303 consensus_queue.cc:1048] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:04.712378 12329 raft_consensus.cc:1275] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:29:04.713608 12373 consensus_peers.cc:597] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:04.717726 13107 tablet_bootstrap.cc:492] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:04.722060 13304 consensus_queue.cc:1048] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:04.708935 12475 raft_consensus.cc:1275] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.718776 13225 consensus_queue.cc:1048] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:29:04.756860 12483 raft_consensus.cc:1275] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:04.741920 13237 consensus_queue.cc:1048] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:29:04.768710 13225 consensus_queue.cc:1048] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
W20260709 12:29:04.794435 12373 consensus_peers.cc:597] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:04.826079 13212 raft_consensus.cc:493] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.826706 13212 raft_consensus.cc:515] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
W20260709 12:29:04.913146 13356 log.cc:927] Time spent T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4: Append to log took a long time: real 0.068s	user 0.003s	sys 0.000s
W20260709 12:29:05.045795 13354 log.cc:927] Time spent T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a: Append to log took a long time: real 0.216s	user 0.000s	sys 0.002s
W20260709 12:29:05.052291 12447 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 50600.7us
W20260709 12:29:05.095914 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 137430us
I20260709 12:29:05.088320 13212 leader_election.cc:290] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
W20260709 12:29:05.119474 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 62572us
I20260709 12:29:05.131371 13311 raft_consensus.cc:493] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:05.131969 13311 raft_consensus.cc:515] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.119742 12478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
W20260709 12:29:05.145844 13355 log.cc:927] Time spent T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a: Append to log took a long time: real 0.296s	user 0.004s	sys 0.000s
I20260709 12:29:05.142249 12478 raft_consensus.cc:2468] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:05.162949 12478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:05.167044 12478 raft_consensus.cc:2468] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
W20260709 12:29:05.171316 13360 log.cc:927] Time spent T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a: Append to log took a long time: real 0.345s	user 0.007s	sys 0.000s
I20260709 12:29:05.173884 13232 raft_consensus.cc:493] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:05.175921 12330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:05.164517 13311 leader_election.cc:290] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:05.190881 13232 raft_consensus.cc:515] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:05.194311 13100 leader_election.cc:304] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:05.196516 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:05.197129 12410 raft_consensus.cc:2468] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
I20260709 12:29:05.211154 13232 leader_election.cc:290] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:05.212527 13311 raft_consensus.cc:2804] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:05.216924 12330 raft_consensus.cc:2393] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1dcded10caa0493da818e67e13176979 in current term 2: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:29:05.226814 12299 leader_election.cc:304] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:05.218212 13311 raft_consensus.cc:493] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:05.239298 13368 raft_consensus.cc:2804] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:05.239815 13368 raft_consensus.cc:493] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:05.240145 13368 raft_consensus.cc:3060] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.228327 13371 raft_consensus.cc:493] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:05.254402 13371 raft_consensus.cc:515] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.251821 13311 raft_consensus.cc:3060] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:05.301915 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 75292.5us
I20260709 12:29:05.304293 13107 tablet_bootstrap.cc:492] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:29:05.305879 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 178162us
I20260709 12:29:05.300807 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:05.311928 13107 tablet_bootstrap.cc:492] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:05.312690 13107 ts_tablet_manager.cc:1403] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.595s	user 0.012s	sys 0.028s
I20260709 12:29:05.327312 12405 raft_consensus.cc:2393] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a800ae68e91f43a794e326badca3320a in current term 2: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:29:05.335224 13204 raft_consensus.cc:493] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.335805 13204 raft_consensus.cc:515] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.337971 13204 leader_election.cc:290] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:29:05.309636 13100 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:33269, user_credentials={real_user=slave}} blocked reactor thread for 43535.6us
I20260709 12:29:05.335172 13311 raft_consensus.cc:515] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.344529 13311 leader_election.cc:290] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:05.347185 13371 leader_election.cc:290] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:05.347879 12329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:05.353020 13157 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:05.348690 12329 raft_consensus.cc:2468] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2.
I20260709 12:29:05.353864 13157 raft_consensus.cc:2468] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 1.
I20260709 12:29:05.356523 12447 leader_election.cc:304] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:05.358260 13295 raft_consensus.cc:2804] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:05.358667 13295 raft_consensus.cc:493] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:05.359040 12477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:05.359045 13295 raft_consensus.cc:3060] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.361804 13378 raft_consensus.cc:493] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.362299 13378 raft_consensus.cc:515] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:05.367978 13378 leader_election.cc:290] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:05.369659 13237 raft_consensus.cc:493] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.370258 13237 raft_consensus.cc:515] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:05.373145 13237 leader_election.cc:290] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:05.382541 13107 raft_consensus.cc:359] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:05.393415 13107 raft_consensus.cc:740] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.394384 13107 consensus_queue.cc:260] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:05.390245 13368 raft_consensus.cc:515] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:05.385409 13295 raft_consensus.cc:515] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.407011 13295 leader_election.cc:290] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:05.407377 13368 leader_election.cc:290] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:05.423517 13107 ts_tablet_manager.cc:1434] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.111s	user 0.008s	sys 0.000s
I20260709 12:29:05.425024 13107 tablet_bootstrap.cc:492] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:05.408331 13151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:05.423298 13303 raft_consensus.cc:493] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
W20260709 12:29:05.433444 12302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 158666us
I20260709 12:29:05.422533 13295 raft_consensus.cc:493] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:05.435712 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 116114us
I20260709 12:29:05.440663 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:05.441504 12410 raft_consensus.cc:2468] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
I20260709 12:29:05.442668 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:05.443427 12409 raft_consensus.cc:3060] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.408834 12477 raft_consensus.cc:2468] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:05.434371 13151 raft_consensus.cc:3060] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.469313 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db639fdafc34f269b3d493540075616" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:05.470029 12484 raft_consensus.cc:3060] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.475976 12482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:05.476639 12482 raft_consensus.cc:2468] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 1.
I20260709 12:29:05.489639 12484 raft_consensus.cc:2468] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 3.
I20260709 12:29:05.447536 13295 raft_consensus.cc:515] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.435350 13303 raft_consensus.cc:515] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:05.498538 12447 leader_election.cc:304] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:05.503688 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:05.487028 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:05.510517 12483 raft_consensus.cc:2468] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
I20260709 12:29:05.503312 13232 raft_consensus.cc:2804] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:05.511376 13232 raft_consensus.cc:493] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e02e5f693ce49a3b161ad90848be7d4)
I20260709 12:29:05.511752 13232 raft_consensus.cc:3060] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:05.512789 12334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:05.513440 12334 raft_consensus.cc:2468] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:29:05.519243 13303 leader_election.cc:290] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:05.521337 13156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:05.522534 13156 raft_consensus.cc:2393] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 in current term 2: Already voted for candidate 1dcded10caa0493da818e67e13176979 in this term.
I20260709 12:29:05.509871 12302 leader_election.cc:304] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:05.527894 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:05.528564 12483 raft_consensus.cc:2468] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:29:05.532055 12329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:05.532876 12329 raft_consensus.cc:2393] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1dcded10caa0493da818e67e13176979 in current term 2: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:29:05.538491 12409 raft_consensus.cc:2468] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 3.
I20260709 12:29:05.528736 12299 leader_election.cc:304] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:05.540143 12373 leader_election.cc:304] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:05.546527 12299 leader_election.cc:304] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:05.530043 12327 raft_consensus.cc:2468] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2.
I20260709 12:29:05.545753 13371 raft_consensus.cc:2804] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:05.559342 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:05.545337 13332 raft_consensus.cc:2804] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:05.557327 12373 leader_election.cc:304] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:05.549912 13373 raft_consensus.cc:2804] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:05.567433 13332 raft_consensus.cc:493] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:05.567891 13369 raft_consensus.cc:2804] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:05.568661 12335 raft_consensus.cc:2393] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2104c8a9b144a419ccb7c07e3f02596 in current term 2: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:29:05.570313 13332 raft_consensus.cc:3060] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:05.565322 13212 raft_consensus.cc:493] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.571602 13212 raft_consensus.cc:515] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.556820 13366 raft_consensus.cc:2804] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:05.579638 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:05.580343 12484 raft_consensus.cc:2468] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
I20260709 12:29:05.581444 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:05.581943 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b87f00ff8c4e1c858037069e3bbb6e" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:05.582201 13159 raft_consensus.cc:2468] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 1.
I20260709 12:29:05.576957 13366 raft_consensus.cc:493] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:05.582582 12332 raft_consensus.cc:3060] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.582937 13366 raft_consensus.cc:3060] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:05.566192 13371 raft_consensus.cc:493] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:05.584385 13371 raft_consensus.cc:3060] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.570566 13369 raft_consensus.cc:493] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:05.586355 13369 raft_consensus.cc:3060] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.582531 13295 leader_election.cc:290] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:05.578625 13373 raft_consensus.cc:697] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:05.591298 13373 consensus_queue.cc:237] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:05.595660 13212 leader_election.cc:290] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:05.596290 12332 raft_consensus.cc:2468] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 3.
I20260709 12:29:05.598975 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:05.599624 13159 raft_consensus.cc:2468] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
W20260709 12:29:05.599848 12299 consensus_peers.cc:597] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:05.605724 12447 leader_election.cc:304] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:05.606702 13387 raft_consensus.cc:2804] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:05.600729 12299 leader_election.cc:304] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:05.611683 13212 raft_consensus.cc:2804] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:05.607240 13387 raft_consensus.cc:697] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 3 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:05.616910 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:05.620000 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:05.630514 13159 raft_consensus.cc:2468] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2.
I20260709 12:29:05.633488 12447 leader_election.cc:304] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:05.623998 13212 raft_consensus.cc:493] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:05.637733 13212 raft_consensus.cc:3060] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.636718 13295 raft_consensus.cc:2804] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:05.624904 13387 consensus_queue.cc:237] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.637259 13232 raft_consensus.cc:515] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:05.649525 13366 raft_consensus.cc:515] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.649560 13295 raft_consensus.cc:493] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:05.668054 13295 raft_consensus.cc:3060] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.675761 13332 raft_consensus.cc:515] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:05.679060 13295 raft_consensus.cc:515] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.680428 13100 leader_election.cc:304] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:05.681504 13311 raft_consensus.cc:2804] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:05.681882 12332 raft_consensus.cc:2468] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2.
I20260709 12:29:05.682034 13311 raft_consensus.cc:697] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:05.682922 13366 leader_election.cc:290] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:05.683187 13150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:05.683841 13150 raft_consensus.cc:3060] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.683997 13332 leader_election.cc:290] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:05.684381 13295 leader_election.cc:290] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:05.686966 13371 raft_consensus.cc:515] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:05.689134 13371 leader_election.cc:290] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:05.693286 13212 raft_consensus.cc:515] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.683007 13311 consensus_queue.cc:237] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.695556 13212 leader_election.cc:290] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:05.699570 13369 raft_consensus.cc:515] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:05.703774 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:05.704501 12483 raft_consensus.cc:2368] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 2. Current term is 3.
I20260709 12:29:05.709136 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:05.711711 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:05.712530 12409 raft_consensus.cc:2393] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a800ae68e91f43a794e326badca3320a in current term 2: Already voted for candidate a2104c8a9b144a419ccb7c07e3f02596 in this term.
I20260709 12:29:05.715185 12329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:05.715504 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:05.705606 12405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d63f43ed8a4bedb35aa02c779fe4d0" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:05.716418 12479 raft_consensus.cc:3060] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.716466 12405 raft_consensus.cc:3060] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.718286 13159 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:05.716004 12329 raft_consensus.cc:2393] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a2104c8a9b144a419ccb7c07e3f02596 in current term 2: Already voted for candidate 4e02e5f693ce49a3b161ad90848be7d4 in this term.
I20260709 12:29:05.725245 13378 raft_consensus.cc:493] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:05.713532 13232 leader_election.cc:290] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:05.725245 12483 raft_consensus.cc:2468] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
I20260709 12:29:05.706238 13150 raft_consensus.cc:2468] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 3.
I20260709 12:29:05.726367 13378 raft_consensus.cc:515] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
W20260709 12:29:05.730895 12373 consensus_peers.cc:597] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:05.734907 13378 leader_election.cc:290] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:05.737213 13154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:05.732048 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab18688981ff456ca286ec171980c8de" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:05.739323 12483 raft_consensus.cc:3060] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.741616 12329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:05.744189 12158 catalog_manager.cc:5697] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:05.755673 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:05.756423 12484 raft_consensus.cc:2468] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 3.
I20260709 12:29:05.726357 13151 raft_consensus.cc:2468] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 3.
I20260709 12:29:05.751513 13369 leader_election.cc:290] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 election: Requested vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:05.760236 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228aa7f69257458ba0c2b5e6c950f217" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:05.760804 13150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979" is_pre_election: true
I20260709 12:29:05.761678 13150 raft_consensus.cc:2393] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 in current term 3: Already voted for candidate 1dcded10caa0493da818e67e13176979 in this term.
I20260709 12:29:05.760855 12332 raft_consensus.cc:3060] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.759357 12299 leader_election.cc:400] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Vote denied by peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677) with higher term. Message: Invalid argument: T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 2. Current term is 3.
I20260709 12:29:05.763252 12299 leader_election.cc:403] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:29:05.763572 12479 raft_consensus.cc:2468] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 3.
I20260709 12:29:05.752910 12447 leader_election.cc:304] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:05.765079 13295 raft_consensus.cc:2804] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:05.765776 13295 raft_consensus.cc:697] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 3 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:05.766618 13295 consensus_queue.cc:237] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.770025 13150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:05.770820 13150 raft_consensus.cc:3060] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.767581 12329 raft_consensus.cc:2468] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2.
I20260709 12:29:05.774353 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:05.759644 12405 raft_consensus.cc:2468] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 3.
I20260709 12:29:05.780227 12302 leader_election.cc:304] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:05.784597 12483 raft_consensus.cc:2468] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 3.
I20260709 12:29:05.771932 12299 leader_election.cc:304] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:05.789409 12332 raft_consensus.cc:2468] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 3.
I20260709 12:29:05.790653 12154 catalog_manager.cc:5697] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 1 to 2, leader changed from a800ae68e91f43a794e326badca3320a (127.3.89.196) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:05.792903 12299 leader_election.cc:304] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:05.782420 12335 raft_consensus.cc:3060] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.791419 13373 raft_consensus.cc:2749] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer a800ae68e91f43a794e326badca3320a (127.3.89.196:45677) with higher term. Message: Invalid argument: T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 2. Current term is 3.
I20260709 12:29:05.797277 13368 raft_consensus.cc:2804] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:05.791589 13212 raft_consensus.cc:2804] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:05.798538 13212 raft_consensus.cc:493] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:05.799119 13212 raft_consensus.cc:3060] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.801529 13107 tablet_bootstrap.cc:492] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.783157 13371 raft_consensus.cc:2804] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:05.796480 12373 leader_election.cc:304] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:05.807614 13371 raft_consensus.cc:697] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:05.807868 13374 raft_consensus.cc:2804] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:05.808393 13374 raft_consensus.cc:697] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:05.808547 13371 consensus_queue.cc:237] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:05.809306 13374 consensus_queue.cc:237] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:05.804739 13107 tablet_bootstrap.cc:492] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:05.804976 13368 raft_consensus.cc:697] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:05.815685 13368 consensus_queue.cc:237] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.812945 12447 leader_election.cc:304] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:05.813728 13107 ts_tablet_manager.cc:1403] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.389s	user 0.032s	sys 0.012s
I20260709 12:29:05.825399 13362 raft_consensus.cc:2804] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:05.828750 13362 raft_consensus.cc:697] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:05.830073 13107 raft_consensus.cc:359] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:05.832530 13107 raft_consensus.cc:740] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.833693 13107 consensus_queue.cc:260] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
W20260709 12:29:05.850498 12299 consensus_peers.cc:597] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:05.854086 13362 consensus_queue.cc:237] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:05.859460 13107 ts_tablet_manager.cc:1434] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 12:29:05.876411 13107 tablet_bootstrap.cc:492] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:05.873383 13212 raft_consensus.cc:515] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:05.916918 13150 raft_consensus.cc:2468] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 3.
I20260709 12:29:05.918902 13212 leader_election.cc:290] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:05.913251 12335 raft_consensus.cc:2468] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 3.
I20260709 12:29:05.926987 12159 catalog_manager.cc:5697] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:05.932441 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:05.940982 13150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1dcded10caa0493da818e67e13176979"
I20260709 12:29:05.941478 13231 raft_consensus.cc:493] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.942045 13150 raft_consensus.cc:2393] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 in current term 3: Already voted for candidate 1dcded10caa0493da818e67e13176979 in this term.
I20260709 12:29:05.942261 13231 raft_consensus.cc:515] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:05.945766 13231 leader_election.cc:290] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:05.951107 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
W20260709 12:29:05.951546 13398 log.cc:927] Time spent T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:29:05.955786 12479 raft_consensus.cc:2468] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 3.
I20260709 12:29:05.965173 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:05.966128 12479 raft_consensus.cc:2468] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 2.
I20260709 12:29:05.969641 12299 leader_election.cc:304] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:29:05.975688 12373 leader_election.cc:304] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:05.978698 13332 raft_consensus.cc:2804] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:05.981678 13332 raft_consensus.cc:697] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:05.982867 13332 consensus_queue.cc:237] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:05.988292 12299 leader_election.cc:304] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:05.990338 13212 raft_consensus.cc:2804] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:05.991134 13212 raft_consensus.cc:493] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:05.991484 13212 raft_consensus.cc:3060] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.989472 13382 raft_consensus.cc:2804] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:05.998029 13382 raft_consensus.cc:697] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:05.998924 13382 consensus_queue.cc:237] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:06.029214 13212 raft_consensus.cc:515] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
W20260709 12:29:06.038743 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 50584.2us
I20260709 12:29:06.051434 12479 raft_consensus.cc:1275] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:06.060154 12409 raft_consensus.cc:1275] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:06.062635 12409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:06.067543 12154 catalog_manager.cc:5697] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 1 to 2, leader changed from a800ae68e91f43a794e326badca3320a (127.3.89.196) to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.061771 12158 catalog_manager.cc:5697] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 2 to 3, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 3 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:06.072919 12158 catalog_manager.cc:5697] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.075305 12158 catalog_manager.cc:5697] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 2 to 3, leader changed from a800ae68e91f43a794e326badca3320a (127.3.89.196) to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 3 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:06.077922 13212 leader_election.cc:290] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:06.071755 13368 consensus_queue.cc:1048] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:29:06.086659 13371 consensus_queue.cc:1048] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:06.096670 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6682a011b3945c59d105bb6951d62f3" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:06.097421 12483 raft_consensus.cc:3060] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:06.105227 12483 raft_consensus.cc:2468] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 3.
I20260709 12:29:06.108551 12299 leader_election.cc:304] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:06.110837 13368 raft_consensus.cc:2804] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:06.112182 13368 raft_consensus.cc:697] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:06.113119 13368 consensus_queue.cc:237] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:06.103595 12159 catalog_manager.cc:5697] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.117915 12159 catalog_manager.cc:5697] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 1 to 2, leader changed from 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194) to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 2 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:06.123088 13107 tablet_bootstrap.cc:492] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.124217 13107 tablet_bootstrap.cc:492] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:06.124929 13107 ts_tablet_manager.cc:1403] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.249s	user 0.019s	sys 0.007s
I20260709 12:29:06.147595 12158 catalog_manager.cc:5697] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 2 to 3, leader changed from a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195) to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 3 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.172331 13107 raft_consensus.cc:359] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:06.173125 13107 raft_consensus.cc:740] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.173964 13107 consensus_queue.cc:260] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:06.176245 13107 ts_tablet_manager.cc:1434] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 12:29:06.177078 13107 tablet_bootstrap.cc:492] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:06.184885 12154 catalog_manager.cc:5697] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:06.192746 13154 raft_consensus.cc:1275] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:29:06.196770 12299 consensus_peers.cc:597] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:06.203186 13386 consensus_queue.cc:1048] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:06.208882 12447 consensus_peers.cc:597] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:06.232300 12479 raft_consensus.cc:1240] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Rejecting Update request from peer a2104c8a9b144a419ccb7c07e3f02596 for earlier term 2. Current term is 3. Ops: []
I20260709 12:29:06.235102 13374 consensus_queue.cc:1059] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:06.236337 13311 raft_consensus.cc:493] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.236903 13311 raft_consensus.cc:515] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:06.239040 13311 leader_election.cc:290] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:06.241968 12329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:06.242606 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a1cc112d1984521b5479f66b4289cce" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:06.243777 13304 raft_consensus.cc:3055] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:06.243724 12479 raft_consensus.cc:2410] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1dcded10caa0493da818e67e13176979 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:29:06.244215 13304 raft_consensus.cc:740] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
W20260709 12:29:06.252111 12447 consensus_peers.cc:597] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:06.245164 13304 consensus_queue.cc:260] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:06.253271 13304 raft_consensus.cc:3060] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:06.256582 13100 leader_election.cc:304] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1dcded10caa0493da818e67e13176979; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:29:06.259035 13311 raft_consensus.cc:2749] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:06.263082 13156 raft_consensus.cc:1275] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:06.264676 13386 consensus_queue.cc:1048] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:06.300129 12335 raft_consensus.cc:1240] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 LEADER]: Rejecting Update request from peer 1dcded10caa0493da818e67e13176979 for earlier term 2. Current term is 3. Ops: []
I20260709 12:29:06.301599 13311 consensus_queue.cc:1059] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:06.302340 13311 consensus_queue.cc:1243] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [LEADER]: Peer 4e02e5f693ce49a3b161ad90848be7d4 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:29:06.302805 13311 raft_consensus.cc:3055] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:06.303143 13311 raft_consensus.cc:740] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:06.305261 12329 raft_consensus.cc:1275] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:06.304046 13311 consensus_queue.cc:260] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:06.307301 13311 raft_consensus.cc:3060] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:06.318907 12335 raft_consensus.cc:1275] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:06.323662 13332 raft_consensus.cc:493] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:06.324294 13332 raft_consensus.cc:515] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:06.326547 13332 leader_election.cc:290] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
W20260709 12:29:06.332985 12447 consensus_peers.cc:597] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:06.344998 13386 consensus_queue.cc:1048] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:06.345444 13362 consensus_queue.cc:1048] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:06.355203 12479 raft_consensus.cc:1275] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:06.356635 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:06.362586 12335 raft_consensus.cc:2468] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 2.
W20260709 12:29:06.363839 12447 consensus_peers.cc:597] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:06.367897 12373 leader_election.cc:304] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:06.372707 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:06.371104 13371 consensus_queue.cc:1048] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:29:06.392230 12479 raft_consensus.cc:1275] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:29:06.396755 12447 consensus_peers.cc:597] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:06.398145 12447 consensus_peers.cc:597] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:06.371944 13304 raft_consensus.cc:2804] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:06.400861 13304 raft_consensus.cc:493] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:06.404438 13304 raft_consensus.cc:3060] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:06.405416 13212 consensus_queue.cc:1048] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:29:06.409801 13154 raft_consensus.cc:1275] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:06.418339 12327 raft_consensus.cc:1275] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:06.418869 13107 tablet_bootstrap.cc:492] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.420930 13107 tablet_bootstrap.cc:492] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:06.421641 13107 ts_tablet_manager.cc:1403] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.245s	user 0.036s	sys 0.069s
I20260709 12:29:06.424232 13107 raft_consensus.cc:359] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:06.424911 13107 raft_consensus.cc:740] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.425626 13107 consensus_queue.cc:260] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:06.427902 13107 ts_tablet_manager.cc:1434] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:06.432538 13107 tablet_bootstrap.cc:492] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:06.440091 13253 consensus_queue.cc:1048] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:29:06.455560 13160 raft_consensus.cc:1275] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:29:06.440893 13368 consensus_queue.cc:1048] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.015s
W20260709 12:29:06.459370 12299 consensus_peers.cc:597] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:06.459772 12408 raft_consensus.cc:1275] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:06.456053 13160 pending_rounds.cc:85] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: Aborting all ops after (but not including) 1
I20260709 12:29:06.478839 13368 consensus_queue.cc:1048] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:06.481110 12329 raft_consensus.cc:1275] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:06.477231 13160 pending_rounds.cc:107] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:29:06.507440 13160 raft_consensus.cc:2889] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:06.520905 13304 raft_consensus.cc:515] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:06.523088 13304 leader_election.cc:290] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:06.496086 13111 consensus_queue.cc:1048] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:29:06.527333 12409 raft_consensus.cc:1240] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Rejecting Update request from peer a800ae68e91f43a794e326badca3320a for earlier term 2. Current term is 3. Ops: []
I20260709 12:29:06.539579 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:06.540854 13400 consensus_queue.cc:1059] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:29:06.549677 13361 raft_consensus.cc:3055] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:06.554497 13361 raft_consensus.cc:740] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:06.555778 13361 consensus_queue.cc:260] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:06.550885 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:06.557148 13361 raft_consensus.cc:3060] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:06.573057 12408 raft_consensus.cc:1275] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:29:06.578127 12408 pending_rounds.cc:85] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596: Aborting all ops after (but not including) 1
W20260709 12:29:06.561793 12299 consensus_peers.cc:597] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:06.578526 12408 pending_rounds.cc:107] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:29:06.562755 12479 raft_consensus.cc:1275] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:06.578984 12408 raft_consensus.cc:2889] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:06.580812 12373 leader_election.cc:304] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a
I20260709 12:29:06.584937 13212 consensus_queue.cc:1048] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:06.586814 13375 raft_consensus.cc:2749] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:29:06.589959 13231 consensus_queue.cc:1048] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:06.590973 12479 raft_consensus.cc:1275] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:06.624702 13160 raft_consensus.cc:1275] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Refusing update from remote peer a2104c8a9b144a419ccb7c07e3f02596: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:06.557389 13373 consensus_queue.cc:1048] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 12:29:06.630537 13375 consensus_queue.cc:1048] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:06.629158 13107 tablet_bootstrap.cc:492] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.634810 13107 tablet_bootstrap.cc:492] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:06.637326 13107 ts_tablet_manager.cc:1403] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.205s	user 0.018s	sys 0.029s
I20260709 12:29:06.637758 13424 consensus_queue.cc:1048] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:06.729949 13107 raft_consensus.cc:359] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:06.730888 13107 raft_consensus.cc:740] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.732976 13107 consensus_queue.cc:260] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:06.735122 13107 ts_tablet_manager.cc:1434] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.097s	user 0.008s	sys 0.000s
W20260709 12:29:06.745647 12302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 73639us
W20260709 12:29:06.747076 12299 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.196:45677, user_credentials={real_user=slave}} blocked reactor thread for 66335.5us
W20260709 12:29:06.746272 13432 log.cc:927] Time spent T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:29:06.767108 13107 tablet_bootstrap.cc:492] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
W20260709 12:29:06.808585 12373 consensus_peers.cc:597] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:06.817155 13151 raft_consensus.cc:3060] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:06.871652 13107 tablet_bootstrap.cc:492] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.872748 13107 tablet_bootstrap.cc:492] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:06.873535 13107 ts_tablet_manager.cc:1403] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.112s	user 0.014s	sys 0.003s
I20260709 12:29:06.876132 13107 raft_consensus.cc:359] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:06.876677 13107 raft_consensus.cc:740] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.877295 13107 consensus_queue.cc:260] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:06.885329 13107 ts_tablet_manager.cc:1434] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.002s
I20260709 12:29:06.889839 13107 tablet_bootstrap.cc:492] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:06.934630 13107 tablet_bootstrap.cc:492] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.935737 13107 tablet_bootstrap.cc:492] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:06.935748 13151 raft_consensus.cc:3060] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:06.936806 13107 ts_tablet_manager.cc:1403] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.047s	user 0.023s	sys 0.004s
I20260709 12:29:06.939807 13107 raft_consensus.cc:359] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:06.941429 13107 raft_consensus.cc:740] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.942281 13107 consensus_queue.cc:260] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:06.944568 13107 ts_tablet_manager.cc:1434] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:29:06.945467 13107 tablet_bootstrap.cc:492] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
W20260709 12:29:06.949226 12447 consensus_peers.cc:597] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:06.949542 12299 consensus_peers.cc:597] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:06.968556 13107 tablet_bootstrap.cc:492] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.969743 13107 tablet_bootstrap.cc:492] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:06.970587 13107 ts_tablet_manager.cc:1403] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.006s
I20260709 12:29:06.973234 13107 raft_consensus.cc:359] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:06.973986 13107 raft_consensus.cc:740] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.974722 13107 consensus_queue.cc:260] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:06.977254 13107 ts_tablet_manager.cc:1434] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:29:06.979702 13107 tablet_bootstrap.cc:492] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:06.998490 13107 tablet_bootstrap.cc:492] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.999615 13107 tablet_bootstrap.cc:492] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:07.000433 13107 ts_tablet_manager.cc:1403] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.000s
I20260709 12:29:07.003181 13107 raft_consensus.cc:359] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.003918 13107 raft_consensus.cc:740] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:07.004594 13107 consensus_queue.cc:260] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.007172 13107 ts_tablet_manager.cc:1434] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:29:07.008070 13107 tablet_bootstrap.cc:492] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:07.026913 13107 tablet_bootstrap.cc:492] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:07.027937 13107 tablet_bootstrap.cc:492] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:07.028700 13107 ts_tablet_manager.cc:1403] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.003s
I20260709 12:29:07.031610 13107 raft_consensus.cc:359] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:07.032348 13107 raft_consensus.cc:740] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:07.033197 13107 consensus_queue.cc:260] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:07.035789 13107 ts_tablet_manager.cc:1434] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:29:07.037355 13107 tablet_bootstrap.cc:492] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
W20260709 12:29:07.064002 12373 consensus_peers.cc:597] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:07.064538 13107 tablet_bootstrap.cc:492] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:07.067397 13107 tablet_bootstrap.cc:492] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:07.068217 13107 ts_tablet_manager.cc:1403] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.008s
I20260709 12:29:07.071321 13107 raft_consensus.cc:359] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.072085 13107 raft_consensus.cc:740] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:07.073048 13107 consensus_queue.cc:260] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.075796 13107 ts_tablet_manager.cc:1434] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:07.077389 13107 tablet_bootstrap.cc:492] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
W20260709 12:29:07.262732 12373 consensus_peers.cc:597] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:07.331198 13160 raft_consensus.cc:3060] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.356539 13154 raft_consensus.cc:3060] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.377806 13107 tablet_bootstrap.cc:492] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:07.382359 13107 tablet_bootstrap.cc:492] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:07.393647 13107 ts_tablet_manager.cc:1403] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.316s	user 0.071s	sys 0.078s
I20260709 12:29:07.399004 13107 raft_consensus.cc:359] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.407020 13107 raft_consensus.cc:740] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:07.416115 13107 consensus_queue.cc:260] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.418412 13107 ts_tablet_manager.cc:1434] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:29:07.420936 13107 tablet_bootstrap.cc:492] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
W20260709 12:29:07.442976 12373 consensus_peers.cc:597] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:07.461812 13151 raft_consensus.cc:3060] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.496265 13107 tablet_bootstrap.cc:492] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:07.497310 13107 tablet_bootstrap.cc:492] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:07.498237 13107 ts_tablet_manager.cc:1403] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.078s	user 0.025s	sys 0.005s
I20260709 12:29:07.501551 13107 raft_consensus.cc:359] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.502348 13107 raft_consensus.cc:740] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:07.503103 13107 consensus_queue.cc:260] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.504931 13107 ts_tablet_manager.cc:1434] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:07.506023 13107 tablet_bootstrap.cc:492] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:07.534888 13107 tablet_bootstrap.cc:492] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:07.535890 13107 tablet_bootstrap.cc:492] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:07.536573 13107 ts_tablet_manager.cc:1403] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.014s
I20260709 12:29:07.539633 13107 raft_consensus.cc:359] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:07.540328 13107 raft_consensus.cc:740] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:07.541059 13107 consensus_queue.cc:260] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:07.544206 13107 ts_tablet_manager.cc:1434] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.001s
I20260709 12:29:07.548804 13107 tablet_bootstrap.cc:492] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:07.588644 13107 tablet_bootstrap.cc:492] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:07.589742 13107 tablet_bootstrap.cc:492] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:07.590469 13107 ts_tablet_manager.cc:1403] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.046s	user 0.021s	sys 0.003s
I20260709 12:29:07.600162 13107 raft_consensus.cc:359] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.600898 13107 raft_consensus.cc:740] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:07.601816 13107 consensus_queue.cc:260] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.611490 13107 ts_tablet_manager.cc:1434] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:29:07.612705 13107 tablet_bootstrap.cc:492] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:07.623093 13154 raft_consensus.cc:3060] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.650321 13107 tablet_bootstrap.cc:492] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:07.651399 13107 tablet_bootstrap.cc:492] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:07.652211 13107 ts_tablet_manager.cc:1403] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.007s
I20260709 12:29:07.655555 13107 raft_consensus.cc:359] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:07.656471 13107 raft_consensus.cc:740] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:07.657603 13107 consensus_queue.cc:260] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:07.660444 13107 ts_tablet_manager.cc:1434] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 12:29:07.661562 13107 tablet_bootstrap.cc:492] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:07.680408 13107 tablet_bootstrap.cc:492] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:07.681465 13107 tablet_bootstrap.cc:492] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:07.682273 13107 ts_tablet_manager.cc:1403] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.021s	user 0.020s	sys 0.000s
I20260709 12:29:07.685179 13107 raft_consensus.cc:359] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.685952 13107 raft_consensus.cc:740] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:07.686728 13107 consensus_queue.cc:260] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.689379 13107 ts_tablet_manager.cc:1434] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:29:07.690757 13107 tablet_bootstrap.cc:492] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:07.714967 13154 raft_consensus.cc:3060] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.732594 13107 tablet_bootstrap.cc:492] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:07.733788 13107 tablet_bootstrap.cc:492] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:07.734699 13107 ts_tablet_manager.cc:1403] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.044s	user 0.019s	sys 0.017s
I20260709 12:29:07.735047 13295 raft_consensus.cc:493] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:07.735796 13295 raft_consensus.cc:515] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:07.738111 13107 raft_consensus.cc:359] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.738759 13295 leader_election.cc:290] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:07.738991 13107 raft_consensus.cc:740] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:07.740113 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:07.741588 12408 raft_consensus.cc:2468] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 3.
I20260709 12:29:07.750108 13107 consensus_queue.cc:260] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.754061 12450 leader_election.cc:304] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:07.755473 13362 raft_consensus.cc:2804] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:07.755940 13362 raft_consensus.cc:493] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:07.756325 13362 raft_consensus.cc:3060] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:07.774121 13107 ts_tablet_manager.cc:1434] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.001s
I20260709 12:29:07.775859 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4" is_pre_election: true
I20260709 12:29:07.794382 13107 tablet_bootstrap.cc:492] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:07.799654 13151 raft_consensus.cc:3060] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.812527 13362 raft_consensus.cc:515] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:07.816922 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:07.818655 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:07.819358 12408 raft_consensus.cc:3060] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:07.825819 13362 leader_election.cc:290] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:07.843418 13107 tablet_bootstrap.cc:492] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:07.844493 13107 tablet_bootstrap.cc:492] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:07.845156 13107 ts_tablet_manager.cc:1403] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.000s
I20260709 12:29:07.846302 12408 raft_consensus.cc:2468] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 4.
I20260709 12:29:07.848004 12450 leader_election.cc:304] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:07.849275 13295 raft_consensus.cc:2804] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:07.849870 13295 raft_consensus.cc:697] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 4 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:07.850862 13295 consensus_queue.cc:237] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:07.847992 13107 raft_consensus.cc:359] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.858944 13107 raft_consensus.cc:740] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:07.859824 13107 consensus_queue.cc:260] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.862629 13107 ts_tablet_manager.cc:1434] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.005s
I20260709 12:29:07.863502 13107 tablet_bootstrap.cc:492] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:07.868214 12158 catalog_manager.cc:5697] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.894234 13107 tablet_bootstrap.cc:492] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:07.895483 13107 tablet_bootstrap.cc:492] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:07.896174 13107 ts_tablet_manager.cc:1403] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.033s	user 0.017s	sys 0.008s
I20260709 12:29:07.898799 13107 raft_consensus.cc:359] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:07.899485 13107 raft_consensus.cc:740] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:07.900279 13107 consensus_queue.cc:260] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:07.902577 13107 ts_tablet_manager.cc:1434] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:29:07.906852 13107 tablet_bootstrap.cc:492] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979: Bootstrap starting.
I20260709 12:29:07.909883 13151 raft_consensus.cc:3060] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:07.948704 13107 tablet_bootstrap.cc:492] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:07.949772 13107 tablet_bootstrap.cc:492] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979: Bootstrap complete.
I20260709 12:29:07.950560 13107 ts_tablet_manager.cc:1403] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979: Time spent bootstrapping tablet: real 0.044s	user 0.009s	sys 0.013s
I20260709 12:29:07.954167 13107 raft_consensus.cc:359] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.954921 13107 raft_consensus.cc:740] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcded10caa0493da818e67e13176979, State: Initialized, Role: FOLLOWER
I20260709 12:29:07.955740 13107 consensus_queue.cc:260] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:07.957962 13107 ts_tablet_manager.cc:1434] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:29:07.998368 13364 raft_consensus.cc:493] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a800ae68e91f43a794e326badca3320a)
I20260709 12:29:07.998884 13364 raft_consensus.cc:515] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:08.000597 13364 leader_election.cc:290] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:08.001536 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:08.001803 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "436db7228d9d4e92a8472906005ae3fb" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:08.002561 12408 raft_consensus.cc:2368] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 3. Current term is 4.
I20260709 12:29:08.004056 12302 leader_election.cc:400] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Vote denied by peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155) with higher term. Message: Invalid argument: T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 3. Current term is 4.
I20260709 12:29:08.004369 12302 leader_election.cc:403] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:08.005060 13364 raft_consensus.cc:3060] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:29:08.014772 13364 raft_consensus.cc:2749] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155) with higher term. Message: Invalid argument: T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e02e5f693ce49a3b161ad90848be7d4 for earlier term 3. Current term is 4.
I20260709 12:29:08.027515 13151 raft_consensus.cc:3060] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:08.234723 12327 raft_consensus.cc:1275] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 4 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:08.236433 13361 consensus_queue.cc:1048] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:08.270337 12408 raft_consensus.cc:1275] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:08.272495 13361 consensus_queue.cc:1048] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:11.508136  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:11.594974 12479 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92"
dest_uuid: "a800ae68e91f43a794e326badca3320a"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:42482
I20260709 12:29:11.595804 12479 raft_consensus.cc:606] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:11.621210 12410 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681"
dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
mode: GRACEFUL
new_leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
 from {username='slave'} at 127.0.0.1:46500
I20260709 12:29:11.621923 12410 raft_consensus.cc:606] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Received request to transfer leadership to TS 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:11.645898 12329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ff454851114445da8be05eda167efc4"
dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:48908
I20260709 12:29:11.646775 12329 raft_consensus.cc:606] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:11.650137 12327 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a"
dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:48908
I20260709 12:29:11.650671 12327 raft_consensus.cc:606] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:11.653208 12327 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2a2e0c681f24264a729f89d08042fd9"
dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
mode: GRACEFUL
new_leader_uuid: "a800ae68e91f43a794e326badca3320a"
 from {username='slave'} at 127.0.0.1:48908
I20260709 12:29:11.653718 12327 raft_consensus.cc:606] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Received request to transfer leadership to TS a800ae68e91f43a794e326badca3320a
I20260709 12:29:11.656106 12327 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd19eddb6c2e45b7acefc5c295d92928"
dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:48908
I20260709 12:29:11.656572 12327 raft_consensus.cc:606] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:11.659160 12327 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e72262c8e2b24a8fa97e1cacbb60b49a"
dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:48908
I20260709 12:29:11.659651 12327 raft_consensus.cc:606] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:11.662919  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:29:11.664815  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:11.666019  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:11.743692 13482 raft_consensus.cc:993] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4: : Instructing follower a800ae68e91f43a794e326badca3320a to start an election
I20260709 12:29:11.744336 13479 raft_consensus.cc:1081] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Signalling peer a800ae68e91f43a794e326badca3320a to start an election
I20260709 12:29:11.746377 12475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2a2e0c681f24264a729f89d08042fd9"
dest_uuid: "a800ae68e91f43a794e326badca3320a"
 from {username='slave'} at 127.0.0.1:59418
I20260709 12:29:11.747107 12475 raft_consensus.cc:493] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:11.747560 12475 raft_consensus.cc:3060] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.757433 12475 raft_consensus.cc:515] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:11.759732 12475 leader_election.cc:290] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:11.762912 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2a2e0c681f24264a729f89d08042fd9" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:11.763181 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2a2e0c681f24264a729f89d08042fd9" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:11.763599 12327 raft_consensus.cc:3055] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:11.763971 12327 raft_consensus.cc:740] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:11.763936 12410 raft_consensus.cc:3060] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.765079 12327 consensus_queue.cc:260] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:11.766722 12327 raft_consensus.cc:3060] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.773244 12410 raft_consensus.cc:2468] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2.
I20260709 12:29:11.774959 12450 leader_election.cc:304] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:11.775874 13361 raft_consensus.cc:2804] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:11.776372 13361 raft_consensus.cc:697] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Becoming Leader. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:11.777254 13361 consensus_queue.cc:237] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:11.780313 12327 raft_consensus.cc:2468] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a800ae68e91f43a794e326badca3320a in term 2.
I20260709 12:29:11.793031 12155 catalog_manager.cc:5697] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a reported cstate change: term changed from 1 to 2, leader changed from 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194) to a800ae68e91f43a794e326badca3320a (127.3.89.196). New cstate: current_term: 2 leader_uuid: "a800ae68e91f43a794e326badca3320a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:11.810067 13482 raft_consensus.cc:993] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:11.814615 13479 raft_consensus.cc:1081] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:11.820506 13151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58654
I20260709 12:29:11.821421 13151 raft_consensus.cc:493] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:11.821964 13151 raft_consensus.cc:3060] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:11.842761 13151 raft_consensus.cc:515] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:11.845427 13151 leader_election.cc:290] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:11.846199 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:11.846915 12327 raft_consensus.cc:3055] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:11.847250 12327 raft_consensus.cc:740] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:11.848141 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89a75dc9fa64c1c94a790754889c07a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:11.848081 12327 consensus_queue.cc:260] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:11.848853 12410 raft_consensus.cc:3060] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:11.858058 12327 raft_consensus.cc:3060] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:11.882493 12410 raft_consensus.cc:2468] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 3.
I20260709 12:29:11.884392 13103 leader_election.cc:304] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:11.891816 13493 raft_consensus.cc:2804] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:11.894151 12327 raft_consensus.cc:2468] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 3.
I20260709 12:29:11.909193 13493 raft_consensus.cc:697] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 3 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:11.910446 13493 consensus_queue.cc:237] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:11.912428 13490 raft_consensus.cc:993] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:11.913050 13479 raft_consensus.cc:1081] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:11.915938 13154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e72262c8e2b24a8fa97e1cacbb60b49a"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58654
I20260709 12:29:11.916579 13154 raft_consensus.cc:493] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:11.917052 13154 raft_consensus.cc:3060] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.934249 13154 raft_consensus.cc:515] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:11.936918 13154 leader_election.cc:290] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:11.939635 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e72262c8e2b24a8fa97e1cacbb60b49a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:11.940250 12479 raft_consensus.cc:3060] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.940039 12155 catalog_manager.cc:5697] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 2 to 3, leader changed from 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 3 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:11.941370 12327 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e72262c8e2b24a8fa97e1cacbb60b49a" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:11.941965 12327 raft_consensus.cc:3055] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:11.942237 12327 raft_consensus.cc:740] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:11.949721 12327 consensus_queue.cc:260] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:11.951617 12327 raft_consensus.cc:3060] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.961795 13479 raft_consensus.cc:993] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:11.963532 13479 raft_consensus.cc:1081] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:11.976586 13480 raft_consensus.cc:993] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:11.977771 13480 raft_consensus.cc:1081] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:11.979887 13154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58646
I20260709 12:29:11.980798 13157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ff454851114445da8be05eda167efc4"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58654
I20260709 12:29:11.981413 13157 raft_consensus.cc:493] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:11.981801 13157 raft_consensus.cc:3060] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.980453 13154 raft_consensus.cc:493] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:11.991087 13154 raft_consensus.cc:3060] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:11.993757 13157 raft_consensus.cc:515] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:11.996390 13157 leader_election.cc:290] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:11.997252 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff454851114445da8be05eda167efc4" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:11.997941 12483 raft_consensus.cc:3060] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:11.998658 12327 raft_consensus.cc:2468] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:12.006099 12483 raft_consensus.cc:2468] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:12.007902 13100 leader_election.cc:304] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:12.009770 13493 raft_consensus.cc:2804] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:12.010187 12479 raft_consensus.cc:2468] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:12.010375 13493 raft_consensus.cc:697] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:12.011560 13100 leader_election.cc:304] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:12.012456 13494 raft_consensus.cc:2804] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:12.013566 13493 consensus_queue.cc:237] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:12.017336 13494 raft_consensus.cc:697] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:12.018586 13494 consensus_queue.cc:237] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:12.036039 13154 raft_consensus.cc:515] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:12.043812 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:12.044548 12479 raft_consensus.cc:3055] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:12.045123 12479 raft_consensus.cc:740] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:12.044539 13154 leader_election.cc:290] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:29:12.046036 12302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:40155, user_credentials={real_user=slave}} blocked reactor thread for 78984.8us
I20260709 12:29:12.049674 12155 catalog_manager.cc:5697] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 1 to 2, leader changed from 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:12.052276 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff454851114445da8be05eda167efc4" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:12.052734 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:12.053472 12335 raft_consensus.cc:3060] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:12.053016 12332 raft_consensus.cc:3055] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:12.059257 12332 raft_consensus.cc:740] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:12.060192 12332 consensus_queue.cc:260] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:12.061452 12332 raft_consensus.cc:3060] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:12.073499 12335 raft_consensus.cc:2468] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 3.
I20260709 12:29:12.076064 12332 raft_consensus.cc:2468] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:12.078078 13100 leader_election.cc:304] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:29:12.081418 13479 raft_consensus.cc:993] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:12.082114 13494 raft_consensus.cc:2804] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:12.082240 13479 raft_consensus.cc:1081] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:12.083106 13494 raft_consensus.cc:697] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 3 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:12.084509 13494 consensus_queue.cc:237] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:12.086727 13157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd19eddb6c2e45b7acefc5c295d92928"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58654
I20260709 12:29:12.087772 13157 raft_consensus.cc:493] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:29:12.088213 12373 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:37927, user_credentials={real_user=slave}} blocked reactor thread for 49015us
I20260709 12:29:12.088229 13157 raft_consensus.cc:3060] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:12.089934 12155 catalog_manager.cc:5697] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 1 to 2, leader changed from 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:12.095434 13472 raft_consensus.cc:993] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596: : Instructing follower 4e02e5f693ce49a3b161ad90848be7d4 to start an election
I20260709 12:29:12.096112 13502 raft_consensus.cc:1081] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Signalling peer 4e02e5f693ce49a3b161ad90848be7d4 to start an election
I20260709 12:29:12.108011 12332 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681"
dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
 from {username='slave'} at 127.0.0.1:56372
I20260709 12:29:12.088379 12479 consensus_queue.cc:260] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:12.124580 12332 raft_consensus.cc:493] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:12.132370 12332 raft_consensus.cc:3060] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:12.144007 12479 raft_consensus.cc:3060] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:12.152230 13157 raft_consensus.cc:515] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:12.154946 13157 leader_election.cc:290] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:12.155622 12329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd19eddb6c2e45b7acefc5c295d92928" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:12.156294 12329 raft_consensus.cc:3055] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:12.156360 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd19eddb6c2e45b7acefc5c295d92928" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:12.156690 12329 raft_consensus.cc:740] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:12.157133 12408 raft_consensus.cc:3060] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:12.157706 12329 consensus_queue.cc:260] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:12.161049 12329 raft_consensus.cc:3060] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:12.152333 12332 raft_consensus.cc:515] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:12.162962 12332 leader_election.cc:290] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:12.165300 12483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:12.165941 12483 raft_consensus.cc:3060] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:12.166574 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fe65f1e2e7d4ae796006f23575cb681" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:12.167191 12410 raft_consensus.cc:3055] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:12.167534 12410 raft_consensus.cc:740] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:12.167842 12408 raft_consensus.cc:2468] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:12.168330 12410 consensus_queue.cc:260] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:12.169534 13103 leader_election.cc:304] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:12.169775 12410 raft_consensus.cc:3060] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:12.170632 13493 raft_consensus.cc:2804] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:12.171185 13493 raft_consensus.cc:697] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:12.172176 13493 consensus_queue.cc:237] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:12.178231 12329 raft_consensus.cc:2468] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:12.185464 12479 raft_consensus.cc:2468] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 3.
I20260709 12:29:12.186445 12483 raft_consensus.cc:2468] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 3.
I20260709 12:29:12.187856 12299 leader_election.cc:304] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:12.189975 13479 raft_consensus.cc:2804] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:12.191044 13479 raft_consensus.cc:697] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 LEADER]: Becoming Leader. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:12.192174 13479 consensus_queue.cc:237] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } }
I20260709 12:29:12.193143 12155 catalog_manager.cc:5697] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 2 to 3, leader changed from a800ae68e91f43a794e326badca3320a (127.3.89.196) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 3 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:12.201373 12410 raft_consensus.cc:2468] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e02e5f693ce49a3b161ad90848be7d4 in term 3.
I20260709 12:29:12.222265 12155 catalog_manager.cc:5697] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 1 to 2, leader changed from 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:12.227733 12158 catalog_manager.cc:5697] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 reported cstate change: term changed from 2 to 3, leader changed from a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195) to 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194). New cstate: current_term: 3 leader_uuid: "4e02e5f693ce49a3b161ad90848be7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:12.304296 12408 raft_consensus.cc:1275] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:12.306310 13504 consensus_queue.cc:1048] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:12.329300 12332 raft_consensus.cc:1275] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:12.332141 13493 consensus_queue.cc:1048] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:12.349278 12329 raft_consensus.cc:1275] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer a800ae68e91f43a794e326badca3320a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:12.354889 13503 consensus_queue.cc:1048] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:12.401058 12410 raft_consensus.cc:1275] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:12.403229 13493 consensus_queue.cc:1048] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:12.420769 12335 raft_consensus.cc:1275] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:12.432848 13498 consensus_queue.cc:1048] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:12.487113 12484 raft_consensus.cc:1275] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:12.488570 12475 raft_consensus.cc:1275] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:12.547684 13498 consensus_queue.cc:1048] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:12.551203 13498 consensus_queue.cc:1048] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:12.576265 12332 raft_consensus.cc:1275] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:12.581542 13498 consensus_queue.cc:1048] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:12.597692 12475 raft_consensus.cc:1275] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:12.599690 13498 consensus_queue.cc:1048] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:12.615352 12335 raft_consensus.cc:1275] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:12.617367 13493 consensus_queue.cc:1048] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:12.644491 12410 raft_consensus.cc:1275] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:12.649767 13482 consensus_queue.cc:1048] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:12.673069 12408 raft_consensus.cc:1275] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:12.676335 13493 consensus_queue.cc:1048] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:12.693147 12479 raft_consensus.cc:1275] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Refusing update from remote peer 4e02e5f693ce49a3b161ad90848be7d4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:12.694866 13497 consensus_queue.cc:1048] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:12.708739  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:12.714574 12332 raft_consensus.cc:1275] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:12.716727 13498 consensus_queue.cc:1048] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:12.808612 12332 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69e3b47e111a479f8582c270c1445658"
dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:48908
I20260709 12:29:12.809234 12332 raft_consensus.cc:606] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:12.811817 12410 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7540665883f74d94b96d1631ac30970d"
dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:46500
I20260709 12:29:12.812382 12410 raft_consensus.cc:606] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:12.815735 12410 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222"
dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:46500
I20260709 12:29:12.816318 12410 raft_consensus.cc:606] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:12.818871 12332 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9"
dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:48908
I20260709 12:29:12.819424 12332 raft_consensus.cc:606] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:12.822638 12410 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60"
dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:46500
I20260709 12:29:12.823942 12410 raft_consensus.cc:606] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:12.826771 12484 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc"
dest_uuid: "a800ae68e91f43a794e326badca3320a"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:42482
I20260709 12:29:12.827279 12484 raft_consensus.cc:606] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:12.828593  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:29:12.830916  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:12.831854  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:12.967758 13490 raft_consensus.cc:993] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:12.968384 13479 raft_consensus.cc:1081] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:12.970873 13157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "69e3b47e111a479f8582c270c1445658"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58654
I20260709 12:29:12.971457 13157 raft_consensus.cc:493] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:12.971882 13157 raft_consensus.cc:3060] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:12.978443 13157 raft_consensus.cc:515] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:12.982280 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e3b47e111a479f8582c270c1445658" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:12.982431 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69e3b47e111a479f8582c270c1445658" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:12.982893 12332 raft_consensus.cc:3055] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:12.982955 12410 raft_consensus.cc:3060] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:12.983284 12332 raft_consensus.cc:740] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:12.984763 12332 consensus_queue.cc:260] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:12.986428 12332 raft_consensus.cc:3060] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:12.988654 13157 leader_election.cc:290] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:12.991034 12410 raft_consensus.cc:2468] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:12.992331 13103 leader_election.cc:304] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:12.993330 13498 raft_consensus.cc:2804] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:12.993863 13498 raft_consensus.cc:697] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:12.994709 13498 consensus_queue.cc:237] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:13.007444 12332 raft_consensus.cc:2468] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:13.017210 12155 catalog_manager.cc:5697] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 1 to 2, leader changed from 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:13.042398 13501 raft_consensus.cc:993] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:13.043123 13481 raft_consensus.cc:1081] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:13.047124 13157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7540665883f74d94b96d1631ac30970d"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58630
I20260709 12:29:13.047708 13157 raft_consensus.cc:493] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:13.048091 13157 raft_consensus.cc:3060] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:13.065732 13490 raft_consensus.cc:993] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:13.066355 13490 raft_consensus.cc:1081] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:13.074321 13154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58654
I20260709 12:29:13.074928 13154 raft_consensus.cc:493] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:13.075268 13154 raft_consensus.cc:3060] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:13.075714 13157 raft_consensus.cc:515] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:13.078171 13157 leader_election.cc:290] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:13.082006 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7540665883f74d94b96d1631ac30970d" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:13.082605 12335 raft_consensus.cc:3060] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:13.087540 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7540665883f74d94b96d1631ac30970d" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:13.088433 12410 raft_consensus.cc:3055] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:13.088824 12410 raft_consensus.cc:740] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:13.096242 13154 raft_consensus.cc:515] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:13.098935 12410 consensus_queue.cc:260] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:13.100234 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:13.100906 12484 raft_consensus.cc:3060] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:13.103163 13154 leader_election.cc:290] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:13.100409 12410 raft_consensus.cc:3060] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:13.107391 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93481f7df95d427eaa757fbbe87895d9" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:13.107995 12332 raft_consensus.cc:3055] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:13.108302 12332 raft_consensus.cc:740] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e02e5f693ce49a3b161ad90848be7d4, State: Running, Role: LEADER
I20260709 12:29:13.109210 12484 raft_consensus.cc:2468] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 3.
I20260709 12:29:13.109079 12332 consensus_queue.cc:260] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:13.110538 13100 leader_election.cc:304] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:13.110715 12332 raft_consensus.cc:3060] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:13.111454 13493 raft_consensus.cc:2804] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:13.111948 13493 raft_consensus.cc:697] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 3 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:13.112879 13493 consensus_queue.cc:237] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:13.114689 12335 raft_consensus.cc:2468] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:13.115968 13100 leader_election.cc:304] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:29:13.116779 13494 raft_consensus.cc:2804] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:13.117324 13494 raft_consensus.cc:697] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:13.118412 13494 consensus_queue.cc:237] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:13.138859 12410 raft_consensus.cc:2468] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:13.154587 12332 raft_consensus.cc:2468] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 3.
I20260709 12:29:13.161090 12155 catalog_manager.cc:5697] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 2 to 3, leader changed from 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 3 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:13.200861 12155 catalog_manager.cc:5697] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 1 to 2, leader changed from a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:13.225338 13361 raft_consensus.cc:993] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:13.226415 13362 raft_consensus.cc:1081] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:13.228099 13154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58646
I20260709 12:29:13.228713 13154 raft_consensus.cc:493] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:13.229065 13154 raft_consensus.cc:3060] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:13.238003 13154 raft_consensus.cc:515] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:13.250753 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:13.251372 12410 raft_consensus.cc:3060] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:13.252246 13472 raft_consensus.cc:993] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:13.252794 13472 raft_consensus.cc:1081] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:13.254993 13154 leader_election.cc:290] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:13.255519 12484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:13.256146 12484 raft_consensus.cc:3055] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:13.256439 12484 raft_consensus.cc:740] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:13.259100 13154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58630
I20260709 12:29:13.260617 13154 raft_consensus.cc:493] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:13.261068 13154 raft_consensus.cc:3060] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:13.266131 12410 raft_consensus.cc:2468] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 3.
I20260709 12:29:13.257233 12484 consensus_queue.cc:260] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:13.267444 13103 leader_election.cc:304] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:13.267473 12484 raft_consensus.cc:3060] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:13.268680 13532 raft_consensus.cc:2804] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:13.269239 13532 raft_consensus.cc:697] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 3 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:13.270337 13532 consensus_queue.cc:237] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:13.275274 13154 raft_consensus.cc:515] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:13.276495 12484 raft_consensus.cc:2468] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 3.
I20260709 12:29:13.279685 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:13.280256 12410 raft_consensus.cc:3055] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:13.280545 12410 raft_consensus.cc:740] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:13.281450 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7c870391ab41aa99577da648d9ad60" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:13.281306 12410 consensus_queue.cc:260] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:13.282126 12335 raft_consensus.cc:3060] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:13.282683 12410 raft_consensus.cc:3060] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:13.292770 12410 raft_consensus.cc:2468] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 3.
I20260709 12:29:13.293453 13154 leader_election.cc:290] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:13.295843 13103 leader_election.cc:304] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596; no voters: 
I20260709 12:29:13.297065 13498 raft_consensus.cc:2804] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:13.297748 13498 raft_consensus.cc:697] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 3 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:13.297837 12335 raft_consensus.cc:2468] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 3.
I20260709 12:29:13.298954 13498 consensus_queue.cc:237] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:13.310623 12155 catalog_manager.cc:5697] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 2 to 3, leader changed from a800ae68e91f43a794e326badca3320a (127.3.89.196) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 3 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:13.331303 12155 catalog_manager.cc:5697] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 2 to 3, leader changed from a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 3 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:13.340037 13472 raft_consensus.cc:993] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:13.340677 13472 raft_consensus.cc:1081] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:13.343107 13157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58630
I20260709 12:29:13.343686 13157 raft_consensus.cc:493] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:13.344060 13157 raft_consensus.cc:3060] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:13.355707 13157 raft_consensus.cc:515] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:13.358074 13157 leader_election.cc:290] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 4 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:13.358986 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:13.359759 12410 raft_consensus.cc:3055] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:13.360095 12410 raft_consensus.cc:740] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:13.360217 12335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87c6d482d9f1412a8f1ec924aa6d1222" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:13.360844 12335 raft_consensus.cc:3060] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:13.360949 12410 consensus_queue.cc:260] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:13.362543 12410 raft_consensus.cc:3060] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:13.368436 12335 raft_consensus.cc:2468] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 4.
I20260709 12:29:13.370045 13100 leader_election.cc:304] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:29:13.374835 13532 raft_consensus.cc:2804] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:13.376899 13532 raft_consensus.cc:697] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 4 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:13.377893 13532 consensus_queue.cc:237] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:13.385355 12410 raft_consensus.cc:2468] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 4.
I20260709 12:29:13.389405 12155 catalog_manager.cc:5697] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 3 to 4, leader changed from a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 4 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:13.434259 12410 raft_consensus.cc:1275] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:13.435545 13532 consensus_queue.cc:1048] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:13.471104 12332 raft_consensus.cc:1275] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:13.473815 13532 consensus_queue.cc:1048] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:13.550010 12332 raft_consensus.cc:1275] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:13.551743 13498 consensus_queue.cc:1048] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:13.553404 12332 raft_consensus.cc:1275] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:13.555254 13498 consensus_queue.cc:1048] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:13.568050 12410 raft_consensus.cc:1275] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:13.570380 13493 consensus_queue.cc:1048] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:13.580772 12479 raft_consensus.cc:1275] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:13.589257 13493 consensus_queue.cc:1048] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:13.819661 12332 raft_consensus.cc:1275] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:13.855216 13493 consensus_queue.cc:1048] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:13.855794 12475 raft_consensus.cc:1275] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:13.858258 13493 consensus_queue.cc:1048] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:13.867884 12410 raft_consensus.cc:1275] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:13.871598 13532 consensus_queue.cc:1048] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:13.900079  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:13.905092 12408 raft_consensus.cc:1275] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:13.907191 13494 consensus_queue.cc:1048] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:13.911530 12332 raft_consensus.cc:1275] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 4 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:13.916729 13494 consensus_queue.cc:1048] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:29:13.958238 13554 log.cc:927] Time spent T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:29:13.990890 12410 raft_consensus.cc:1275] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:13.993238 13494 consensus_queue.cc:1048] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:14.057341 12475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34"
dest_uuid: "a800ae68e91f43a794e326badca3320a"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:42482
I20260709 12:29:14.058030 12475 raft_consensus.cc:606] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:14.064646 12408 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3"
dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:46500
I20260709 12:29:14.065263 12408 raft_consensus.cc:606] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:14.070116 12408 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8d9482906bb4ca4a34c22c21f178cab"
dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
mode: GRACEFUL
new_leader_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:46500
I20260709 12:29:14.070744 12408 raft_consensus.cc:606] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Received request to transfer leadership to TS 1dcded10caa0493da818e67e13176979
I20260709 12:29:14.072841  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:14.075197  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:14.076135  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:14.301240 13548 raft_consensus.cc:993] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:14.301796 13548 raft_consensus.cc:1081] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:14.303397 13157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58646
I20260709 12:29:14.303952 13157 raft_consensus.cc:493] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:14.304294 13157 raft_consensus.cc:3060] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:14.312417 13157 raft_consensus.cc:515] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:14.314553 13157 leader_election.cc:290] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:14.316599 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:14.317060 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606a9d73d38c472698c0f88704e59b34" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:14.317400 12479 raft_consensus.cc:3055] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:14.317795 12479 raft_consensus.cc:740] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a800ae68e91f43a794e326badca3320a, State: Running, Role: LEADER
I20260709 12:29:14.317807 12332 raft_consensus.cc:3060] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:14.318715 12479 consensus_queue.cc:260] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:14.319747 12479 raft_consensus.cc:3060] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:14.325064 12479 raft_consensus.cc:2468] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:14.325937 12332 raft_consensus.cc:2468] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:14.326399 13100 leader_election.cc:304] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:14.327340 13494 raft_consensus.cc:2804] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:14.327858 13494 raft_consensus.cc:697] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:14.328783 13494 consensus_queue.cc:237] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:14.355661 12156 catalog_manager.cc:5697] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 1 to 2, leader changed from a800ae68e91f43a794e326badca3320a (127.3.89.196) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:14.383316 13552 raft_consensus.cc:993] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:14.383862 13552 raft_consensus.cc:1081] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:14.386253 13157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58630
I20260709 12:29:14.386773 13157 raft_consensus.cc:493] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:14.387105 13157 raft_consensus.cc:3060] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:14.402544 13157 raft_consensus.cc:515] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:14.404196 13157 leader_election.cc:290] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Requested vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), a800ae68e91f43a794e326badca3320a (127.3.89.196:45677)
I20260709 12:29:14.405009 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:14.405483 12479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa6f87b452d4a7fbc6803c9fd74e9c3" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:14.405779 12410 raft_consensus.cc:3055] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:14.406215 12410 raft_consensus.cc:740] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:14.406184 12479 raft_consensus.cc:3060] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:14.407087 12410 consensus_queue.cc:260] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:14.408259 12410 raft_consensus.cc:3060] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:14.418522 12479 raft_consensus.cc:2468] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 3.
I20260709 12:29:14.419880 13100 leader_election.cc:304] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:14.420248 12410 raft_consensus.cc:2468] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 3.
I20260709 12:29:14.421419 13532 raft_consensus.cc:2804] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:14.421993 13532 raft_consensus.cc:697] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 3 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:14.422947 13532 consensus_queue.cc:237] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:14.453408 12156 catalog_manager.cc:5697] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 2 to 3, leader changed from a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 3 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:14.545082 13552 raft_consensus.cc:993] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596: : Instructing follower 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:14.545672 13472 raft_consensus.cc:1081] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Signalling peer 1dcded10caa0493da818e67e13176979 to start an election
I20260709 12:29:14.547523 13157 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8d9482906bb4ca4a34c22c21f178cab"
dest_uuid: "1dcded10caa0493da818e67e13176979"
 from {username='slave'} at 127.0.0.1:58630
I20260709 12:29:14.548014 13157 raft_consensus.cc:493] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:14.548303 13157 raft_consensus.cc:3060] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:14.557116 13157 raft_consensus.cc:515] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:14.559665 13157 leader_election.cc:290] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Requested vote from peers 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269), a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155)
I20260709 12:29:14.560093 12332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d9482906bb4ca4a34c22c21f178cab" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e02e5f693ce49a3b161ad90848be7d4"
I20260709 12:29:14.560755 12332 raft_consensus.cc:3060] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:14.561233 12408 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8d9482906bb4ca4a34c22c21f178cab" candidate_uuid: "1dcded10caa0493da818e67e13176979" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596"
I20260709 12:29:14.561921 12408 raft_consensus.cc:3055] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:14.562232 12408 raft_consensus.cc:740] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:14.563078 12408 consensus_queue.cc:260] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:14.564661 12408 raft_consensus.cc:3060] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:14.568549 12332 raft_consensus.cc:2468] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:14.570091 13100 leader_election.cc:304] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4; no voters: 
I20260709 12:29:14.571591 13532 raft_consensus.cc:2804] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:14.572058 13532 raft_consensus.cc:697] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Becoming Leader. State: Replica: 1dcded10caa0493da818e67e13176979, State: Running, Role: LEADER
I20260709 12:29:14.572909 13532 consensus_queue.cc:237] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:14.577518 12408 raft_consensus.cc:2468] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dcded10caa0493da818e67e13176979 in term 2.
I20260709 12:29:14.599475 12156 catalog_manager.cc:5697] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 reported cstate change: term changed from 1 to 2, leader changed from a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195) to 1dcded10caa0493da818e67e13176979 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "1dcded10caa0493da818e67e13176979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:14.843931 12479 raft_consensus.cc:1275] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:14.846927 13551 consensus_queue.cc:1048] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:14.893216 12332 raft_consensus.cc:1275] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:14.897720 13532 consensus_queue.cc:1048] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:14.953713 12479 raft_consensus.cc:1275] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:14.955160 13494 consensus_queue.cc:1048] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:14.989315 12408 raft_consensus.cc:1275] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:14.992115 13494 consensus_queue.cc:1048] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:15.008291 12332 raft_consensus.cc:1275] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:15.015458 13494 consensus_queue.cc:1048] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:15.048087 12408 raft_consensus.cc:1275] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Refusing update from remote peer 1dcded10caa0493da818e67e13176979: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:15.049762 13551 consensus_queue.cc:1048] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:15.133980  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:15.175232  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:15.177007  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:15.177666  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:15.255986  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:15.337165  3431 tablet_server.cc:179] TabletServer@127.3.89.193:37927 shutting down...
W20260709 12:29:15.340873 12373 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 87 similar messages]
W20260709 12:29:15.362747 12447 consensus_peers.cc:597] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.366112 12373 consensus_peers.cc:597] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.399364 12299 consensus_peers.cc:597] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.401881  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:15.403005  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.415849  3431 raft_consensus.cc:2243] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.416458  3431 raft_consensus.cc:2272] T 2e2871c5312642a28d3d3d65a5941a3a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.419243  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.422658  3431 raft_consensus.cc:2243] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:15.423431  3431 raft_consensus.cc:2272] T b89a75dc9fa64c1c94a790754889c07a P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.425782  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.426597  3431 raft_consensus.cc:2243] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.427107  3431 raft_consensus.cc:2272] T c67e469fc10a43bf9660c48b5ae1335a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.429637  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.430392  3431 raft_consensus.cc:2243] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:15.431221  3431 raft_consensus.cc:2272] T ce7f904b0c124a17881b5b725eaa1bdc P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.433491  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.434448  3431 raft_consensus.cc:2243] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:15.435530  3431 raft_consensus.cc:2272] T e72262c8e2b24a8fa97e1cacbb60b49a P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.438603  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.439488  3431 raft_consensus.cc:2243] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:15.440382  3431 raft_consensus.cc:2272] T 69e3b47e111a479f8582c270c1445658 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.442787  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.443377  3431 raft_consensus.cc:2243] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.443843  3431 raft_consensus.cc:2272] T e94567d0d49d4bf0882ca910a8757cbc P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.446815  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.447829  3431 raft_consensus.cc:2243] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.448369  3431 raft_consensus.cc:2272] T 83d73b1bb20e4e79819072554a1ee076 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:15.451373 12299 consensus_peers.cc:597] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.461340  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.462481  3431 raft_consensus.cc:2243] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.463078  3431 raft_consensus.cc:2272] T b6cc6fec74b6423bb49415fbcdd93e55 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:15.463532 12299 consensus_peers.cc:597] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.469338  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.470486  3431 raft_consensus.cc:2243] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.471212  3431 raft_consensus.cc:2272] T fcca3076fc03410aa86e82c060c29335 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:15.473290 12299 consensus_peers.cc:597] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.474500  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.475584  3431 raft_consensus.cc:2243] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.476181  3431 raft_consensus.cc:2272] T 1263de4693a84afe93a508aff7f1dc29 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.479310  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.480012  3431 raft_consensus.cc:2243] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.480530  3431 raft_consensus.cc:2272] T d0c8b89ad96240b588abe04f13ea2f9d P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.483143  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.483929  3431 raft_consensus.cc:2243] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:15.484905  3431 raft_consensus.cc:2272] T 7ff454851114445da8be05eda167efc4 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.487995  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.488919  3431 raft_consensus.cc:2243] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.489530  3431 raft_consensus.cc:2272] T ab18688981ff456ca286ec171980c8de P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.492367  3431 tablet_replica.cc:333] stopping tablet replica
W20260709 12:29:15.492424 12373 consensus_peers.cc:597] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.493278  3431 raft_consensus.cc:2243] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.493865  3431 raft_consensus.cc:2272] T a6b87f00ff8c4e1c858037069e3bbb6e P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.496583  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.497503  3431 raft_consensus.cc:2243] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:15.498724  3431 raft_consensus.cc:2272] T 93481f7df95d427eaa757fbbe87895d9 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.501448  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.502310  3431 raft_consensus.cc:2243] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:15.502518 12447 consensus_peers.cc:597] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.502884  3431 raft_consensus.cc:2272] T 7cf8c3b9ee65415785e06fb8c085ada7 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.505736  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.506620  3431 raft_consensus.cc:2243] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.507225  3431 raft_consensus.cc:2272] T b0b789be887a492fb0e51b08f2475a97 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.510351  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.511086  3431 raft_consensus.cc:2243] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.511538  3431 raft_consensus.cc:2272] T 164179fdd9ee45ff80ea3530e210088a P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.514333  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.515374  3431 raft_consensus.cc:2243] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:15.516438  3431 raft_consensus.cc:2272] T bf7c870391ab41aa99577da648d9ad60 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.519419  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.520344  3431 raft_consensus.cc:2243] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:15.521363  3431 raft_consensus.cc:2272] T 307ee3a81a6649ebb96772cbd5e29f92 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.524613  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.525842  3431 raft_consensus.cc:2243] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:15.526851  3431 raft_consensus.cc:2272] T 7540665883f74d94b96d1631ac30970d P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:15.527540 12447 consensus_peers.cc:597] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.529172  3431 tablet_replica.cc:333] stopping tablet replica
W20260709 12:29:15.529163 12447 consensus_peers.cc:597] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.530391  3431 raft_consensus.cc:2243] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:29:15.530436 12447 consensus_peers.cc:597] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.531421  3431 raft_consensus.cc:2272] T c8d9482906bb4ca4a34c22c21f178cab P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.534740  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.535574  3431 raft_consensus.cc:2243] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.536136  3431 raft_consensus.cc:2272] T 485001bca51c4145939dc8bd38b1169a P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.539258  3431 tablet_replica.cc:333] stopping tablet replica
W20260709 12:29:15.539482 12447 consensus_peers.cc:597] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.540164  3431 raft_consensus.cc:2243] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.540813  3431 raft_consensus.cc:2272] T f39a7bfe4bff44b18d1007ced129ae09 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.543418  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.544080  3431 raft_consensus.cc:2243] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.544562  3431 raft_consensus.cc:2272] T 2ad2a742adea4bf989def5e758a3a81c P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:15.544780 12447 consensus_peers.cc:597] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.547447  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.548213  3431 raft_consensus.cc:2243] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.548650  3431 raft_consensus.cc:2272] T 8b89bcc99e094d1bbaf83979c2b78081 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.550961  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.551621  3431 raft_consensus.cc:2243] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.552143  3431 raft_consensus.cc:2272] T 6a1cc112d1984521b5479f66b4289cce P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.554482  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.555171  3431 raft_consensus.cc:2243] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.555721  3431 raft_consensus.cc:2272] T 6417cb26ca8d401f8e73d3512aeab50b P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.558810  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.559684  3431 raft_consensus.cc:2243] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.560253  3431 raft_consensus.cc:2272] T a03f6aed8fcb4ea2b45b6432634c00ae P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:15.563019 12373 consensus_peers.cc:597] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.563305  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.564292  3431 raft_consensus.cc:2243] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.564826  3431 raft_consensus.cc:2272] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.567049  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.567793  3431 raft_consensus.cc:2243] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.568339  3431 raft_consensus.cc:2272] T f5ae8840033347bea42f3c0ceb4d316a P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.571529  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.572961  3431 raft_consensus.cc:2243] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.573555  3431 raft_consensus.cc:2272] T 228aa7f69257458ba0c2b5e6c950f217 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.576520  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.577307  3431 raft_consensus.cc:2243] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:15.578379  3431 raft_consensus.cc:2272] T 606a9d73d38c472698c0f88704e59b34 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.581701  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.582612  3431 raft_consensus.cc:2243] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.583222  3431 raft_consensus.cc:2272] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 1dcded10caa0493da818e67e13176979 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.586041  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.586836  3431 raft_consensus.cc:2243] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.587340  3431 raft_consensus.cc:2272] T fd891675417c4265b77695b4227d980f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.590157  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.591033  3431 raft_consensus.cc:2243] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:15.592108  3431 raft_consensus.cc:2272] T dd19eddb6c2e45b7acefc5c295d92928 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.595177  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.596015  3431 raft_consensus.cc:2243] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.596611  3431 raft_consensus.cc:2272] T 7916cfbec26b4115a9251411482bed41 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.599977  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.600992  3431 raft_consensus.cc:2243] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:15.602108  3431 raft_consensus.cc:2272] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P 1dcded10caa0493da818e67e13176979 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.606142  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.607036  3431 raft_consensus.cc:2243] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.607568  3431 raft_consensus.cc:2272] T 864a0424f5d64be0a7d4d8b5d38907c6 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.610268  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.611161  3431 raft_consensus.cc:2243] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.611599  3431 raft_consensus.cc:2272] T 13100e49825f4fcfa78791b73b8d7555 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:15.611581 12373 consensus_peers.cc:597] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.614751  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.615622  3431 raft_consensus.cc:2243] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.616292  3431 raft_consensus.cc:2272] T 8d8f5c63c391477fad5c9155f5022b07 P 1dcded10caa0493da818e67e13176979 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.625605  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.626487  3431 raft_consensus.cc:2243] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.627115  3431 raft_consensus.cc:2272] T 81f0d8651a6144ca9fd21ccc0f57edd9 P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.630209  3431 tablet_replica.cc:333] stopping tablet replica
W20260709 12:29:15.630949 12299 consensus_peers.cc:597] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.630990  3431 raft_consensus.cc:2243] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.632206  3431 raft_consensus.cc:2272] T 05dfc5c361a94fe5af3e9a5462a0b12f P 1dcded10caa0493da818e67e13176979 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:15.633833 12299 consensus_peers.cc:597] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.635994  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:15.636852  3431 raft_consensus.cc:2243] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:15.637948  3431 raft_consensus.cc:2272] T 87c6d482d9f1412a8f1ec924aa6d1222 P 1dcded10caa0493da818e67e13176979 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:15.652379 12373 consensus_peers.cc:597] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.654246 12447 consensus_peers.cc:597] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.658356 12447 consensus_peers.cc:597] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.668726 12447 consensus_peers.cc:597] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.694610 12447 consensus_peers.cc:597] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.705173 12373 consensus_peers.cc:597] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.748559 12373 consensus_peers.cc:597] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.752871 12299 consensus_peers.cc:597] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.781416 12299 consensus_peers.cc:597] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.781917 12299 consensus_peers.cc:597] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.840215 12373 consensus_peers.cc:597] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.875934 12447 consensus_peers.cc:597] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.894487 12373 consensus_peers.cc:597] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.914927 12373 consensus_peers.cc:597] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Couldn't send request to peer 1dcded10caa0493da818e67e13176979. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.377125  3431 tablet_server.cc:196] TabletServer@127.3.89.193:37927 shutdown complete.
I20260709 12:29:16.416838 13550 raft_consensus.cc:493] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.417632 13550 raft_consensus.cc:515] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:16.420783 12410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "a800ae68e91f43a794e326badca3320a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2104c8a9b144a419ccb7c07e3f02596" is_pre_election: true
I20260709 12:29:16.423308 13550 leader_election.cc:290] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
W20260709 12:29:16.425741 12447 leader_election.cc:336] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:16.426357 12447 leader_election.cc:304] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:16.427390 13550 raft_consensus.cc:2749] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:16.479498  3431 tablet_server.cc:179] TabletServer@127.3.89.194:0 shutting down...
W20260709 12:29:16.494361 12447 consensus_peers.cc:597] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:16.523032 12447 consensus_peers.cc:597] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.527487  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:16.528988  3431 tablet_replica.cc:333] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.529968  3431 raft_consensus.cc:2243] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.530782  3431 raft_consensus.cc:2272] T 4fe65f1e2e7d4ae796006f23575cb681 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.532984  3431 tablet_replica.cc:333] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.533877  3431 raft_consensus.cc:2243] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.534451  3431 raft_consensus.cc:2272] T fcca3076fc03410aa86e82c060c29335 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.537371  3431 tablet_replica.cc:333] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.538232  3431 raft_consensus.cc:2243] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.539412  3431 raft_consensus.cc:2272] T d6682a011b3945c59d105bb6951d62f3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.542258  3431 tablet_replica.cc:333] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.543417  3431 raft_consensus.cc:2243] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.543929  3431 raft_consensus.cc:2272] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P 4e02e5f693ce49a3b161ad90848be7d4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.546422  3431 tablet_replica.cc:333] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.547008  3431 raft_consensus.cc:2243] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.547448  3431 raft_consensus.cc:2272] T 606a9d73d38c472698c0f88704e59b34 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.549341  3431 tablet_replica.cc:333] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.550001  3431 raft_consensus.cc:2243] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.551011  3431 raft_consensus.cc:2272] T 76b0d4a9a29a4de9bc3a8ab529535d68 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.553534  3431 tablet_replica.cc:333] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.554303  3431 raft_consensus.cc:2243] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 4 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:16.554488 12373 consensus_peers.cc:597] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.554780  3431 raft_consensus.cc:2272] T 436db7228d9d4e92a8472906005ae3fb P 4e02e5f693ce49a3b161ad90848be7d4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.556807  3431 tablet_replica.cc:333] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.557531  3431 raft_consensus.cc:2243] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.558614  3431 raft_consensus.cc:2272] T eace44731c5747ca98999cf217b561a3 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.561223  3431 tablet_replica.cc:333] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.561899  3431 raft_consensus.cc:2243] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.562713  3431 raft_consensus.cc:2272] T ab18688981ff456ca286ec171980c8de P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.565227  3431 tablet_replica.cc:333] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.566015  3431 raft_consensus.cc:2243] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.566884  3431 raft_consensus.cc:2272] T 6417cb26ca8d401f8e73d3512aeab50b P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.569409  3431 tablet_replica.cc:333] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.570077  3431 raft_consensus.cc:2243] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.570591  3431 raft_consensus.cc:2272] T b89a75dc9fa64c1c94a790754889c07a P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.573230  3431 tablet_replica.cc:333] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.574052  3431 raft_consensus.cc:2243] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.574553  3431 raft_consensus.cc:2272] T e72262c8e2b24a8fa97e1cacbb60b49a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.577124  3431 tablet_replica.cc:333] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.577937  3431 raft_consensus.cc:2243] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:16.578076 12447 consensus_peers.cc:597] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.578511  3431 raft_consensus.cc:2272] T 864a0424f5d64be0a7d4d8b5d38907c6 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.581169  3431 tablet_replica.cc:333] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.581720 13588 raft_consensus.cc:493] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.581880  3431 raft_consensus.cc:2243] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.582288 13588 raft_consensus.cc:515] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:16.582852  3431 raft_consensus.cc:2272] T 13100e49825f4fcfa78791b73b8d7555 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.584280 13588 leader_election.cc:290] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:16.585608  3431 tablet_replica.cc:333] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.586505  3431 raft_consensus.cc:2243] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.587517  3431 raft_consensus.cc:2272] T f5ae8840033347bea42f3c0ceb4d316a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:16.588186 12447 leader_election.cc:336] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:16.588728 12447 leader_election.cc:336] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:16.589092 12447 leader_election.cc:304] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:16.589727 13588 raft_consensus.cc:2749] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:16.590130  3431 tablet_replica.cc:333] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.590811  3431 raft_consensus.cc:2243] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.591260  3431 raft_consensus.cc:2272] T 228aa7f69257458ba0c2b5e6c950f217 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.593245  3431 tablet_replica.cc:333] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.593910  3431 raft_consensus.cc:2243] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.594543  3431 raft_consensus.cc:2272] T d0c8b89ad96240b588abe04f13ea2f9d P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.597323  3431 tablet_replica.cc:333] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.598148  3431 raft_consensus.cc:2243] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:16.598290 12373 consensus_peers.cc:597] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.598572  3431 raft_consensus.cc:2272] T 7ff454851114445da8be05eda167efc4 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.600857  3431 tablet_replica.cc:333] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.601426  3431 raft_consensus.cc:2243] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.602027  3431 raft_consensus.cc:2272] T fe640756185b42d7874674a29af747da P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.604916  3431 tablet_replica.cc:333] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.605710  3431 raft_consensus.cc:2243] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.606519  3431 raft_consensus.cc:2272] T 6a1cc112d1984521b5479f66b4289cce P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.606766 13588 raft_consensus.cc:493] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.607422 13588 raft_consensus.cc:515] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:16.608659  3431 tablet_replica.cc:333] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.609252 13588 leader_election.cc:290] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:16.609347  3431 raft_consensus.cc:2243] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.610168 13490 raft_consensus.cc:493] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.610675  3431 raft_consensus.cc:2272] T 164179fdd9ee45ff80ea3530e210088a P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.610745 13490 raft_consensus.cc:515] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:16.613257 13490 leader_election.cc:290] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:16.613721  3431 tablet_replica.cc:333] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.613829 12475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "307ee3a81a6649ebb96772cbd5e29f92" candidate_uuid: "4e02e5f693ce49a3b161ad90848be7d4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:16.614670  3431 raft_consensus.cc:2243] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.615235  3431 raft_consensus.cc:2272] T bf7c870391ab41aa99577da648d9ad60 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:16.616199 12447 leader_election.cc:336] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
W20260709 12:29:16.616986 12447 leader_election.cc:336] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:16.617374 12447 leader_election.cc:304] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:16.618000  3431 tablet_replica.cc:333] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
W20260709 12:29:16.618047 12299 leader_election.cc:336] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:16.618817 12299 leader_election.cc:304] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e02e5f693ce49a3b161ad90848be7d4; no voters: 1dcded10caa0493da818e67e13176979, a800ae68e91f43a794e326badca3320a
I20260709 12:29:16.619071  3431 raft_consensus.cc:2243] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.619731 13490 raft_consensus.cc:2749] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:16.618392 13588 raft_consensus.cc:2749] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:16.619777  3431 raft_consensus.cc:2272] T 1a81175bbecf4ba6b6be50b490564d9b P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:16.622725 12447 consensus_peers.cc:597] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.622972  3431 tablet_replica.cc:333] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.623705  3431 raft_consensus.cc:2243] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.624187  3431 raft_consensus.cc:2272] T dd19eddb6c2e45b7acefc5c295d92928 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.627665  3431 tablet_replica.cc:333] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.628435  3431 raft_consensus.cc:2243] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.629637  3431 raft_consensus.cc:2272] T 7916cfbec26b4115a9251411482bed41 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.631379 13586 raft_consensus.cc:493] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.632337  3431 tablet_replica.cc:333] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.631928 13586 raft_consensus.cc:515] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:16.633270  3431 raft_consensus.cc:2243] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.633947  3431 raft_consensus.cc:2272] T 1263de4693a84afe93a508aff7f1dc29 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.634721 13586 leader_election.cc:290] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:16.636956  3431 tablet_replica.cc:333] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.637948  3431 raft_consensus.cc:2243] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.638619  3431 raft_consensus.cc:2272] T 74e672f5fa0b4b68ad22e6fd8002d68d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:16.639889 12373 leader_election.cc:336] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
W20260709 12:29:16.640684 12373 leader_election.cc:336] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:16.641073 12373 leader_election.cc:304] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:16.641635  3431 tablet_replica.cc:333] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.642077 13586 raft_consensus.cc:2749] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:16.642612  3431 raft_consensus.cc:2243] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.643680  3431 raft_consensus.cc:2272] T 60d63f43ed8a4bedb35aa02c779fe4d0 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.646413  3431 tablet_replica.cc:333] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.647056  3431 raft_consensus.cc:2243] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.647850  3431 raft_consensus.cc:2272] T 485001bca51c4145939dc8bd38b1169a P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.650051  3431 tablet_replica.cc:333] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.650668  3431 raft_consensus.cc:2243] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.651175  3431 raft_consensus.cc:2272] T a6b87f00ff8c4e1c858037069e3bbb6e P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.653383  3431 tablet_replica.cc:333] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.654280  3431 raft_consensus.cc:2243] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.654757  3431 raft_consensus.cc:2272] T 93481f7df95d427eaa757fbbe87895d9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.657263  3431 tablet_replica.cc:333] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
W20260709 12:29:16.657591 12447 consensus_peers.cc:597] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.658174  3431 raft_consensus.cc:2243] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.660192  3431 raft_consensus.cc:2272] T 8d8f5c63c391477fad5c9155f5022b07 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.662741  3431 tablet_replica.cc:333] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.663383  3431 raft_consensus.cc:2243] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.663859  3431 raft_consensus.cc:2272] T d2a2e0c681f24264a729f89d08042fd9 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.666342  3431 tablet_replica.cc:333] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.667045  3431 raft_consensus.cc:2243] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.667665  3431 raft_consensus.cc:2272] T 2f01aed6d25447b0be01b95f4518c8ef P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:16.668324 12447 consensus_peers.cc:597] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.670472  3431 tablet_replica.cc:333] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.671273  3431 raft_consensus.cc:2243] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.671748  3431 raft_consensus.cc:2272] T 69e3b47e111a479f8582c270c1445658 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.674093  3431 tablet_replica.cc:333] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.674739  3431 raft_consensus.cc:2243] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.675336  3431 raft_consensus.cc:2272] T 05dfc5c361a94fe5af3e9a5462a0b12f P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.678033  3431 tablet_replica.cc:333] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.678903  3431 raft_consensus.cc:2243] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.679380  3431 raft_consensus.cc:2272] T c8d9482906bb4ca4a34c22c21f178cab P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.681459  3431 tablet_replica.cc:333] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.682113  3431 raft_consensus.cc:2243] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.682602  3431 raft_consensus.cc:2272] T 7540665883f74d94b96d1631ac30970d P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.683735 13586 raft_consensus.cc:493] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.684329 13586 raft_consensus.cc:515] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:16.684847  3431 tablet_replica.cc:333] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.685521  3431 raft_consensus.cc:2243] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.686098  3431 raft_consensus.cc:2272] T 87c6d482d9f1412a8f1ec924aa6d1222 P 4e02e5f693ce49a3b161ad90848be7d4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.686225 13586 leader_election.cc:290] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:16.687052 12475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a" is_pre_election: true
I20260709 12:29:16.687757 12475 raft_consensus.cc:2468] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 3.
I20260709 12:29:16.688843 12373 leader_election.cc:304] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 
I20260709 12:29:16.689147  3431 tablet_replica.cc:333] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.689113 13586 raft_consensus.cc:493] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.689654 13472 raft_consensus.cc:2804] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:16.690112 13472 raft_consensus.cc:493] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.690227  3431 raft_consensus.cc:2243] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.689881 13586 raft_consensus.cc:515] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:16.690528 13472 raft_consensus.cc:3060] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:16.690912  3431 raft_consensus.cc:2272] T 33683dafd0e7419e8f52bdf98e28e33c P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:16.691495 12373 leader_election.cc:336] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:16.692186 13586 leader_election.cc:290] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:16.694072  3431 tablet_replica.cc:333] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.694893  3431 raft_consensus.cc:2243] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.695461  3431 raft_consensus.cc:2272] T 3d163b2f24c845d8b11b35d222659673 P 4e02e5f693ce49a3b161ad90848be7d4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:16.696735 12373 leader_election.cc:336] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
W20260709 12:29:16.697734 12373 leader_election.cc:336] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:16.698135 12373 leader_election.cc:304] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:16.698331  3431 tablet_replica.cc:333] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.699239  3431 raft_consensus.cc:2243] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.698975 13472 raft_consensus.cc:515] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:16.699900  3431 raft_consensus.cc:2272] T 307ee3a81a6649ebb96772cbd5e29f92 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.701119 13472 leader_election.cc:290] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 election: Requested vote from peers a800ae68e91f43a794e326badca3320a (127.3.89.196:45677), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
I20260709 12:29:16.701143 13586 raft_consensus.cc:2749] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:16.702957  3431 tablet_replica.cc:333] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.703078 12475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce7f904b0c124a17881b5b725eaa1bdc" candidate_uuid: "a2104c8a9b144a419ccb7c07e3f02596" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a800ae68e91f43a794e326badca3320a"
I20260709 12:29:16.703709 12475 raft_consensus.cc:3060] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:16.703936  3431 raft_consensus.cc:2243] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.705106  3431 raft_consensus.cc:2272] T 0db639fdafc34f269b3d493540075616 P 4e02e5f693ce49a3b161ad90848be7d4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:16.705819 12373 consensus_peers.cc:597] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:16.708179 12373 leader_election.cc:336] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:16.708194  3431 tablet_replica.cc:333] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4: stopping tablet replica
I20260709 12:29:16.710727  3431 raft_consensus.cc:2243] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.712425  3431 raft_consensus.cc:2272] T b0b789be887a492fb0e51b08f2475a97 P 4e02e5f693ce49a3b161ad90848be7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.712615 12475 raft_consensus.cc:2468] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2104c8a9b144a419ccb7c07e3f02596 in term 4.
W20260709 12:29:16.715200 12373 consensus_peers.cc:597] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.715869 12373 leader_election.cc:304] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596, a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979
I20260709 12:29:16.716661 13586 raft_consensus.cc:2804] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:16.717267 13586 raft_consensus.cc:697] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 4 LEADER]: Becoming Leader. State: Replica: a2104c8a9b144a419ccb7c07e3f02596, State: Running, Role: LEADER
I20260709 12:29:16.718166 13586 consensus_queue.cc:237] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:16.740828 12156 catalog_manager.cc:5697] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 reported cstate change: term changed from 3 to 4, leader changed from 1dcded10caa0493da818e67e13176979 (127.3.89.193) to a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195). New cstate: current_term: 4 leader_uuid: "a2104c8a9b144a419ccb7c07e3f02596" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:16.768868 12447 consensus_peers.cc:597] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.774909 13586 raft_consensus.cc:493] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.775650 13586 raft_consensus.cc:515] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:16.777938 13586 leader_election.cc:290] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:29:16.782793 12373 leader_election.cc:336] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:16.783464 12373 leader_election.cc:336] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:16.783876 12373 leader_election.cc:304] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:16.784669 13586 raft_consensus.cc:2749] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:16.801681  3431 tablet_server.cc:196] TabletServer@127.3.89.194:0 shutdown complete.
I20260709 12:29:16.802896 13586 raft_consensus.cc:493] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.803527 13586 raft_consensus.cc:515] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:16.805001 13550 raft_consensus.cc:493] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.805506 13550 raft_consensus.cc:515] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:16.805886 13586 leader_election.cc:290] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:16.807690 13550 leader_election.cc:290] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:29:16.814379 12447 leader_election.cc:336] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
W20260709 12:29:16.815186 12447 leader_election.cc:336] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Network error: Client connection negotiation failed: client connection to 127.3.89.194:33269: connect: Connection refused (error 111)
I20260709 12:29:16.815582 12447 leader_election.cc:304] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
W20260709 12:29:16.815927 12373 leader_election.cc:336] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Network error: Client connection negotiation failed: client connection to 127.3.89.194:33269: connect: Connection refused (error 111)
I20260709 12:29:16.816311 13550 raft_consensus.cc:2749] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:29:16.817222 12373 leader_election.cc:336] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:16.817710 12373 leader_election.cc:304] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:16.818346 13586 raft_consensus.cc:2749] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:29:16.836153 12447 consensus_peers.cc:597] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:33269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:16.847705 12373 consensus_peers.cc:597] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:33269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.880084  3431 tablet_server.cc:179] TabletServer@127.3.89.195:0 shutting down...
W20260709 12:29:16.888688 12373 consensus_peers.cc:597] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:33269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:16.891599 12373 consensus_peers.cc:597] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:33269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:16.893841 12373 consensus_peers.cc:597] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 -> Peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Couldn't send request to peer 4e02e5f693ce49a3b161ad90848be7d4. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:33269: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:16.894744 13581 raft_consensus.cc:493] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.895323 13581 raft_consensus.cc:515] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:16.897509 13581 leader_election.cc:290] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
W20260709 12:29:16.904836 12447 leader_election.cc:336] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
W20260709 12:29:16.905541 12447 leader_election.cc:336] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Network error: Client connection negotiation failed: client connection to 127.3.89.194:33269: connect: Connection refused (error 111)
I20260709 12:29:16.906013 12447 leader_election.cc:304] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:16.906858 13581 raft_consensus.cc:2749] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:16.915688 13586 raft_consensus.cc:493] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.916288 13586 raft_consensus.cc:515] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:16.918256 13586 leader_election.cc:290] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:16.920389 13581 raft_consensus.cc:493] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.920989 13581 raft_consensus.cc:515] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } } peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } }
I20260709 12:29:16.923200 13581 leader_election.cc:290] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155), 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927)
W20260709 12:29:16.924470 12373 leader_election.cc:336] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
W20260709 12:29:16.925608 12373 leader_election.cc:336] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Network error: Client connection negotiation failed: client connection to 127.3.89.194:33269: connect: Connection refused (error 111)
I20260709 12:29:16.925987 12373 leader_election.cc:304] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
W20260709 12:29:16.926223 12450 leader_election.cc:336] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:16.926682 13586 raft_consensus.cc:2749] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:29:16.928514 12447 leader_election.cc:336] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:16.928953 12447 leader_election.cc:304] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, a2104c8a9b144a419ccb7c07e3f02596
I20260709 12:29:16.929550 13581 raft_consensus.cc:2749] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:16.929960  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:16.930948  3431 tablet_replica.cc:333] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.931974  3431 raft_consensus.cc:2243] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.932550  3431 raft_consensus.cc:2272] T 4fe65f1e2e7d4ae796006f23575cb681 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.935755  3431 tablet_replica.cc:333] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.936414  3431 raft_consensus.cc:2243] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.937139  3431 raft_consensus.cc:2272] T 75f9d02452bc41bbab1eb4cb7e1ccf44 P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.940023  3431 tablet_replica.cc:333] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.940670  3431 raft_consensus.cc:2243] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.941218  3431 raft_consensus.cc:2272] T d6682a011b3945c59d105bb6951d62f3 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.943305  3431 tablet_replica.cc:333] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.943903  3431 raft_consensus.cc:2243] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.944686  3431 raft_consensus.cc:2272] T 2ad2a742adea4bf989def5e758a3a81c P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.946748  3431 tablet_replica.cc:333] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.947431  3431 raft_consensus.cc:2243] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.947892  3431 raft_consensus.cc:2272] T f39a7bfe4bff44b18d1007ced129ae09 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.950035  3431 tablet_replica.cc:333] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.950626  3431 raft_consensus.cc:2243] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.951068  3431 raft_consensus.cc:2272] T b89a75dc9fa64c1c94a790754889c07a P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.953073  3431 tablet_replica.cc:333] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.953692  3431 raft_consensus.cc:2243] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.954126  3431 raft_consensus.cc:2272] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.956168  3431 tablet_replica.cc:333] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.956754  3431 raft_consensus.cc:2243] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.957154  3431 raft_consensus.cc:2272] T 436db7228d9d4e92a8472906005ae3fb P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.959679  3431 tablet_replica.cc:333] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.960373  3431 raft_consensus.cc:2243] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.960877  3431 raft_consensus.cc:2272] T eace44731c5747ca98999cf217b561a3 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.963294  3431 tablet_replica.cc:333] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.963929  3431 raft_consensus.cc:2243] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.964375  3431 raft_consensus.cc:2272] T 6417cb26ca8d401f8e73d3512aeab50b P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.965258 13596 raft_consensus.cc:493] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.965773 13596 raft_consensus.cc:515] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:16.966892  3431 tablet_replica.cc:333] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.967790  3431 raft_consensus.cc:2243] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.967942 13596 leader_election.cc:290] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:16.968372  3431 raft_consensus.cc:2272] T a03f6aed8fcb4ea2b45b6432634c00ae P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.971087  3431 tablet_replica.cc:333] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.972014  3431 raft_consensus.cc:2243] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:16.972157 12373 leader_election.cc:336] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:16.972613  3431 raft_consensus.cc:2272] T c67e469fc10a43bf9660c48b5ae1335a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:16.972961 12373 leader_election.cc:336] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Network error: Client connection negotiation failed: client connection to 127.3.89.194:33269: connect: Connection refused (error 111)
I20260709 12:29:16.973344 12373 leader_election.cc:304] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:16.974028 13596 raft_consensus.cc:2749] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:16.975185  3431 tablet_replica.cc:333] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.975862  3431 raft_consensus.cc:2243] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.976630  3431 raft_consensus.cc:2272] T 3d163b2f24c845d8b11b35d222659673 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.978704  3431 tablet_replica.cc:333] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.979270  3431 raft_consensus.cc:2243] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.979988  3431 raft_consensus.cc:2272] T d0c8b89ad96240b588abe04f13ea2f9d P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.981956  3431 tablet_replica.cc:333] T fe640756185b42d7874674a29af747da P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.982462  3431 raft_consensus.cc:2243] T fe640756185b42d7874674a29af747da P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.982841  3431 raft_consensus.cc:2272] T fe640756185b42d7874674a29af747da P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.984658  3431 tablet_replica.cc:333] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.985128  3431 raft_consensus.cc:2243] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.985932  3431 raft_consensus.cc:2272] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.988406  3431 tablet_replica.cc:333] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.988907  3431 raft_consensus.cc:2243] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.989343  3431 raft_consensus.cc:2272] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.991739  3431 tablet_replica.cc:333] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.992509  3431 raft_consensus.cc:2243] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.993263  3431 pending_rounds.cc:70] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596: Trying to abort 1 pending ops.
I20260709 12:29:16.993508  3431 pending_rounds.cc:77] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305626241934602240 op_type: NO_OP noop_request { }
I20260709 12:29:16.993927  3431 raft_consensus.cc:2889] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 4 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:16.994211  3431 raft_consensus.cc:2272] T ce7f904b0c124a17881b5b725eaa1bdc P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.996639  3431 tablet_replica.cc:333] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:16.997269  3431 raft_consensus.cc:2243] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.997802  3431 raft_consensus.cc:2272] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.998974 13581 raft_consensus.cc:493] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:16.999599 13581 raft_consensus.cc:515] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a800ae68e91f43a794e326badca3320a" member_type: VOTER last_known_addr { host: "127.3.89.196" port: 45677 } } peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } }
I20260709 12:29:17.000262  3431 tablet_replica.cc:333] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.001153  3431 raft_consensus.cc:2243] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.001726 13581 leader_election.cc:290] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:17.002161  3431 raft_consensus.cc:2272] T 74e672f5fa0b4b68ad22e6fd8002d68d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.004882  3431 tablet_replica.cc:333] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.005517  3431 raft_consensus.cc:2243] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:29:17.005924 12447 leader_election.cc:336] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:17.006604  3431 raft_consensus.cc:2272] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:17.006618 12447 leader_election.cc:336] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Network error: Client connection negotiation failed: client connection to 127.3.89.194:33269: connect: Connection refused (error 111)
I20260709 12:29:17.007186 12447 leader_election.cc:304] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a800ae68e91f43a794e326badca3320a; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:17.007871 13581 raft_consensus.cc:2749] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:17.009536  3431 tablet_replica.cc:333] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.010424  3431 raft_consensus.cc:2243] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.010980  3431 raft_consensus.cc:2272] T 8d8f5c63c391477fad5c9155f5022b07 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.013249  3431 tablet_replica.cc:333] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.013901  3431 raft_consensus.cc:2243] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.014325  3431 raft_consensus.cc:2272] T d2a2e0c681f24264a729f89d08042fd9 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.016439  3431 tablet_replica.cc:333] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.017009  3431 raft_consensus.cc:2243] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.017896  3431 raft_consensus.cc:2272] T 7cf8c3b9ee65415785e06fb8c085ada7 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.019758  3431 tablet_replica.cc:333] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.020334  3431 raft_consensus.cc:2243] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.020756  3431 raft_consensus.cc:2272] T bf7c870391ab41aa99577da648d9ad60 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.022667  3431 tablet_replica.cc:333] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.023195  3431 raft_consensus.cc:2243] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.023948  3431 raft_consensus.cc:2272] T 1a81175bbecf4ba6b6be50b490564d9b P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.026211  3431 tablet_replica.cc:333] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.026813  3431 raft_consensus.cc:2243] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.027318  3431 raft_consensus.cc:2272] T dd19eddb6c2e45b7acefc5c295d92928 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.029522  3431 tablet_replica.cc:333] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.030071  3431 raft_consensus.cc:2243] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.030510  3431 raft_consensus.cc:2272] T fd891675417c4265b77695b4227d980f P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:17.031087 12450 consensus_peers.cc:597] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:17.032672  3431 tablet_replica.cc:333] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.033195 13596 raft_consensus.cc:493] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1dcded10caa0493da818e67e13176979)
I20260709 12:29:17.033547  3431 raft_consensus.cc:2243] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.033725 13596 raft_consensus.cc:515] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dcded10caa0493da818e67e13176979" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37927 } } peers { permanent_uuid: "4e02e5f693ce49a3b161ad90848be7d4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33269 } } peers { permanent_uuid: "a2104c8a9b144a419ccb7c07e3f02596" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40155 } }
I20260709 12:29:17.034258  3431 raft_consensus.cc:2272] T 7916cfbec26b4115a9251411482bed41 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.035986 13596 leader_election.cc:290] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927), 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269)
I20260709 12:29:17.037194  3431 tablet_replica.cc:333] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.038084  3431 raft_consensus.cc:2243] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.039079  3431 raft_consensus.cc:2272] T 2f01aed6d25447b0be01b95f4518c8ef P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.041769  3431 tablet_replica.cc:333] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
W20260709 12:29:17.042447 12373 leader_election.cc:336] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1dcded10caa0493da818e67e13176979 (127.3.89.193:37927): Network error: Client connection negotiation failed: client connection to 127.3.89.193:37927: connect: Connection refused (error 111)
I20260709 12:29:17.042590  3431 raft_consensus.cc:2243] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:17.043269 12373 leader_election.cc:336] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4e02e5f693ce49a3b161ad90848be7d4 (127.3.89.194:33269): Network error: Client connection negotiation failed: client connection to 127.3.89.194:33269: connect: Connection refused (error 111)
W20260709 12:29:17.043251 12450 consensus_peers.cc:597] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:17.043244  3431 raft_consensus.cc:2272] T e94567d0d49d4bf0882ca910a8757cbc P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.043725 12373 leader_election.cc:304] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2104c8a9b144a419ccb7c07e3f02596; no voters: 1dcded10caa0493da818e67e13176979, 4e02e5f693ce49a3b161ad90848be7d4
I20260709 12:29:17.044421 13596 raft_consensus.cc:2749] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:17.046306  3431 tablet_replica.cc:333] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.047204  3431 raft_consensus.cc:2243] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.047789  3431 raft_consensus.cc:2272] T 69e3b47e111a479f8582c270c1445658 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.050354  3431 tablet_replica.cc:333] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.051151  3431 raft_consensus.cc:2243] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.051656  3431 raft_consensus.cc:2272] T c8d9482906bb4ca4a34c22c21f178cab P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:17.052338 12450 consensus_peers.cc:597] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:17.054319  3431 tablet_replica.cc:333] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.055115  3431 raft_consensus.cc:2243] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.055619  3431 raft_consensus.cc:2272] T 7540665883f74d94b96d1631ac30970d P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:17.056105 12450 consensus_peers.cc:597] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:17.058247  3431 tablet_replica.cc:333] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.059041  3431 raft_consensus.cc:2243] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.060021  3431 raft_consensus.cc:2272] T b6cc6fec74b6423bb49415fbcdd93e55 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.062744  3431 tablet_replica.cc:333] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.063419  3431 raft_consensus.cc:2243] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.063935  3431 raft_consensus.cc:2272] T 83d73b1bb20e4e79819072554a1ee076 P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.066380  3431 tablet_replica.cc:333] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.067060  3431 raft_consensus.cc:2243] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.068063  3431 raft_consensus.cc:2272] T 05dfc5c361a94fe5af3e9a5462a0b12f P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.070329  3431 tablet_replica.cc:333] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.070991  3431 raft_consensus.cc:2243] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.071919  3431 raft_consensus.cc:2272] T 8b89bcc99e094d1bbaf83979c2b78081 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.074028  3431 tablet_replica.cc:333] T 2e2871c5312642a28d3d3d65a5941a3a P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.074755  3431 raft_consensus.cc:2243] T 2e2871c5312642a28d3d3d65a5941a3a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.075350  3431 raft_consensus.cc:2272] T 2e2871c5312642a28d3d3d65a5941a3a P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.077396  3431 tablet_replica.cc:333] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.077962  3431 raft_consensus.cc:2243] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.078392  3431 raft_consensus.cc:2272] T 0db639fdafc34f269b3d493540075616 P a2104c8a9b144a419ccb7c07e3f02596 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.080245  3431 tablet_replica.cc:333] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.080765  3431 raft_consensus.cc:2243] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.081614  3431 raft_consensus.cc:2272] T 864a0424f5d64be0a7d4d8b5d38907c6 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.083521  3431 tablet_replica.cc:333] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.084008  3431 raft_consensus.cc:2243] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.084424  3431 raft_consensus.cc:2272] T 13100e49825f4fcfa78791b73b8d7555 P a2104c8a9b144a419ccb7c07e3f02596 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.086269  3431 tablet_replica.cc:333] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.086737  3431 raft_consensus.cc:2243] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.087143  3431 raft_consensus.cc:2272] T 87c6d482d9f1412a8f1ec924aa6d1222 P a2104c8a9b144a419ccb7c07e3f02596 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.088907  3431 tablet_replica.cc:333] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596: stopping tablet replica
I20260709 12:29:17.089341  3431 raft_consensus.cc:2243] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.090152  3431 raft_consensus.cc:2272] T 33683dafd0e7419e8f52bdf98e28e33c P a2104c8a9b144a419ccb7c07e3f02596 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:17.113140 12450 consensus_peers.cc:597] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:17.160979  3431 tablet_server.cc:196] TabletServer@127.3.89.195:0 shutdown complete.
W20260709 12:29:17.170468 12450 consensus_peers.cc:597] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:40155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:17.251185 12450 consensus_peers.cc:597] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:40155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:17.326366  3431 tablet_server.cc:179] TabletServer@127.3.89.196:0 shutting down...
W20260709 12:29:17.334012 12450 consensus_peers.cc:597] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:40155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:17.343081 12450 consensus_peers.cc:597] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:40155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:17.388046  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:17.389045  3431 tablet_replica.cc:333] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.390275  3431 raft_consensus.cc:2243] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.391283  3431 raft_consensus.cc:2272] T 4fe65f1e2e7d4ae796006f23575cb681 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.395011  3431 tablet_replica.cc:333] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.396428  3431 raft_consensus.cc:2243] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.398053  3431 raft_consensus.cc:2272] T fcca3076fc03410aa86e82c060c29335 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:17.400079 12450 consensus_peers.cc:597] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a -> Peer a2104c8a9b144a419ccb7c07e3f02596 (127.3.89.195:40155): Couldn't send request to peer a2104c8a9b144a419ccb7c07e3f02596. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.195:40155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:17.401758  3431 tablet_replica.cc:333] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.402711  3431 raft_consensus.cc:2243] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.404003  3431 raft_consensus.cc:2272] T f39a7bfe4bff44b18d1007ced129ae09 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.406617  3431 tablet_replica.cc:333] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.407356  3431 raft_consensus.cc:2243] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.407951  3431 raft_consensus.cc:2272] T 2ad2a742adea4bf989def5e758a3a81c P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.410612  3431 tablet_replica.cc:333] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.411412  3431 raft_consensus.cc:2243] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.412024  3431 raft_consensus.cc:2272] T d6682a011b3945c59d105bb6951d62f3 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.414685  3431 tablet_replica.cc:333] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.415421  3431 raft_consensus.cc:2243] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.416646  3431 raft_consensus.cc:2272] T a03f6aed8fcb4ea2b45b6432634c00ae P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.419637  3431 tablet_replica.cc:333] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.420635  3431 raft_consensus.cc:2243] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.421734  3431 raft_consensus.cc:2272] T 606a9d73d38c472698c0f88704e59b34 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.425803  3431 tablet_replica.cc:333] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.426539  3431 raft_consensus.cc:2243] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.427222  3431 raft_consensus.cc:2272] T 164179fdd9ee45ff80ea3530e210088a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.429701  3431 tablet_replica.cc:333] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.430339  3431 raft_consensus.cc:2243] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.430979  3431 raft_consensus.cc:2272] T 1a81175bbecf4ba6b6be50b490564d9b P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.434183  3431 tablet_replica.cc:333] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.435424  3431 raft_consensus.cc:2243] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.436197  3431 raft_consensus.cc:2272] T 76b0d4a9a29a4de9bc3a8ab529535d68 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.439824  3431 tablet_replica.cc:333] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.440838  3431 raft_consensus.cc:2243] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.442821  3431 raft_consensus.cc:2272] T 436db7228d9d4e92a8472906005ae3fb P a800ae68e91f43a794e326badca3320a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.446480  3431 tablet_replica.cc:333] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.447304  3431 raft_consensus.cc:2243] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.448189  3431 raft_consensus.cc:2272] T eace44731c5747ca98999cf217b561a3 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.452301  3431 tablet_replica.cc:333] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.453287  3431 raft_consensus.cc:2243] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.454416  3431 raft_consensus.cc:2272] T e72262c8e2b24a8fa97e1cacbb60b49a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.458277  3431 tablet_replica.cc:333] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.459288  3431 raft_consensus.cc:2243] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.460150  3431 raft_consensus.cc:2272] T ab18688981ff456ca286ec171980c8de P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.464644  3431 tablet_replica.cc:333] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.465773  3431 raft_consensus.cc:2243] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.468027  3431 raft_consensus.cc:2272] T c67e469fc10a43bf9660c48b5ae1335a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.471818  3431 tablet_replica.cc:333] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.472697  3431 raft_consensus.cc:2243] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.474345  3431 raft_consensus.cc:2272] T b0b789be887a492fb0e51b08f2475a97 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.478055  3431 tablet_replica.cc:333] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.479024  3431 raft_consensus.cc:2243] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.479842  3431 raft_consensus.cc:2272] T 7cf8c3b9ee65415785e06fb8c085ada7 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.483517  3431 tablet_replica.cc:333] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.484409  3431 raft_consensus.cc:2243] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.485217  3431 raft_consensus.cc:2272] T f5ae8840033347bea42f3c0ceb4d316a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.489091  3431 tablet_replica.cc:333] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.490056  3431 raft_consensus.cc:2243] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.491041  3431 raft_consensus.cc:2272] T f4bdda56dfcd42de9ca75e3e5a9b6c73 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.494693  3431 tablet_replica.cc:333] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.495657  3431 raft_consensus.cc:2243] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.497293  3431 raft_consensus.cc:2272] T 228aa7f69257458ba0c2b5e6c950f217 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.500495  3431 tablet_replica.cc:333] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.501448  3431 raft_consensus.cc:2243] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.502249  3431 raft_consensus.cc:2272] T 7ff454851114445da8be05eda167efc4 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.505400  3431 tablet_replica.cc:333] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.506281  3431 raft_consensus.cc:2243] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.507736  3431 raft_consensus.cc:2272] T fe640756185b42d7874674a29af747da P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.511317  3431 tablet_replica.cc:333] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.512388  3431 raft_consensus.cc:2243] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.513198  3431 raft_consensus.cc:2272] T 6a1cc112d1984521b5479f66b4289cce P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.516479  3431 tablet_replica.cc:333] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.517445  3431 raft_consensus.cc:2243] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.518205  3431 raft_consensus.cc:2272] T 8b89bcc99e094d1bbaf83979c2b78081 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.521270  3431 tablet_replica.cc:333] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.522022  3431 raft_consensus.cc:2243] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.523020  3431 raft_consensus.cc:2272] T 1263de4693a84afe93a508aff7f1dc29 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.525274  3431 tablet_replica.cc:333] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.525982  3431 raft_consensus.cc:2243] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.526539  3431 raft_consensus.cc:2272] T 74e672f5fa0b4b68ad22e6fd8002d68d P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.529222  3431 tablet_replica.cc:333] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.530068  3431 raft_consensus.cc:2243] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.530821  3431 raft_consensus.cc:2272] T ce7f904b0c124a17881b5b725eaa1bdc P a800ae68e91f43a794e326badca3320a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.533881  3431 tablet_replica.cc:333] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.534512  3431 raft_consensus.cc:2243] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.535125  3431 raft_consensus.cc:2272] T 60d63f43ed8a4bedb35aa02c779fe4d0 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.537477  3431 tablet_replica.cc:333] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.538174  3431 raft_consensus.cc:2243] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.538702  3431 raft_consensus.cc:2272] T 9aa6f87b452d4a7fbc6803c9fd74e9c3 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.540882  3431 tablet_replica.cc:333] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.541688  3431 raft_consensus.cc:2243] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.542883  3431 raft_consensus.cc:2272] T e94567d0d49d4bf0882ca910a8757cbc P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.545689  3431 tablet_replica.cc:333] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.546304  3431 raft_consensus.cc:2243] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.546813  3431 raft_consensus.cc:2272] T 81f0d8651a6144ca9fd21ccc0f57edd9 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.549016  3431 tablet_replica.cc:333] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.549721  3431 raft_consensus.cc:2243] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.550841  3431 raft_consensus.cc:2272] T d2a2e0c681f24264a729f89d08042fd9 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.553399  3431 tablet_replica.cc:333] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.554080  3431 raft_consensus.cc:2243] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.555274  3431 raft_consensus.cc:2272] T a6b87f00ff8c4e1c858037069e3bbb6e P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.557767  3431 tablet_replica.cc:333] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.558413  3431 raft_consensus.cc:2243] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.558955  3431 raft_consensus.cc:2272] T 93481f7df95d427eaa757fbbe87895d9 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.561192  3431 tablet_replica.cc:333] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.561863  3431 raft_consensus.cc:2243] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.562376  3431 raft_consensus.cc:2272] T 33683dafd0e7419e8f52bdf98e28e33c P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.565349  3431 tablet_replica.cc:333] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.566164  3431 raft_consensus.cc:2243] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.566941  3431 raft_consensus.cc:2272] T 3d163b2f24c845d8b11b35d222659673 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.570425  3431 tablet_replica.cc:333] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.571060  3431 raft_consensus.cc:2243] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.571878  3431 raft_consensus.cc:2272] T 307ee3a81a6649ebb96772cbd5e29f92 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.574483  3431 tablet_replica.cc:333] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.575085  3431 raft_consensus.cc:2243] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.576040  3431 raft_consensus.cc:2272] T fd891675417c4265b77695b4227d980f P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.578637  3431 tablet_replica.cc:333] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.579234  3431 raft_consensus.cc:2243] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.579715  3431 raft_consensus.cc:2272] T b6cc6fec74b6423bb49415fbcdd93e55 P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.582336  3431 tablet_replica.cc:333] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.583259  3431 raft_consensus.cc:2243] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.584615  3431 raft_consensus.cc:2272] T 83d73b1bb20e4e79819072554a1ee076 P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.587716  3431 tablet_replica.cc:333] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.588546  3431 raft_consensus.cc:2243] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.589291  3431 raft_consensus.cc:2272] T 485001bca51c4145939dc8bd38b1169a P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.592249  3431 tablet_replica.cc:333] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.592772  3431 raft_consensus.cc:2243] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.593760  3431 raft_consensus.cc:2272] T 2e2871c5312642a28d3d3d65a5941a3a P a800ae68e91f43a794e326badca3320a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.596176  3431 tablet_replica.cc:333] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.596675  3431 raft_consensus.cc:2243] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.597144  3431 raft_consensus.cc:2272] T 0db639fdafc34f269b3d493540075616 P a800ae68e91f43a794e326badca3320a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.599309  3431 tablet_replica.cc:333] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a: stopping tablet replica
I20260709 12:29:17.599759  3431 raft_consensus.cc:2243] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:17.600195  3431 raft_consensus.cc:2272] T 2f01aed6d25447b0be01b95f4518c8ef P a800ae68e91f43a794e326badca3320a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.676581  3431 tablet_server.cc:196] TabletServer@127.3.89.196:0 shutdown complete.
I20260709 12:29:17.751060  3431 master.cc:562] Master@127.3.89.254:45675 shutting down...
I20260709 12:29:17.783497  3431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:17.784215  3431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:17.784615  3431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6e9b094ea1e7401993ec5ad7a7e427d6: stopping tablet replica
I20260709 12:29:17.809969  3431 master.cc:584] Master@127.3.89.254:45675 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (32829 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:29:17.911608  3431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.3.89.254:33703
I20260709 12:29:17.913015  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:17.920239 13600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:17.920279 13601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:17.920776 13603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:17.922261  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:17.923435  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:17.923668  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:17.923848  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600157923834 us; error 0 us; skew 500 ppm
I20260709 12:29:17.924435  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:17.927366  3431 webserver.cc:533] Webserver started at http://127.3.89.254:40445/ using document root <none> and password file <none>
I20260709 12:29:17.927893  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:17.928088  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:17.928399  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:17.929788  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/master-0-root/instance:
uuid: "44291da4ca1944c699c48c997f26cfdc"
format_stamp: "Formatted at 2026-07-09 12:29:17 on dist-test-slave-3hrd"
I20260709 12:29:17.934628  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:29:17.938692 13608 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:17.939630  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:29:17.939893  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/master-0-root
uuid: "44291da4ca1944c699c48c997f26cfdc"
format_stamp: "Formatted at 2026-07-09 12:29:17 on dist-test-slave-3hrd"
I20260709 12:29:17.940145  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:17.958189  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:17.959410  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:18.000587  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:33703
I20260709 12:29:18.000699 13659 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.254:33703 every 8 connection(s)
I20260709 12:29:18.005422 13660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.017525 13660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc: Bootstrap starting.
I20260709 12:29:18.022625 13660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.027292 13660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc: No bootstrap required, opened a new log
I20260709 12:29:18.029865 13660 raft_consensus.cc:359] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44291da4ca1944c699c48c997f26cfdc" member_type: VOTER }
I20260709 12:29:18.030617 13660 raft_consensus.cc:385] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.030997 13660 raft_consensus.cc:740] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44291da4ca1944c699c48c997f26cfdc, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.031754 13660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44291da4ca1944c699c48c997f26cfdc" member_type: VOTER }
I20260709 12:29:18.032306 13660 raft_consensus.cc:399] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:18.032603 13660 raft_consensus.cc:493] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:18.032931 13660 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:18.038566 13660 raft_consensus.cc:515] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44291da4ca1944c699c48c997f26cfdc" member_type: VOTER }
I20260709 12:29:18.039227 13660 leader_election.cc:304] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44291da4ca1944c699c48c997f26cfdc; no voters: 
I20260709 12:29:18.040689 13660 leader_election.cc:290] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:18.041025 13663 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:18.042541 13663 raft_consensus.cc:697] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [term 1 LEADER]: Becoming Leader. State: Replica: 44291da4ca1944c699c48c997f26cfdc, State: Running, Role: LEADER
I20260709 12:29:18.043248 13663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44291da4ca1944c699c48c997f26cfdc" member_type: VOTER }
I20260709 12:29:18.044026 13660 sys_catalog.cc:565] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:18.049965 13665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44291da4ca1944c699c48c997f26cfdc. Latest consensus state: current_term: 1 leader_uuid: "44291da4ca1944c699c48c997f26cfdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44291da4ca1944c699c48c997f26cfdc" member_type: VOTER } }
I20260709 12:29:18.050657 13665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:18.050676 13664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44291da4ca1944c699c48c997f26cfdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44291da4ca1944c699c48c997f26cfdc" member_type: VOTER } }
I20260709 12:29:18.051690 13664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:18.052757 13671 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:18.059444 13671 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:18.060571  3431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:18.069289 13671 catalog_manager.cc:1383] Generated new cluster ID: e96d7355490748e687d694ed1bbeec7d
I20260709 12:29:18.069667 13671 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:18.097751 13671 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:18.099068 13671 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:18.113669 13671 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc: Generated new TSK 0
I20260709 12:29:18.114302 13671 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:18.127298  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:18.134130 13681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:18.135028 13682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:18.136741 13684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:18.137981  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:18.139017  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:18.139196  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:18.139407  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600158139388 us; error 0 us; skew 500 ppm
I20260709 12:29:18.139973  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:18.142445  3431 webserver.cc:533] Webserver started at http://127.3.89.193:35197/ using document root <none> and password file <none>
I20260709 12:29:18.142971  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:18.143136  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:18.143463  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:18.144665  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-0-root/instance:
uuid: "f6bb736605e6460e91c7b8a131e0e84b"
format_stamp: "Formatted at 2026-07-09 12:29:18 on dist-test-slave-3hrd"
I20260709 12:29:18.149343  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:29:18.152808 13689 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:18.153733  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:29:18.154058  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-0-root
uuid: "f6bb736605e6460e91c7b8a131e0e84b"
format_stamp: "Formatted at 2026-07-09 12:29:18 on dist-test-slave-3hrd"
I20260709 12:29:18.154490  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:18.163780  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:18.165072  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:18.166607  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:18.168921  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:18.169153  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:18.169418  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:18.169626  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:29:18.218287  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:45845
I20260709 12:29:18.218423 13751 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.193:45845 every 8 connection(s)
I20260709 12:29:18.223654  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:18.232316 13756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:18.233182 13757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:18.236390 13759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:18.236788  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:18.238320  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:18.238587  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:18.238783  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600158238764 us; error 0 us; skew 500 ppm
I20260709 12:29:18.239416  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:18.242911  3431 webserver.cc:533] Webserver started at http://127.3.89.194:36121/ using document root <none> and password file <none>
I20260709 12:29:18.243489  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:18.243717  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:18.243996  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:18.244375 13752 heartbeater.cc:344] Connected to a master server at 127.3.89.254:33703
I20260709 12:29:18.244818 13752 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:18.245426  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-1-root/instance:
uuid: "d6045f6899cf43f7aa7af074983fcae8"
format_stamp: "Formatted at 2026-07-09 12:29:18 on dist-test-slave-3hrd"
I20260709 12:29:18.245708 13752 heartbeater.cc:507] Master 127.3.89.254:33703 requested a full tablet report, sending...
I20260709 12:29:18.248065 13625 ts_manager.cc:194] Registered new tserver with Master: f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845)
I20260709 12:29:18.250308 13625 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43784
I20260709 12:29:18.251159  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:29:18.256067 13764 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:18.257037  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:29:18.257396  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-1-root
uuid: "d6045f6899cf43f7aa7af074983fcae8"
format_stamp: "Formatted at 2026-07-09 12:29:18 on dist-test-slave-3hrd"
I20260709 12:29:18.257790  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:18.304924  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:18.306289  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:18.307866  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:18.310189  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:18.310437  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:18.310700  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:18.310878  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:18.354668  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:40121
I20260709 12:29:18.354766 13826 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.194:40121 every 8 connection(s)
I20260709 12:29:18.360004  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:18.368499 13830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:18.370663 13831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:18.374390 13827 heartbeater.cc:344] Connected to a master server at 127.3.89.254:33703
I20260709 12:29:18.375031 13827 heartbeater.cc:461] Registering TS with master...
W20260709 12:29:18.375640 13833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:18.375957  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:18.376185 13827 heartbeater.cc:507] Master 127.3.89.254:33703 requested a full tablet report, sending...
I20260709 12:29:18.377074  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:18.377318  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:18.377527  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600158377503 us; error 0 us; skew 500 ppm
I20260709 12:29:18.378201  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:18.378909 13625 ts_manager.cc:194] Registered new tserver with Master: d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:18.380777 13625 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43798
I20260709 12:29:18.381191  3431 webserver.cc:533] Webserver started at http://127.3.89.195:46827/ using document root <none> and password file <none>
I20260709 12:29:18.381847  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:18.382076  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:18.382362  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:18.383965  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-2-root/instance:
uuid: "0a1b5a26007a4097887ffb7b875380af"
format_stamp: "Formatted at 2026-07-09 12:29:18 on dist-test-slave-3hrd"
I20260709 12:29:18.389257  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:29:18.393937 13838 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:18.394937  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:29:18.395282  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-2-root
uuid: "0a1b5a26007a4097887ffb7b875380af"
format_stamp: "Formatted at 2026-07-09 12:29:18 on dist-test-slave-3hrd"
I20260709 12:29:18.395617  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:18.418123  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:18.419368  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:18.420997  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:18.423446  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:18.423686  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:18.423949  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:18.424139  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:18.469772  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:40343
I20260709 12:29:18.469874 13900 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.195:40343 every 8 connection(s)
I20260709 12:29:18.484725 13901 heartbeater.cc:344] Connected to a master server at 127.3.89.254:33703
I20260709 12:29:18.485229 13901 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:18.486030 13901 heartbeater.cc:507] Master 127.3.89.254:33703 requested a full tablet report, sending...
I20260709 12:29:18.488113 13625 ts_manager.cc:194] Registered new tserver with Master: 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:18.489156  3431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016071705s
I20260709 12:29:18.489624 13625 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43802
I20260709 12:29:18.517601 13625 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43814:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:18.520576 13625 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:18.599602 13717 tablet_service.cc:1511] Processing CreateTablet for tablet c762518c0d2c453a97841958622ffe38 (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:29:18.601176 13717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c762518c0d2c453a97841958622ffe38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.601516 13866 tablet_service.cc:1511] Processing CreateTablet for tablet c762518c0d2c453a97841958622ffe38 (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:29:18.602968 13866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c762518c0d2c453a97841958622ffe38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.606706 13716 tablet_service.cc:1511] Processing CreateTablet for tablet 20cf0dad70be47f18b0d83766741777a (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:29:18.607215 13864 tablet_service.cc:1511] Processing CreateTablet for tablet d21be6a87739412694ceb7ae4cf3e42d (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:29:18.608253 13716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20cf0dad70be47f18b0d83766741777a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.605283 13865 tablet_service.cc:1511] Processing CreateTablet for tablet 20cf0dad70be47f18b0d83766741777a (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:29:18.610584 13864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d21be6a87739412694ceb7ae4cf3e42d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.610788 13715 tablet_service.cc:1511] Processing CreateTablet for tablet d21be6a87739412694ceb7ae4cf3e42d (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:29:18.612304 13715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d21be6a87739412694ceb7ae4cf3e42d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.613598 13865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20cf0dad70be47f18b0d83766741777a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.615859 13714 tablet_service.cc:1511] Processing CreateTablet for tablet c6bf9b18c2d047babed6c7b74b9dbf01 (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:29:18.616124 13862 tablet_service.cc:1511] Processing CreateTablet for tablet 46731d8225284b3c85730af89cb0a871 (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:29:18.617691 13862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46731d8225284b3c85730af89cb0a871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.618243 13713 tablet_service.cc:1511] Processing CreateTablet for tablet 46731d8225284b3c85730af89cb0a871 (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:29:18.606612 13863 tablet_service.cc:1511] Processing CreateTablet for tablet c6bf9b18c2d047babed6c7b74b9dbf01 (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:29:18.620138 13863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6bf9b18c2d047babed6c7b74b9dbf01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.623090 13712 tablet_service.cc:1511] Processing CreateTablet for tablet c5cea694c7104f96a2431e7015a54edd (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:29:18.623127 13861 tablet_service.cc:1511] Processing CreateTablet for tablet c5cea694c7104f96a2431e7015a54edd (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:29:18.625787 13861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5cea694c7104f96a2431e7015a54edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.627306 13711 tablet_service.cc:1511] Processing CreateTablet for tablet f95b4e77affd440fafced08a62b6135d (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:29:18.629616 13860 tablet_service.cc:1511] Processing CreateTablet for tablet f95b4e77affd440fafced08a62b6135d (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:29:18.630470 13713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46731d8225284b3c85730af89cb0a871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.630968 13860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f95b4e77affd440fafced08a62b6135d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.632056 13711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f95b4e77affd440fafced08a62b6135d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.636253 13712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5cea694c7104f96a2431e7015a54edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.637912 13714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6bf9b18c2d047babed6c7b74b9dbf01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.634058 13710 tablet_service.cc:1511] Processing CreateTablet for tablet a52eaa3c1df34e6c8a1ccea63bf39b1d (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:29:18.640105 13710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a52eaa3c1df34e6c8a1ccea63bf39b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.674059 13859 tablet_service.cc:1511] Processing CreateTablet for tablet a52eaa3c1df34e6c8a1ccea63bf39b1d (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:29:18.675657 13859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a52eaa3c1df34e6c8a1ccea63bf39b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.680727 13858 tablet_service.cc:1511] Processing CreateTablet for tablet 7ade9f433393457391d1bc7f19b224de (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:29:18.682252 13858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ade9f433393457391d1bc7f19b224de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.686481 13857 tablet_service.cc:1511] Processing CreateTablet for tablet 005c038223964ceb88b90a7698bc658e (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:29:18.687953 13857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 005c038223964ceb88b90a7698bc658e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.689757 13709 tablet_service.cc:1511] Processing CreateTablet for tablet 7ade9f433393457391d1bc7f19b224de (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:29:18.693279 13709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ade9f433393457391d1bc7f19b224de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.699013 13792 tablet_service.cc:1511] Processing CreateTablet for tablet c762518c0d2c453a97841958622ffe38 (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:29:18.700582 13792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c762518c0d2c453a97841958622ffe38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.691776 13708 tablet_service.cc:1511] Processing CreateTablet for tablet 005c038223964ceb88b90a7698bc658e (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:29:18.705152 13708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 005c038223964ceb88b90a7698bc658e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.711742 13790 tablet_service.cc:1511] Processing CreateTablet for tablet d21be6a87739412694ceb7ae4cf3e42d (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:29:18.713752 13790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d21be6a87739412694ceb7ae4cf3e42d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.711742 13791 tablet_service.cc:1511] Processing CreateTablet for tablet 20cf0dad70be47f18b0d83766741777a (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:29:18.719808 13791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20cf0dad70be47f18b0d83766741777a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.723479 13789 tablet_service.cc:1511] Processing CreateTablet for tablet c6bf9b18c2d047babed6c7b74b9dbf01 (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:29:18.724968 13789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6bf9b18c2d047babed6c7b74b9dbf01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.724453 13788 tablet_service.cc:1511] Processing CreateTablet for tablet 46731d8225284b3c85730af89cb0a871 (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:29:18.727247 13788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46731d8225284b3c85730af89cb0a871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.730257 13787 tablet_service.cc:1511] Processing CreateTablet for tablet c5cea694c7104f96a2431e7015a54edd (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:29:18.732263 13787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5cea694c7104f96a2431e7015a54edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.773299 13786 tablet_service.cc:1511] Processing CreateTablet for tablet f95b4e77affd440fafced08a62b6135d (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:29:18.775393 13786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f95b4e77affd440fafced08a62b6135d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.807276 13785 tablet_service.cc:1511] Processing CreateTablet for tablet a52eaa3c1df34e6c8a1ccea63bf39b1d (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:29:18.808355 13783 tablet_service.cc:1511] Processing CreateTablet for tablet 005c038223964ceb88b90a7698bc658e (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:29:18.808883 13785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a52eaa3c1df34e6c8a1ccea63bf39b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.809772 13783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 005c038223964ceb88b90a7698bc658e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.811214 13784 tablet_service.cc:1511] Processing CreateTablet for tablet 7ade9f433393457391d1bc7f19b224de (DEFAULT_TABLE table=test-workload [id=344cca5bdea84cb285572bf5ca08cc69]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:29:18.812644 13784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ade9f433393457391d1bc7f19b224de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:18.832448 13916 tablet_bootstrap.cc:492] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af: Bootstrap starting.
I20260709 12:29:18.860189 13917 tablet_bootstrap.cc:492] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:18.868592 13916 tablet_bootstrap.cc:654] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.901942 13917 tablet_bootstrap.cc:654] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.903863 13916 tablet_bootstrap.cc:492] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af: No bootstrap required, opened a new log
I20260709 12:29:18.904482 13916 ts_tablet_manager.cc:1403] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af: Time spent bootstrapping tablet: real 0.072s	user 0.009s	sys 0.010s
I20260709 12:29:18.905238 13918 tablet_bootstrap.cc:492] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8: Bootstrap starting.
I20260709 12:29:18.908056 13916 raft_consensus.cc:359] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:18.908604 13917 tablet_bootstrap.cc:492] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b: No bootstrap required, opened a new log
I20260709 12:29:18.908874 13916 raft_consensus.cc:385] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.909163 13917 ts_tablet_manager.cc:1403] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.049s	user 0.008s	sys 0.006s
I20260709 12:29:18.909192 13916 raft_consensus.cc:740] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.910215 13916 consensus_queue.cc:260] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:18.911643 13918 tablet_bootstrap.cc:654] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.912724 13901 heartbeater.cc:499] Master 127.3.89.254:33703 was elected leader, sending a full tablet report...
I20260709 12:29:18.913084 13916 ts_tablet_manager.cc:1434] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:29:18.912694 13917 raft_consensus.cc:359] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:18.914199 13916 tablet_bootstrap.cc:492] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af: Bootstrap starting.
I20260709 12:29:18.914456 13917 raft_consensus.cc:385] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.914911 13917 raft_consensus.cc:740] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.916224 13917 consensus_queue.cc:260] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:18.921288 13916 tablet_bootstrap.cc:654] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.924891 13918 tablet_bootstrap.cc:492] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8: No bootstrap required, opened a new log
I20260709 12:29:18.925770 13918 ts_tablet_manager.cc:1403] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.012s
I20260709 12:29:18.930164 13918 raft_consensus.cc:359] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:18.932346 13918 raft_consensus.cc:385] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.932535 13752 heartbeater.cc:499] Master 127.3.89.254:33703 was elected leader, sending a full tablet report...
I20260709 12:29:18.932752 13918 raft_consensus.cc:740] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.932972 13917 ts_tablet_manager.cc:1434] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.023s	user 0.013s	sys 0.007s
I20260709 12:29:18.934336 13917 tablet_bootstrap.cc:492] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:18.934010 13918 consensus_queue.cc:260] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:18.942812 13917 tablet_bootstrap.cc:654] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.943578 13918 ts_tablet_manager.cc:1434] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8: Time spent starting tablet: real 0.017s	user 0.001s	sys 0.013s
I20260709 12:29:18.944445 13827 heartbeater.cc:499] Master 127.3.89.254:33703 was elected leader, sending a full tablet report...
I20260709 12:29:18.956369 13918 tablet_bootstrap.cc:492] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8: Bootstrap starting.
I20260709 12:29:18.968662 13918 tablet_bootstrap.cc:654] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:18.973107 13753 tablet.cc:2406] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:29:18.976434 13917 tablet_bootstrap.cc:492] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b: No bootstrap required, opened a new log
I20260709 12:29:18.976428 13916 tablet_bootstrap.cc:492] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af: No bootstrap required, opened a new log
I20260709 12:29:18.977514 13917 ts_tablet_manager.cc:1403] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.043s	user 0.019s	sys 0.020s
I20260709 12:29:18.977593 13916 ts_tablet_manager.cc:1403] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af: Time spent bootstrapping tablet: real 0.064s	user 0.023s	sys 0.033s
I20260709 12:29:18.981439 13916 raft_consensus.cc:359] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:18.982388 13916 raft_consensus.cc:385] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.982038 13917 raft_consensus.cc:359] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:18.982820 13916 raft_consensus.cc:740] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.983300 13917 raft_consensus.cc:385] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.984627 13917 raft_consensus.cc:740] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.985260 13916 consensus_queue.cc:260] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:18.985893 13917 consensus_queue.cc:260] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:18.987671 13916 ts_tablet_manager.cc:1434] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:29:18.989087 13916 tablet_bootstrap.cc:492] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af: Bootstrap starting.
I20260709 12:29:18.989769 13917 ts_tablet_manager.cc:1434] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 12:29:18.991398 13917 tablet_bootstrap.cc:492] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:18.993494 13918 tablet_bootstrap.cc:492] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8: No bootstrap required, opened a new log
I20260709 12:29:18.994195 13918 ts_tablet_manager.cc:1403] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.019s
I20260709 12:29:18.996580 13916 tablet_bootstrap.cc:654] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.997551 13918 raft_consensus.cc:359] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:18.998487 13918 raft_consensus.cc:385] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:18.998070 13917 tablet_bootstrap.cc:654] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:18.998862 13918 raft_consensus.cc:740] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Initialized, Role: FOLLOWER
I20260709 12:29:18.999842 13918 consensus_queue.cc:260] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.002544 13918 ts_tablet_manager.cc:1434] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:29:19.003773 13918 tablet_bootstrap.cc:492] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8: Bootstrap starting.
I20260709 12:29:19.010953 13917 tablet_bootstrap.cc:492] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b: No bootstrap required, opened a new log
I20260709 12:29:19.011663 13917 ts_tablet_manager.cc:1403] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 12:29:19.012868 13918 tablet_bootstrap.cc:654] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.014565 13923 raft_consensus.cc:493] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.014998 13917 raft_consensus.cc:359] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.015813 13917 raft_consensus.cc:385] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.015362 13923 raft_consensus.cc:515] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.025743 13923 leader_election.cc:290] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.025704 13918 tablet_bootstrap.cc:492] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8: No bootstrap required, opened a new log
I20260709 12:29:19.027006 13918 ts_tablet_manager.cc:1403] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.011s
I20260709 12:29:19.027417 13917 raft_consensus.cc:740] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.028065 13916 tablet_bootstrap.cc:492] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af: No bootstrap required, opened a new log
I20260709 12:29:19.028303 13917 consensus_queue.cc:260] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.030258 13918 raft_consensus.cc:359] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.031472 13918 raft_consensus.cc:385] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.031816 13918 raft_consensus.cc:740] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.028972 13916 ts_tablet_manager.cc:1403] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af: Time spent bootstrapping tablet: real 0.040s	user 0.021s	sys 0.011s
I20260709 12:29:19.032869 13918 consensus_queue.cc:260] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.035563 13918 ts_tablet_manager.cc:1434] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:19.031890 13917 ts_tablet_manager.cc:1434] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:29:19.035706 13916 raft_consensus.cc:359] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.036751 13916 raft_consensus.cc:385] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.036904 13917 tablet_bootstrap.cc:492] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:19.037178 13916 raft_consensus.cc:740] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.038367 13916 consensus_queue.cc:260] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.045249 13918 tablet_bootstrap.cc:492] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8: Bootstrap starting.
I20260709 12:29:19.046784 13916 ts_tablet_manager.cc:1434] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260709 12:29:19.048163 13916 tablet_bootstrap.cc:492] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af: Bootstrap starting.
I20260709 12:29:19.048427 13917 tablet_bootstrap.cc:654] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.054436 13918 tablet_bootstrap.cc:654] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.060776 13916 tablet_bootstrap.cc:654] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.060703 13917 tablet_bootstrap.cc:492] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: No bootstrap required, opened a new log
I20260709 12:29:19.061386 13876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21be6a87739412694ceb7ae4cf3e42d" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
I20260709 12:29:19.061954 13917 ts_tablet_manager.cc:1403] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.008s
I20260709 12:29:19.062434 13876 raft_consensus.cc:2468] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 0.
I20260709 12:29:19.063001 13925 raft_consensus.cc:493] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:19.055619 13673 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:19.063488 13925 raft_consensus.cc:515] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.064491 13690 leader_election.cc:304] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:19.065115 13917 raft_consensus.cc:359] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.065521 13923 raft_consensus.cc:2804] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.066030 13923 raft_consensus.cc:493] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.066375 13923 raft_consensus.cc:3060] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.066414 13925 leader_election.cc:290] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.067035 13917 raft_consensus.cc:385] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.067418 13917 raft_consensus.cc:740] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.075768 13917 consensus_queue.cc:260] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.091753 13923 raft_consensus.cc:515] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.093722 13921 raft_consensus.cc:493] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.094403 13921 raft_consensus.cc:515] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:19.098006 13917 ts_tablet_manager.cc:1434] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.003s
I20260709 12:29:19.102628 13921 leader_election.cc:290] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:19.104053 13923 leader_election.cc:290] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 election: Requested vote from peers d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.104782 13917 tablet_bootstrap.cc:492] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:19.111052 13876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21be6a87739412694ceb7ae4cf3e42d" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:19.112052 13876 raft_consensus.cc:3060] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.112990 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21be6a87739412694ceb7ae4cf3e42d" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:19.114012 13802 raft_consensus.cc:2468] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 0.
I20260709 12:29:19.116045 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21be6a87739412694ceb7ae4cf3e42d" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:19.116762 13802 raft_consensus.cc:3060] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.131788 13876 raft_consensus.cc:2468] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 1.
I20260709 12:29:19.132653 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c762518c0d2c453a97841958622ffe38" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:19.132929 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf9b18c2d047babed6c7b74b9dbf01" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
I20260709 12:29:19.134407 13690 leader_election.cc:304] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:19.135282 13923 raft_consensus.cc:2804] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:19.135788 13923 raft_consensus.cc:697] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: Becoming Leader. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Running, Role: LEADER
I20260709 12:29:19.136629 13923 consensus_queue.cc:237] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.137243 13917 tablet_bootstrap.cc:654] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:19.141382 13842 leader_election.cc:343] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:19.142556 13765 leader_election.cc:343] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.145455 13918 tablet_bootstrap.cc:492] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8: No bootstrap required, opened a new log
I20260709 12:29:19.149288 13918 ts_tablet_manager.cc:1403] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8: Time spent bootstrapping tablet: real 0.104s	user 0.022s	sys 0.008s
I20260709 12:29:19.149137 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf9b18c2d047babed6c7b74b9dbf01" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b" is_pre_election: true
I20260709 12:29:19.149559 13802 raft_consensus.cc:2468] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 1.
I20260709 12:29:19.151015 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c762518c0d2c453a97841958622ffe38" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b" is_pre_election: true
I20260709 12:29:19.152395 13917 tablet_bootstrap.cc:492] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b: No bootstrap required, opened a new log
I20260709 12:29:19.152593 13918 raft_consensus.cc:359] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
W20260709 12:29:19.152778 13841 leader_election.cc:343] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.147073 13625 catalog_manager.cc:5697] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b reported cstate change: term changed from 0 to 1, leader changed from <none> to f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193). New cstate: current_term: 1 leader_uuid: "f6bb736605e6460e91c7b8a131e0e84b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:19.153393 13918 raft_consensus.cc:385] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:29:19.152834 13767 leader_election.cc:343] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.153669 13841 leader_election.cc:304] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af; no voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b
I20260709 12:29:19.153885 13918 raft_consensus.cc:740] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.154057 13767 leader_election.cc:304] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8; no voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b
I20260709 12:29:19.154917 13925 raft_consensus.cc:2749] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.154852 13918 consensus_queue.cc:260] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.157047 13918 ts_tablet_manager.cc:1434] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:29:19.146891 13916 tablet_bootstrap.cc:492] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af: No bootstrap required, opened a new log
I20260709 12:29:19.152992 13917 ts_tablet_manager.cc:1403] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.054s	user 0.011s	sys 0.002s
I20260709 12:29:19.157748 13916 ts_tablet_manager.cc:1403] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af: Time spent bootstrapping tablet: real 0.110s	user 0.024s	sys 0.010s
I20260709 12:29:19.155057 13921 raft_consensus.cc:2749] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.160187 13918 tablet_bootstrap.cc:492] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8: Bootstrap starting.
I20260709 12:29:19.161968 13917 raft_consensus.cc:359] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.162806 13917 raft_consensus.cc:385] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.163134 13917 raft_consensus.cc:740] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.164050 13917 consensus_queue.cc:260] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.166859 13917 ts_tablet_manager.cc:1434] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:29:19.167929 13917 tablet_bootstrap.cc:492] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:19.172392 13916 raft_consensus.cc:359] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.173250 13916 raft_consensus.cc:385] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.173632 13916 raft_consensus.cc:740] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.173842 13918 tablet_bootstrap.cc:654] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.174472 13917 tablet_bootstrap.cc:654] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.174805 13916 consensus_queue.cc:260] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.187505 13916 ts_tablet_manager.cc:1434] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.003s
I20260709 12:29:19.192176 13923 raft_consensus.cc:493] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.192394 13918 tablet_bootstrap.cc:492] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8: No bootstrap required, opened a new log
I20260709 12:29:19.192845 13923 raft_consensus.cc:515] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.193432 13918 ts_tablet_manager.cc:1403] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.015s
I20260709 12:29:19.193771 13916 tablet_bootstrap.cc:492] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af: Bootstrap starting.
I20260709 12:29:19.195219 13923 leader_election.cc:290] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.197003 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "005c038223964ceb88b90a7698bc658e" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
I20260709 12:29:19.197225 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "005c038223964ceb88b90a7698bc658e" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
W20260709 12:29:19.199455 13690 leader_election.cc:343] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:19.202826 13693 leader_election.cc:343] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.203464 13693 leader_election.cc:304] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6bb736605e6460e91c7b8a131e0e84b; no voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8
I20260709 12:29:19.204535 13923 raft_consensus.cc:2749] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.209777 13918 raft_consensus.cc:359] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.210656 13918 raft_consensus.cc:385] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.210949 13918 raft_consensus.cc:740] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.211429 13916 tablet_bootstrap.cc:654] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.211783 13918 consensus_queue.cc:260] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.213665 13917 tablet_bootstrap.cc:492] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: No bootstrap required, opened a new log
I20260709 12:29:19.214286 13917 ts_tablet_manager.cc:1403] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.026s
I20260709 12:29:19.214679 13918 ts_tablet_manager.cc:1434] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.005s
I20260709 12:29:19.215936 13918 tablet_bootstrap.cc:492] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8: Bootstrap starting.
I20260709 12:29:19.219369 13917 raft_consensus.cc:359] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:19.220420 13917 raft_consensus.cc:385] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.220893 13917 raft_consensus.cc:740] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.222060 13917 consensus_queue.cc:260] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:19.223177 13918 tablet_bootstrap.cc:654] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.227401 13916 tablet_bootstrap.cc:492] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af: No bootstrap required, opened a new log
I20260709 12:29:19.228741 13916 ts_tablet_manager.cc:1403] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.006s
I20260709 12:29:19.231048 13917 ts_tablet_manager.cc:1434] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.002s
I20260709 12:29:19.232234 13917 tablet_bootstrap.cc:492] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:19.232033 13916 raft_consensus.cc:359] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.233001 13916 raft_consensus.cc:385] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.233394 13916 raft_consensus.cc:740] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.234407 13916 consensus_queue.cc:260] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.239015 13916 ts_tablet_manager.cc:1434] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:29:19.240199 13916 tablet_bootstrap.cc:492] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af: Bootstrap starting.
I20260709 12:29:19.242219 13923 raft_consensus.cc:493] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.242833 13923 raft_consensus.cc:515] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.245174 13923 leader_election.cc:290] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.248184 13917 tablet_bootstrap.cc:654] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.248267 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a52eaa3c1df34e6c8a1ccea63bf39b1d" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:19.248708 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a52eaa3c1df34e6c8a1ccea63bf39b1d" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
W20260709 12:29:19.250228 13693 leader_election.cc:343] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:19.250860 13690 leader_election.cc:343] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.251324 13690 leader_election.cc:304] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6bb736605e6460e91c7b8a131e0e84b; no voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8
I20260709 12:29:19.252697 13923 raft_consensus.cc:2749] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.254482 13918 tablet_bootstrap.cc:492] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8: No bootstrap required, opened a new log
I20260709 12:29:19.254922 13916 tablet_bootstrap.cc:654] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.255070 13918 ts_tablet_manager.cc:1403] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.014s
I20260709 12:29:19.256199 13917 tablet_bootstrap.cc:492] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: No bootstrap required, opened a new log
I20260709 12:29:19.256734 13917 ts_tablet_manager.cc:1403] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.001s
I20260709 12:29:19.258455 13918 raft_consensus.cc:359] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.259290 13918 raft_consensus.cc:385] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.259693 13918 raft_consensus.cc:740] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.259874 13917 raft_consensus.cc:359] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.260653 13917 raft_consensus.cc:385] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.261001 13917 raft_consensus.cc:740] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.260517 13918 consensus_queue.cc:260] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.262681 13917 consensus_queue.cc:260] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.263461 13918 ts_tablet_manager.cc:1434] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:29:19.264539 13918 tablet_bootstrap.cc:492] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8: Bootstrap starting.
I20260709 12:29:19.265781 13916 tablet_bootstrap.cc:492] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af: No bootstrap required, opened a new log
I20260709 12:29:19.266403 13916 ts_tablet_manager.cc:1403] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af: Time spent bootstrapping tablet: real 0.026s	user 0.005s	sys 0.011s
I20260709 12:29:19.269274 13917 ts_tablet_manager.cc:1434] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.004s
I20260709 12:29:19.269539 13916 raft_consensus.cc:359] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.270488 13917 tablet_bootstrap.cc:492] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:19.270512 13916 raft_consensus.cc:385] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.271580 13916 raft_consensus.cc:740] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.271687 13918 tablet_bootstrap.cc:654] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.272557 13916 consensus_queue.cc:260] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.275422 13916 ts_tablet_manager.cc:1434] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:29:19.276536 13916 tablet_bootstrap.cc:492] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af: Bootstrap starting.
I20260709 12:29:19.279489 13917 tablet_bootstrap.cc:654] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.283417 13916 tablet_bootstrap.cc:654] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.284477 13918 tablet_bootstrap.cc:492] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8: No bootstrap required, opened a new log
I20260709 12:29:19.285117 13918 ts_tablet_manager.cc:1403] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.008s
I20260709 12:29:19.289590 13918 raft_consensus.cc:359] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:19.290732 13918 raft_consensus.cc:385] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.291190 13918 raft_consensus.cc:740] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.292647 13918 consensus_queue.cc:260] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:19.305130 13918 ts_tablet_manager.cc:1434] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:29:19.307224 13918 tablet_bootstrap.cc:492] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8: Bootstrap starting.
I20260709 12:29:19.308796 13916 tablet_bootstrap.cc:492] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af: No bootstrap required, opened a new log
I20260709 12:29:19.309450 13916 ts_tablet_manager.cc:1403] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.016s
I20260709 12:29:19.311043 13917 tablet_bootstrap.cc:492] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: No bootstrap required, opened a new log
I20260709 12:29:19.311617 13917 ts_tablet_manager.cc:1403] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.041s	user 0.025s	sys 0.010s
I20260709 12:29:19.312785 13916 raft_consensus.cc:359] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.313664 13916 raft_consensus.cc:385] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.314011 13916 raft_consensus.cc:740] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.314263 13918 tablet_bootstrap.cc:654] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.314482 13917 raft_consensus.cc:359] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.315191 13917 raft_consensus.cc:385] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.315448 13917 raft_consensus.cc:740] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.314890 13916 consensus_queue.cc:260] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.316121 13917 consensus_queue.cc:260] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.318156 13916 ts_tablet_manager.cc:1434] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:19.318521 13917 ts_tablet_manager.cc:1434] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 12:29:19.319523 13917 tablet_bootstrap.cc:492] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:19.320755 13916 tablet_bootstrap.cc:492] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af: Bootstrap starting.
I20260709 12:29:19.325304 13917 tablet_bootstrap.cc:654] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.327698 13916 tablet_bootstrap.cc:654] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.330906 13918 tablet_bootstrap.cc:492] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8: No bootstrap required, opened a new log
I20260709 12:29:19.331508 13918 ts_tablet_manager.cc:1403] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.004s
I20260709 12:29:19.334302 13918 raft_consensus.cc:359] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.335016 13918 raft_consensus.cc:385] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.335285 13918 raft_consensus.cc:740] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.335947 13918 consensus_queue.cc:260] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.338650 13918 ts_tablet_manager.cc:1434] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:19.339627 13918 tablet_bootstrap.cc:492] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8: Bootstrap starting.
I20260709 12:29:19.346783 13918 tablet_bootstrap.cc:654] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.348896 13917 tablet_bootstrap.cc:492] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b: No bootstrap required, opened a new log
I20260709 12:29:19.349805 13917 ts_tablet_manager.cc:1403] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.014s
I20260709 12:29:19.353730 13917 raft_consensus.cc:359] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.355012 13917 raft_consensus.cc:385] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.355459 13917 raft_consensus.cc:740] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.356283 13917 consensus_queue.cc:260] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.358366 13916 tablet_bootstrap.cc:492] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af: No bootstrap required, opened a new log
I20260709 12:29:19.358584 13917 ts_tablet_manager.cc:1434] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:29:19.359045 13918 tablet_bootstrap.cc:492] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8: No bootstrap required, opened a new log
I20260709 12:29:19.359076 13916 ts_tablet_manager.cc:1403] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af: Time spent bootstrapping tablet: real 0.039s	user 0.022s	sys 0.015s
I20260709 12:29:19.359714 13917 tablet_bootstrap.cc:492] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:19.362761 13916 raft_consensus.cc:359] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.363466 13916 raft_consensus.cc:385] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.363722 13916 raft_consensus.cc:740] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.364463 13916 consensus_queue.cc:260] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.359866 13918 ts_tablet_manager.cc:1403] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8: Time spent bootstrapping tablet: real 0.020s	user 0.005s	sys 0.012s
I20260709 12:29:19.366645 13916 ts_tablet_manager.cc:1434] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:29:19.367343 13917 tablet_bootstrap.cc:654] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.368453 13923 raft_consensus.cc:493] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.368956 13918 raft_consensus.cc:359] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.369884 13918 raft_consensus.cc:385] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.370308 13918 raft_consensus.cc:740] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.372243 13916 tablet_bootstrap.cc:492] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af: Bootstrap starting.
I20260709 12:29:19.371593 13918 consensus_queue.cc:260] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.368989 13923 raft_consensus.cc:515] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.376317 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46731d8225284b3c85730af89cb0a871" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
W20260709 12:29:19.377875 13690 leader_election.cc:343] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.378795 13916 tablet_bootstrap.cc:654] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.379832 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46731d8225284b3c85730af89cb0a871" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:19.380304 13918 ts_tablet_manager.cc:1434] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 12:29:19.380651 13802 raft_consensus.cc:2468] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 0.
I20260709 12:29:19.382200 13918 tablet_bootstrap.cc:492] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8: Bootstrap starting.
I20260709 12:29:19.389995 13923 leader_election.cc:290] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.382290 13693 leader_election.cc:304] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 0a1b5a26007a4097887ffb7b875380af
I20260709 12:29:19.394687 13921 raft_consensus.cc:493] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.395437 13921 raft_consensus.cc:515] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.395823 13923 raft_consensus.cc:2804] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.396261 13923 raft_consensus.cc:493] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.396584 13923 raft_consensus.cc:3060] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.397830 13921 leader_election.cc:290] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:19.401125 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:19.401937 13802 raft_consensus.cc:2468] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 0.
I20260709 12:29:19.402891 13918 tablet_bootstrap.cc:654] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.403281 13842 leader_election.cc:304] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
I20260709 12:29:19.404557 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b" is_pre_election: true
I20260709 12:29:19.405267 13726 raft_consensus.cc:2468] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 0.
I20260709 12:29:19.407332 13921 raft_consensus.cc:493] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.407904 13921 raft_consensus.cc:515] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.411743 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b" is_pre_election: true
I20260709 12:29:19.412603 13726 raft_consensus.cc:2468] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 0.
I20260709 12:29:19.414307 13841 leader_election.cc:304] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:19.417274 13918 tablet_bootstrap.cc:492] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8: No bootstrap required, opened a new log
I20260709 12:29:19.418156 13918 ts_tablet_manager.cc:1403] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.008s
I20260709 12:29:19.418799 13921 leader_election.cc:290] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:19.418654 13923 raft_consensus.cc:515] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.419452 13921 raft_consensus.cc:2804] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.419823 13921 raft_consensus.cc:493] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.420182 13921 raft_consensus.cc:3060] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.422636 13923 leader_election.cc:290] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 election: Requested vote from peers d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.422780 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46731d8225284b3c85730af89cb0a871" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:19.418056 13962 raft_consensus.cc:2804] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.422907 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:19.423414 13802 raft_consensus.cc:3060] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.423870 13801 raft_consensus.cc:2468] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 0.
I20260709 12:29:19.423439 13962 raft_consensus.cc:493] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.425619 13962 raft_consensus.cc:3060] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.439183 13923 raft_consensus.cc:493] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.440012 13923 raft_consensus.cc:515] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.442426 13923 leader_election.cc:290] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.442190 13918 raft_consensus.cc:359] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.443464 13918 raft_consensus.cc:385] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.444646 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95b4e77affd440fafced08a62b6135d" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:19.445668 13801 raft_consensus.cc:2468] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 0.
I20260709 12:29:19.446987 13693 leader_election.cc:304] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:19.443943 13918 raft_consensus.cc:740] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.447697 13917 tablet_bootstrap.cc:492] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: No bootstrap required, opened a new log
I20260709 12:29:19.448369 13917 ts_tablet_manager.cc:1403] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.089s	user 0.029s	sys 0.055s
I20260709 12:29:19.448550 13923 raft_consensus.cc:2804] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.449033 13923 raft_consensus.cc:493] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.448712 13918 consensus_queue.cc:260] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.449421 13923 raft_consensus.cc:3060] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.450601 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46731d8225284b3c85730af89cb0a871" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:19.451175 13917 raft_consensus.cc:359] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.451860 13917 raft_consensus.cc:385] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.452067 13916 tablet_bootstrap.cc:492] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af: No bootstrap required, opened a new log
I20260709 12:29:19.452220 13917 raft_consensus.cc:740] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.452667 13916 ts_tablet_manager.cc:1403] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af: Time spent bootstrapping tablet: real 0.081s	user 0.016s	sys 0.007s
I20260709 12:29:19.453181 13917 consensus_queue.cc:260] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.454982 13876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95b4e77affd440fafced08a62b6135d" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
I20260709 12:29:19.455322 13917 ts_tablet_manager.cc:1434] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:29:19.455685 13876 raft_consensus.cc:2468] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 0.
I20260709 12:29:19.457279 13918 ts_tablet_manager.cc:1434] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8: Time spent starting tablet: real 0.038s	user 0.014s	sys 0.000s
W20260709 12:29:19.457307 13690 leader_election.cc:343] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:19.463505 13916 raft_consensus.cc:359] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.464515 13916 raft_consensus.cc:385] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.464807 13916 raft_consensus.cc:740] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.465899 13916 consensus_queue.cc:260] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.467054 13962 raft_consensus.cc:515] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.468122 13916 ts_tablet_manager.cc:1434] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:29:19.469373 13962 leader_election.cc:290] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:19.470204 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:19.470888 13726 raft_consensus.cc:3060] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.475188 13923 raft_consensus.cc:515] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.475495 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:19.476219 13801 raft_consensus.cc:3060] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.478003 13923 leader_election.cc:290] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 election: Requested vote from peers d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.478777 13921 raft_consensus.cc:515] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.479126 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95b4e77affd440fafced08a62b6135d" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:19.480818 13800 raft_consensus.cc:3060] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.480684 13876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95b4e77affd440fafced08a62b6135d" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:19.481467 13876 raft_consensus.cc:3060] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.483803 13962 raft_consensus.cc:493] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.484372 13962 raft_consensus.cc:515] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.485307 13801 raft_consensus.cc:2468] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 1.
I20260709 12:29:19.486689 13842 leader_election.cc:304] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
I20260709 12:29:19.486689 13962 leader_election.cc:290] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:19.487540 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "005c038223964ceb88b90a7698bc658e" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b" is_pre_election: true
I20260709 12:29:19.488018 13964 raft_consensus.cc:2804] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:19.488407 13727 raft_consensus.cc:2468] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 0.
I20260709 12:29:19.488572 13964 raft_consensus.cc:697] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Becoming Leader. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:19.490010 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "005c038223964ceb88b90a7698bc658e" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:19.489666 13964 consensus_queue.cc:237] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.490979 13801 raft_consensus.cc:2468] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 0.
I20260709 12:29:19.493242 13842 leader_election.cc:304] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
I20260709 12:29:19.480147 13802 raft_consensus.cc:2468] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 1.
I20260709 12:29:19.494117 13962 raft_consensus.cc:2804] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.494632 13962 raft_consensus.cc:493] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.495048 13962 raft_consensus.cc:3060] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.494817 13693 leader_election.cc:304] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 0a1b5a26007a4097887ffb7b875380af
I20260709 12:29:19.496491 13923 raft_consensus.cc:2804] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:19.497191 13923 raft_consensus.cc:697] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: Becoming Leader. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Running, Role: LEADER
I20260709 12:29:19.498576 13923 consensus_queue.cc:237] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.502076 13726 raft_consensus.cc:2468] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 1.
I20260709 12:29:19.503055 13916 tablet_bootstrap.cc:492] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af: Bootstrap starting.
I20260709 12:29:19.507604 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:19.504653 13876 raft_consensus.cc:2468] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 1.
I20260709 12:29:19.508262 13727 raft_consensus.cc:3060] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.509871 13936 raft_consensus.cc:493] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.510672 13936 raft_consensus.cc:515] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:19.511552 13690 leader_election.cc:304] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:19.515594 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c762518c0d2c453a97841958622ffe38" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:19.516569 13801 raft_consensus.cc:2468] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 0.
I20260709 12:29:19.518504 13693 leader_election.cc:304] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:19.504724 13800 raft_consensus.cc:2468] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 1.
I20260709 12:29:19.520093 13936 leader_election.cc:290] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:19.520895 13936 raft_consensus.cc:2804] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:19.521389 13936 raft_consensus.cc:697] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: Becoming Leader. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Running, Role: LEADER
I20260709 12:29:19.522323 13936 consensus_queue.cc:237] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.522778 13876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c762518c0d2c453a97841958622ffe38" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
I20260709 12:29:19.523499 13876 raft_consensus.cc:2468] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 0.
I20260709 12:29:19.528481 13916 tablet_bootstrap.cc:654] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:19.529868 13727 raft_consensus.cc:2468] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 1.
I20260709 12:29:19.531091 13841 leader_election.cc:304] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:19.531961 13962 raft_consensus.cc:515] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.532469 13967 raft_consensus.cc:493] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.533218 13967 raft_consensus.cc:515] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.521414 13968 raft_consensus.cc:2804] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.535182 13968 raft_consensus.cc:493] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.535656 13968 raft_consensus.cc:3060] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.544579 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a52eaa3c1df34e6c8a1ccea63bf39b1d" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:19.546232 13800 raft_consensus.cc:2468] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 0.
I20260709 12:29:19.547955 13842 leader_election.cc:304] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
I20260709 12:29:19.548631 13921 leader_election.cc:290] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:19.552192 13967 leader_election.cc:290] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:19.553133 13967 raft_consensus.cc:2804] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.553537 13967 raft_consensus.cc:493] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.553977 13967 raft_consensus.cc:3060] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.549552 13921 raft_consensus.cc:2804] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:19.560591 13921 raft_consensus.cc:697] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Becoming Leader. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:19.561429 13962 leader_election.cc:290] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:19.561894 13921 consensus_queue.cc:237] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
W20260709 12:29:19.565958 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.3.89.194:40121, user_credentials={real_user=slave}} blocked reactor thread for 47583.3us
I20260709 12:29:19.568302 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "005c038223964ceb88b90a7698bc658e" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:19.569164 13726 raft_consensus.cc:3060] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.569658 13625 catalog_manager.cc:5697] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b reported cstate change: term changed from 0 to 1, leader changed from <none> to f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193). New cstate: current_term: 1 leader_uuid: "f6bb736605e6460e91c7b8a131e0e84b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:19.570367 13624 catalog_manager.cc:5697] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a1b5a26007a4097887ffb7b875380af (127.3.89.195). New cstate: current_term: 1 leader_uuid: "0a1b5a26007a4097887ffb7b875380af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:19.574568 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "005c038223964ceb88b90a7698bc658e" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:19.575243 13801 raft_consensus.cc:3060] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.568298 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a52eaa3c1df34e6c8a1ccea63bf39b1d" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b" is_pre_election: true
I20260709 12:29:19.578603 13727 raft_consensus.cc:2468] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 0.
I20260709 12:29:19.581844 13972 raft_consensus.cc:493] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.583283 13801 raft_consensus.cc:2468] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 1.
I20260709 12:29:19.585032 13842 leader_election.cc:304] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
I20260709 12:29:19.582695 13972 raft_consensus.cc:515] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.585477 13726 raft_consensus.cc:2468] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 1.
I20260709 12:29:19.589015 13925 raft_consensus.cc:493] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.592672 13925 raft_consensus.cc:515] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.587208 13973 raft_consensus.cc:2804] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:19.593804 13973 raft_consensus.cc:697] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Becoming Leader. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:19.594726 13973 consensus_queue.cc:237] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.595152 13624 catalog_manager.cc:5697] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b reported cstate change: term changed from 0 to 1, leader changed from <none> to f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193). New cstate: current_term: 1 leader_uuid: "f6bb736605e6460e91c7b8a131e0e84b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:19.596549 13925 leader_election.cc:290] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.597316 13972 leader_election.cc:290] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:19.598254 13969 raft_consensus.cc:493] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.598449 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b" is_pre_election: true
I20260709 12:29:19.599172 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46731d8225284b3c85730af89cb0a871" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b" is_pre_election: true
I20260709 12:29:19.598888 13969 raft_consensus.cc:515] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.600646 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46731d8225284b3c85730af89cb0a871" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:19.601513 13801 raft_consensus.cc:2393] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a1b5a26007a4097887ffb7b875380af in current term 1: Already voted for candidate f6bb736605e6460e91c7b8a131e0e84b in this term.
I20260709 12:29:19.601154 13727 raft_consensus.cc:2468] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 0.
I20260709 12:29:19.602458 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:19.603127 13802 raft_consensus.cc:2468] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 0.
I20260709 12:29:19.603439 13936 raft_consensus.cc:493] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.604238 13876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
I20260709 12:29:19.604564 13693 leader_election.cc:304] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:19.604998 13876 raft_consensus.cc:2468] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 0.
I20260709 12:29:19.605477 13923 raft_consensus.cc:2804] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.605867 13923 raft_consensus.cc:493] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.606210 13923 raft_consensus.cc:3060] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.604142 13936 raft_consensus.cc:515] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.606837 13842 leader_election.cc:304] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af; no voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b
I20260709 12:29:19.607712 13972 raft_consensus.cc:3060] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.606210 13765 leader_election.cc:304] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
I20260709 12:29:19.613484 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf9b18c2d047babed6c7b74b9dbf01" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:19.614282 13969 leader_election.cc:290] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.614440 13802 raft_consensus.cc:2468] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 0.
I20260709 12:29:19.589761 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:19.615962 13693 leader_election.cc:304] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:19.618319 13876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
I20260709 12:29:19.618996 13876 raft_consensus.cc:2468] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 0.
I20260709 12:29:19.619230 13916 tablet_bootstrap.cc:492] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af: No bootstrap required, opened a new log
I20260709 12:29:19.619786 13916 ts_tablet_manager.cc:1403] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af: Time spent bootstrapping tablet: real 0.117s	user 0.022s	sys 0.039s
I20260709 12:29:19.592507 13968 raft_consensus.cc:515] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:19.620612 13936 leader_election.cc:290] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.621158 13936 raft_consensus.cc:2804] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.621493 13936 raft_consensus.cc:493] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.621817 13936 raft_consensus.cc:3060] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.622529 13916 raft_consensus.cc:359] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.623092 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c762518c0d2c453a97841958622ffe38" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:19.623762 13802 raft_consensus.cc:3060] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.623261 13916 raft_consensus.cc:385] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:19.624665 13916 raft_consensus.cc:740] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Initialized, Role: FOLLOWER
I20260709 12:29:19.616133 13925 raft_consensus.cc:2804] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:19.625495 13916 consensus_queue.cc:260] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.626025 13925 raft_consensus.cc:493] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.626389 13925 raft_consensus.cc:3060] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.627743 13916 ts_tablet_manager.cc:1434] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:19.628520 13876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf9b18c2d047babed6c7b74b9dbf01" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
I20260709 12:29:19.629155 13876 raft_consensus.cc:2468] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 0.
I20260709 12:29:19.630728 13876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c762518c0d2c453a97841958622ffe38" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:19.631623 13876 raft_consensus.cc:3060] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.632289 13802 raft_consensus.cc:2468] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 1.
I20260709 12:29:19.633522 13693 leader_election.cc:304] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:19.634750 13969 raft_consensus.cc:2804] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:19.636196 13968 leader_election.cc:290] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 election: Requested vote from peers 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:19.616792 13800 raft_consensus.cc:3060] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.641822 13967 raft_consensus.cc:515] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.615983 13923 raft_consensus.cc:515] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.615823 13972 raft_consensus.cc:2749] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.642688 13969 raft_consensus.cc:697] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: Becoming Leader. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Running, Role: LEADER
I20260709 12:29:19.646116 13876 raft_consensus.cc:2468] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 1.
I20260709 12:29:19.645699 13969 consensus_queue.cc:237] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:19.646790 13925 raft_consensus.cc:515] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.650259 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:19.651156 13727 raft_consensus.cc:2393] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d6045f6899cf43f7aa7af074983fcae8 in current term 1: Already voted for candidate f6bb736605e6460e91c7b8a131e0e84b in this term.
I20260709 12:29:19.652714 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a52eaa3c1df34e6c8a1ccea63bf39b1d" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:19.652774 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:19.653368 13725 raft_consensus.cc:3060] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.653586 13802 raft_consensus.cc:2393] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f6bb736605e6460e91c7b8a131e0e84b in current term 1: Already voted for candidate d6045f6899cf43f7aa7af074983fcae8 in this term.
I20260709 12:29:19.654424 13800 raft_consensus.cc:2468] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 1.
I20260709 12:29:19.656008 13923 leader_election.cc:290] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 election: Requested vote from peers d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.657269 13967 leader_election.cc:290] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:19.658113 13876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:19.663059 13876 raft_consensus.cc:3060] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.664362 13936 raft_consensus.cc:515] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.658816 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a52eaa3c1df34e6c8a1ccea63bf39b1d" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:19.666262 13624 catalog_manager.cc:5697] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b reported cstate change: term changed from 0 to 1, leader changed from <none> to f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193). New cstate: current_term: 1 leader_uuid: "f6bb736605e6460e91c7b8a131e0e84b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:19.668421 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf9b18c2d047babed6c7b74b9dbf01" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:19.662189 13875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:19.683074 13936 leader_election.cc:290] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 election: Requested vote from peers d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.683341 13802 raft_consensus.cc:3060] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.684077 13765 leader_election.cc:304] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8; no voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b
I20260709 12:29:19.683355 13625 catalog_manager.cc:5697] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a1b5a26007a4097887ffb7b875380af (127.3.89.195). New cstate: current_term: 1 leader_uuid: "0a1b5a26007a4097887ffb7b875380af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:19.669113 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf9b18c2d047babed6c7b74b9dbf01" candidate_uuid: "f6bb736605e6460e91c7b8a131e0e84b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:19.676028 13923 consensus_queue.cc:1048] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:29:19.686210 13874 raft_consensus.cc:3060] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.686671 13800 raft_consensus.cc:3060] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:19.660912 13925 leader_election.cc:290] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 1 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:19.676553 13725 raft_consensus.cc:2468] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 1.
I20260709 12:29:19.692732 13841 leader_election.cc:304] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:19.693673 13925 raft_consensus.cc:2749] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:19.694409 13921 raft_consensus.cc:2804] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:19.694990 13921 raft_consensus.cc:697] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Becoming Leader. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:19.695874 13921 consensus_queue.cc:237] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.705120 13874 raft_consensus.cc:2468] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 1.
I20260709 12:29:19.707275 13876 raft_consensus.cc:2468] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 1.
I20260709 12:29:19.709903 13690 leader_election.cc:304] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b; no voters: d6045f6899cf43f7aa7af074983fcae8
I20260709 12:29:19.711011 13802 raft_consensus.cc:2468] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb736605e6460e91c7b8a131e0e84b in term 1.
I20260709 12:29:19.711242 13923 raft_consensus.cc:2804] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:19.711759 13923 raft_consensus.cc:697] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: Becoming Leader. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Running, Role: LEADER
I20260709 12:29:19.712211 13693 leader_election.cc:304] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:19.712711 13923 consensus_queue.cc:237] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.713179 13936 raft_consensus.cc:2804] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:19.713846 13936 raft_consensus.cc:697] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: Becoming Leader. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Running, Role: LEADER
I20260709 12:29:19.715602 13936 consensus_queue.cc:237] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:19.726531 13624 catalog_manager.cc:5697] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a1b5a26007a4097887ffb7b875380af (127.3.89.195). New cstate: current_term: 1 leader_uuid: "0a1b5a26007a4097887ffb7b875380af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:19.730365 13624 catalog_manager.cc:5697] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a1b5a26007a4097887ffb7b875380af (127.3.89.195). New cstate: current_term: 1 leader_uuid: "0a1b5a26007a4097887ffb7b875380af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:19.731005 13800 raft_consensus.cc:2468] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 1.
I20260709 12:29:19.733302 13969 consensus_queue.cc:1048] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:19.746940 13622 catalog_manager.cc:5697] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b reported cstate change: term changed from 0 to 1, leader changed from <none> to f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193). New cstate: current_term: 1 leader_uuid: "f6bb736605e6460e91c7b8a131e0e84b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:19.769536 13624 catalog_manager.cc:5697] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b reported cstate change: term changed from 0 to 1, leader changed from <none> to f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193). New cstate: current_term: 1 leader_uuid: "f6bb736605e6460e91c7b8a131e0e84b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:19.787101  3431 ts_manager.cc:295] Set tserver state for f6bb736605e6460e91c7b8a131e0e84b to MAINTENANCE_MODE
I20260709 12:29:19.788098  3431 tablet_server.cc:179] TabletServer@127.3.89.193:0 shutting down...
I20260709 12:29:19.809685  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:19.810912  3431 tablet_replica.cc:333] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: stopping tablet replica
I20260709 12:29:19.811842  3431 raft_consensus.cc:2243] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:19.812491  3431 pending_rounds.cc:70] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: Trying to abort 1 pending ops.
I20260709 12:29:19.812733  3431 pending_rounds.cc:77] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305626254208598016 op_type: NO_OP noop_request { }
I20260709 12:29:19.813038  3431 raft_consensus.cc:2889] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:19.813336  3431 raft_consensus.cc:2272] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:19.816110  3431 tablet_replica.cc:333] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b: stopping tablet replica
I20260709 12:29:19.816732  3431 raft_consensus.cc:2243] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:19.817149  3431 raft_consensus.cc:2272] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:19.819259  3431 tablet_replica.cc:333] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: stopping tablet replica
I20260709 12:29:19.819828  3431 raft_consensus.cc:2243] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:19.820451  3431 pending_rounds.cc:70] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: Trying to abort 1 pending ops.
I20260709 12:29:19.820606  3431 pending_rounds.cc:77] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305626254206181376 op_type: NO_OP noop_request { }
I20260709 12:29:19.820897  3431 raft_consensus.cc:2889] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:19.821209  3431 raft_consensus.cc:2272] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:19.823601  3431 tablet_replica.cc:333] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: stopping tablet replica
I20260709 12:29:19.824167  3431 raft_consensus.cc:2243] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:19.824769  3431 pending_rounds.cc:70] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: Trying to abort 1 pending ops.
I20260709 12:29:19.824926  3431 pending_rounds.cc:77] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305626253413580800 op_type: NO_OP noop_request { }
I20260709 12:29:19.825230  3431 raft_consensus.cc:2889] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:19.825518  3431 raft_consensus.cc:2272] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:19.827719  3431 tablet_replica.cc:333] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b: stopping tablet replica
I20260709 12:29:19.828289  3431 raft_consensus.cc:2243] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:19.828676  3431 raft_consensus.cc:2272] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:19.830433  3431 tablet_replica.cc:333] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b: stopping tablet replica
I20260709 12:29:19.830947  3431 raft_consensus.cc:2243] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:19.831310  3431 raft_consensus.cc:2272] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:19.832895  3431 tablet_replica.cc:333] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b: stopping tablet replica
I20260709 12:29:19.833413  3431 raft_consensus.cc:2243] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:19.833845  3431 raft_consensus.cc:2272] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:19.835807  3431 tablet_replica.cc:333] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: stopping tablet replica
I20260709 12:29:19.836288  3431 raft_consensus.cc:2243] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:19.836884  3431 pending_rounds.cc:70] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: Trying to abort 1 pending ops.
I20260709 12:29:19.837108  3431 pending_rounds.cc:77] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305626253318193152 op_type: NO_OP noop_request { }
I20260709 12:29:19.837479  3431 raft_consensus.cc:2889] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:19.837839  3431 raft_consensus.cc:2272] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:19.839962  3431 tablet_replica.cc:333] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: stopping tablet replica
I20260709 12:29:19.840448  3431 raft_consensus.cc:2243] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:19.840976  3431 pending_rounds.cc:70] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: Trying to abort 1 pending ops.
I20260709 12:29:19.841122  3431 pending_rounds.cc:77] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305626253923880960 op_type: NO_OP noop_request { }
I20260709 12:29:19.841334  3431 raft_consensus.cc:2889] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:19.841717  3431 raft_consensus.cc:2272] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:19.843897  3431 tablet_replica.cc:333] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b: stopping tablet replica
I20260709 12:29:19.844442  3431 raft_consensus.cc:2243] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:19.845374  3431 raft_consensus.cc:2272] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:19.880364  3431 tablet_server.cc:196] TabletServer@127.3.89.193:0 shutdown complete.
W20260709 12:29:19.952709 13841 consensus_peers.cc:597] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:19.971572 13921 consensus_queue.cc:1048] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:19.993947 13973 consensus_queue.cc:1048] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:20.009917 13841 consensus_peers.cc:597] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:20.064355 13673 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:20.075297 13967 consensus_queue.cc:1048] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:20.089828 13841 consensus_peers.cc:597] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:20.170332 13967 consensus_queue.cc:1048] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:20.185174 13841 consensus_peers.cc:597] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:20.399868 13841 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111) [suppressed 179 similar messages]
I20260709 12:29:21.023401 13994 raft_consensus.cc:493] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.023929 13994 raft_consensus.cc:515] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.025681 13994 leader_election.cc:290] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:21.027810 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95b4e77affd440fafced08a62b6135d" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
I20260709 12:29:21.028585 13874 raft_consensus.cc:2468] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 1.
I20260709 12:29:21.030408 13765 leader_election.cc:304] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
I20260709 12:29:21.031301 13994 raft_consensus.cc:2804] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:21.031700 13994 raft_consensus.cc:493] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.032038 13994 raft_consensus.cc:3060] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:21.032481 13767 leader_election.cc:336] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111)
I20260709 12:29:21.037889 13994 raft_consensus.cc:515] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.039688 13994 leader_election.cc:290] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:21.040884 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95b4e77affd440fafced08a62b6135d" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:21.041625 13874 raft_consensus.cc:3060] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:21.043532 13767 leader_election.cc:336] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111)
I20260709 12:29:21.047351 13874 raft_consensus.cc:2468] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 2.
I20260709 12:29:21.048491 13765 leader_election.cc:304] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: f6bb736605e6460e91c7b8a131e0e84b
I20260709 12:29:21.049193 13994 raft_consensus.cc:2804] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:21.050199 13994 raft_consensus.cc:697] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Becoming Leader. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:21.051028 13994 consensus_queue.cc:237] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.058385 13624 catalog_manager.cc:5697] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 reported cstate change: term changed from 1 to 2, leader changed from f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193) to d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "d6045f6899cf43f7aa7af074983fcae8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:21.065495 13673 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:21.234177 13994 raft_consensus.cc:493] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f6bb736605e6460e91c7b8a131e0e84b)
I20260709 12:29:21.234838 13994 raft_consensus.cc:515] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.236826 13994 leader_election.cc:290] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:21.238125 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21be6a87739412694ceb7ae4cf3e42d" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
W20260709 12:29:21.241123 13767 leader_election.cc:336] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111)
I20260709 12:29:21.241750 13767 leader_election.cc:304] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8; no voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b
I20260709 12:29:21.242520 13994 raft_consensus.cc:2749] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:21.257042 13967 raft_consensus.cc:493] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f6bb736605e6460e91c7b8a131e0e84b)
I20260709 12:29:21.257730 13967 raft_consensus.cc:515] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.259553 13967 leader_election.cc:290] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:21.260715 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21be6a87739412694ceb7ae4cf3e42d" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:21.261462 13802 raft_consensus.cc:2468] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 1.
I20260709 12:29:21.262759 13842 leader_election.cc:304] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
I20260709 12:29:21.263695 13967 raft_consensus.cc:2804] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:29:21.263926 13841 leader_election.cc:336] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111)
I20260709 12:29:21.264253 13967 raft_consensus.cc:493] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Starting leader election (detected failure of leader f6bb736605e6460e91c7b8a131e0e84b)
I20260709 12:29:21.264663 13967 raft_consensus.cc:3060] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:21.270463 13967 raft_consensus.cc:515] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.272670 13967 leader_election.cc:290] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:21.273988 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21be6a87739412694ceb7ae4cf3e42d" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:21.274631 13802 raft_consensus.cc:3060] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:21.276340 13841 leader_election.cc:336] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111)
I20260709 12:29:21.280316 13802 raft_consensus.cc:2468] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 2.
I20260709 12:29:21.281517 13842 leader_election.cc:304] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: f6bb736605e6460e91c7b8a131e0e84b
I20260709 12:29:21.282310 13967 raft_consensus.cc:2804] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:21.283192 13967 raft_consensus.cc:697] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 2 LEADER]: Becoming Leader. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:21.284277 13967 consensus_queue.cc:237] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.291860 13624 catalog_manager.cc:5697] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af reported cstate change: term changed from 1 to 2, leader changed from f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193) to 0a1b5a26007a4097887ffb7b875380af (127.3.89.195). New cstate: current_term: 2 leader_uuid: "0a1b5a26007a4097887ffb7b875380af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:21.401417 13967 raft_consensus.cc:493] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.402048 13967 raft_consensus.cc:515] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.403970 13967 leader_election.cc:290] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:21.405207 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46731d8225284b3c85730af89cb0a871" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:21.406055 13802 raft_consensus.cc:2468] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 1.
I20260709 12:29:21.407330 13842 leader_election.cc:304] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
W20260709 12:29:21.408128 13841 leader_election.cc:336] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111)
I20260709 12:29:21.408192 13967 raft_consensus.cc:2804] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:21.408775 13967 raft_consensus.cc:493] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.409169 13967 raft_consensus.cc:3060] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:21.415001 13967 raft_consensus.cc:515] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.417038 13967 leader_election.cc:290] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:21.418409 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46731d8225284b3c85730af89cb0a871" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:21.419065 13802 raft_consensus.cc:3060] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:21.421156 13841 leader_election.cc:336] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111)
I20260709 12:29:21.425314 13802 raft_consensus.cc:2468] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 2.
I20260709 12:29:21.426849 13842 leader_election.cc:304] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: f6bb736605e6460e91c7b8a131e0e84b
I20260709 12:29:21.427812 14003 raft_consensus.cc:2804] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:21.428273 14003 raft_consensus.cc:697] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 2 LEADER]: Becoming Leader. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:21.429289 14003 consensus_queue.cc:237] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.436312 13624 catalog_manager.cc:5697] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af reported cstate change: term changed from 1 to 2, leader changed from f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193) to 0a1b5a26007a4097887ffb7b875380af (127.3.89.195). New cstate: current_term: 2 leader_uuid: "0a1b5a26007a4097887ffb7b875380af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:21.563663 14003 raft_consensus.cc:493] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.564181 14003 raft_consensus.cc:515] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.565955 14003 leader_election.cc:290] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:21.567107 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf9b18c2d047babed6c7b74b9dbf01" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8" is_pre_election: true
I20260709 12:29:21.567781 13802 raft_consensus.cc:2468] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 1.
I20260709 12:29:21.568783 13842 leader_election.cc:304] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
I20260709 12:29:21.569490 14003 raft_consensus.cc:2804] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:21.569871 14003 raft_consensus.cc:493] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:21.569897 13841 leader_election.cc:336] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111)
I20260709 12:29:21.570207 14003 raft_consensus.cc:3060] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:21.575853 14003 raft_consensus.cc:515] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.577533 14003 leader_election.cc:290] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:21.578645 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf9b18c2d047babed6c7b74b9dbf01" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:21.579252 13802 raft_consensus.cc:3060] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:21.581385 13841 leader_election.cc:336] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111)
I20260709 12:29:21.582782 13994 raft_consensus.cc:493] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.583369 13994 raft_consensus.cc:515] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.584769 13802 raft_consensus.cc:2468] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 2.
I20260709 12:29:21.585320 13994 leader_election.cc:290] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:21.586514 13842 leader_election.cc:304] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: f6bb736605e6460e91c7b8a131e0e84b
I20260709 12:29:21.586722 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
I20260709 12:29:21.587716 13874 raft_consensus.cc:2468] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 1.
I20260709 12:29:21.587765 14003 raft_consensus.cc:2804] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:21.588505 14003 raft_consensus.cc:697] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 2 LEADER]: Becoming Leader. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:21.589179 13765 leader_election.cc:304] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
W20260709 12:29:21.590051 13767 leader_election.cc:336] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111)
I20260709 12:29:21.589485 14003 consensus_queue.cc:237] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.590286 13994 raft_consensus.cc:2804] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:21.590718 13994 raft_consensus.cc:493] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.591069 13994 raft_consensus.cc:3060] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:21.595590 13874 raft_consensus.cc:1275] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:29:21.597158 14006 consensus_queue.cc:1048] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:21.599291 13624 catalog_manager.cc:5697] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af reported cstate change: term changed from 1 to 2, leader changed from f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193) to 0a1b5a26007a4097887ffb7b875380af (127.3.89.195). New cstate: current_term: 2 leader_uuid: "0a1b5a26007a4097887ffb7b875380af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:21.601326 13994 raft_consensus.cc:515] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:21.603761 13994 leader_election.cc:290] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:21.605964 13876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:21.606662 13876 raft_consensus.cc:3060] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:21.610004 13767 leader_election.cc:336] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111)
I20260709 12:29:21.615154 13876 raft_consensus.cc:2468] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 2.
I20260709 12:29:21.616623 13765 leader_election.cc:304] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: f6bb736605e6460e91c7b8a131e0e84b
I20260709 12:29:21.617859 14006 raft_consensus.cc:2804] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:21.618573 14006 raft_consensus.cc:697] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Becoming Leader. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:21.619699 14006 consensus_queue.cc:237] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
W20260709 12:29:21.626644 13767 consensus_peers.cc:597] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:21.634832 13624 catalog_manager.cc:5697] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 reported cstate change: term changed from 1 to 2, leader changed from f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193) to d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "d6045f6899cf43f7aa7af074983fcae8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:21.770386 14006 raft_consensus.cc:493] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:21.771163 14006 raft_consensus.cc:515] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:21.773119 14006 leader_election.cc:290] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:21.774498 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c762518c0d2c453a97841958622ffe38" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af" is_pre_election: true
I20260709 12:29:21.775285 13874 raft_consensus.cc:2468] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 1.
I20260709 12:29:21.776503 13765 leader_election.cc:304] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
I20260709 12:29:21.777325 14006 raft_consensus.cc:2804] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:29:21.777443 13767 leader_election.cc:336] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111)
I20260709 12:29:21.777726 14006 raft_consensus.cc:493] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:21.778146 14006 raft_consensus.cc:3060] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:21.784119 14006 raft_consensus.cc:515] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:21.785964 14006 leader_election.cc:290] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:21.787202 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c762518c0d2c453a97841958622ffe38" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:21.787942 13874 raft_consensus.cc:3060] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:21.789510 13767 leader_election.cc:336] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111)
I20260709 12:29:21.793550 13874 raft_consensus.cc:2468] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 2.
I20260709 12:29:21.794894 13765 leader_election.cc:304] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: f6bb736605e6460e91c7b8a131e0e84b
I20260709 12:29:21.795588 14006 raft_consensus.cc:2804] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:21.796051 14006 raft_consensus.cc:697] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Becoming Leader. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:21.796705 14006 consensus_queue.cc:237] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:21.804630 13624 catalog_manager.cc:5697] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 reported cstate change: term changed from 1 to 2, leader changed from f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193) to d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "d6045f6899cf43f7aa7af074983fcae8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:21.857728 13802 raft_consensus.cc:1275] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:29:21.859294 14003 consensus_queue.cc:1048] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:21.873690 13841 consensus_peers.cc:597] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:21.892968 13841 consensus_peers.cc:597] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:21.903030 13802 raft_consensus.cc:1275] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:21.904328 14016 consensus_queue.cc:1048] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:22.037524 13802 raft_consensus.cc:1275] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:29:22.039017 13967 consensus_queue.cc:1048] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:22.053887 13841 consensus_peers.cc:597] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:22.056066 13874 raft_consensus.cc:1275] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:29:22.058221 14006 consensus_queue.cc:1048] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:22.066596 13673 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:29:22.072944 13767 consensus_peers.cc:597] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:22.226786 13767 consensus_peers.cc:597] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:22.262995 13841 consensus_peers.cc:597] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:22.380820 13874 raft_consensus.cc:1275] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:29:22.382161 14011 consensus_queue.cc:1048] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:22.554713 13841 consensus_peers.cc:597] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:22.590430 13841 consensus_peers.cc:597] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:22.652385 13841 consensus_peers.cc:597] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:23.067873 13673 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:29:23.856462 13767 consensus_peers.cc:597] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:24.069087 13673 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:29:24.337765 13841 consensus_peers.cc:597] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:24.596786 13767 consensus_peers.cc:597] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:24.670985 13767 consensus_peers.cc:597] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:24.716630 13841 consensus_peers.cc:597] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:24.717100 13841 consensus_peers.cc:597] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:24.722365 13841 consensus_peers.cc:597] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:24.993363  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:25.004945 14022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:25.009716 14023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:25.016217 14025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:25.018313  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:25.019719  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:25.020849  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:25.021096  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600165021076 us; error 0 us; skew 500 ppm
I20260709 12:29:25.021932  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:25.028949  3431 webserver.cc:533] Webserver started at http://127.3.89.193:35197/ using document root <none> and password file <none>
I20260709 12:29:25.029673  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:25.029975  3431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:29:25.039124  3431 fs_manager.cc:714] Time spent opening directory manager: real 0.008s	user 0.003s	sys 0.004s
W20260709 12:29:25.047216 13841 consensus_peers.cc:597] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:25.052062 14031 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:25.053299  3431 fs_manager.cc:730] Time spent opening block manager: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:29:25.053874  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-0-root
uuid: "f6bb736605e6460e91c7b8a131e0e84b"
format_stamp: "Formatted at 2026-07-09 12:29:18 on dist-test-slave-3hrd"
I20260709 12:29:25.054376  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599794910850-3431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 12:29:25.070246 13673 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:25.084746  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:25.086203  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:25.088057  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:25.092074 14038 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 12:29:25.098996 13841 consensus_peers.cc:597] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:25.152599  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:29:25.153029  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.062s	user 0.003s	sys 0.000s
I20260709 12:29:25.153414  3431 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:29:25.160183 14038 tablet_bootstrap.cc:492] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:25.176815 14038 tablet_bootstrap.cc:492] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:29:25.177896 14038 tablet_bootstrap.cc:492] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap complete.
I20260709 12:29:25.178537 14038 ts_tablet_manager.cc:1403] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 12:29:25.181533 14038 raft_consensus.cc:359] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:25.182749 14038 raft_consensus.cc:740] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:25.183811 14038 consensus_queue.cc:260] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:25.187126 14038 ts_tablet_manager.cc:1434] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:25.188181 14038 tablet_bootstrap.cc:492] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:25.202404 14038 tablet_bootstrap.cc:492] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:29:25.203274 14038 tablet_bootstrap.cc:492] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap complete.
I20260709 12:29:25.203900 14038 ts_tablet_manager.cc:1403] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.004s
I20260709 12:29:25.206090 14038 raft_consensus.cc:359] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.206815 14038 raft_consensus.cc:740] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:25.207571 14038 consensus_queue.cc:260] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.207973  3431 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:29:25.208415  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.055s	user 0.055s	sys 0.000s
I20260709 12:29:25.209627 14038 ts_tablet_manager.cc:1434] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:29:25.210510 14038 tablet_bootstrap.cc:492] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:25.227612 14038 tablet_bootstrap.cc:492] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:25.228956 14038 tablet_bootstrap.cc:492] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap complete.
I20260709 12:29:25.229935 14038 ts_tablet_manager.cc:1403] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 12:29:25.233669 14038 raft_consensus.cc:359] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.234720 14038 raft_consensus.cc:740] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:25.235998 14038 consensus_queue.cc:260] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.239264 14038 ts_tablet_manager.cc:1434] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:29:25.241308 14038 tablet_bootstrap.cc:492] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:25.258278 14038 tablet_bootstrap.cc:492] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:29:25.259361 14038 tablet_bootstrap.cc:492] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap complete.
I20260709 12:29:25.260076 14038 ts_tablet_manager.cc:1403] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 12:29:25.262096 14038 raft_consensus.cc:359] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.262811 14038 raft_consensus.cc:740] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:25.263499 14038 consensus_queue.cc:260] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.265141 14038 ts_tablet_manager.cc:1434] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:29:25.266063 14038 tablet_bootstrap.cc:492] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
W20260709 12:29:25.276448 13841 consensus_peers.cc:597] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:25.278512 14038 tablet_bootstrap.cc:492] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:25.279349 14038 tablet_bootstrap.cc:492] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap complete.
I20260709 12:29:25.279954 14038 ts_tablet_manager.cc:1403] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 12:29:25.282222 14038 raft_consensus.cc:359] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.282829 14038 raft_consensus.cc:740] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:25.283589 14038 consensus_queue.cc:260] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.285238 14038 ts_tablet_manager.cc:1434] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:29:25.285691  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:45845
I20260709 12:29:25.285830 14100 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.193:45845 every 8 connection(s)
I20260709 12:29:25.286010 14038 tablet_bootstrap.cc:492] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:25.290848  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:25.318487 14038 tablet_bootstrap.cc:492] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:25.319602 14038 tablet_bootstrap.cc:492] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap complete.
I20260709 12:29:25.320456 14038 ts_tablet_manager.cc:1403] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.035s	user 0.023s	sys 0.000s
I20260709 12:29:25.323345 14038 raft_consensus.cc:359] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.324190 14038 raft_consensus.cc:740] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:25.325006 14038 consensus_queue.cc:260] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.326648 14074 raft_consensus.cc:3060] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.327395 14038 ts_tablet_manager.cc:1434] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:25.328537 14038 tablet_bootstrap.cc:492] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:25.335502 14101 heartbeater.cc:344] Connected to a master server at 127.3.89.254:33703
I20260709 12:29:25.335952 14074 pending_rounds.cc:85] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: Aborting all ops after (but not including) 0
I20260709 12:29:25.336160 14101 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:25.336400 14074 pending_rounds.cc:107] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:29:25.336686 13874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
mode: GRACEFUL
new_leader_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:40438
I20260709 12:29:25.336969 14074 raft_consensus.cc:2889] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:25.337596 14101 heartbeater.cc:507] Master 127.3.89.254:33703 requested a full tablet report, sending...
I20260709 12:29:25.337631 13874 raft_consensus.cc:606] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Received request to transfer leadership to TS d6045f6899cf43f7aa7af074983fcae8
I20260709 12:29:25.342053 13874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
mode: GRACEFUL
new_leader_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:40438
I20260709 12:29:25.342763 13874 raft_consensus.cc:606] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Received request to transfer leadership to TS d6045f6899cf43f7aa7af074983fcae8
I20260709 12:29:25.345467 14038 tablet_bootstrap.cc:492] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:25.346525 14038 tablet_bootstrap.cc:492] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap complete.
I20260709 12:29:25.346400 13874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6bf9b18c2d047babed6c7b74b9dbf01"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
mode: GRACEFUL
new_leader_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:40438
I20260709 12:29:25.347466 14038 ts_tablet_manager.cc:1403] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.000s
I20260709 12:29:25.347782 13874 raft_consensus.cc:606] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 2 LEADER]: Received request to transfer leadership to TS d6045f6899cf43f7aa7af074983fcae8
I20260709 12:29:25.349186  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:25.349892  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:25.350611  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:25.350152 14038 raft_consensus.cc:359] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.351048 14038 raft_consensus.cc:740] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:25.351866 14038 consensus_queue.cc:260] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.352747 13622 ts_manager.cc:194] Re-registered known tserver with Master: f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845)
I20260709 12:29:25.354364 14038 ts_tablet_manager.cc:1434] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:25.355327 14038 tablet_bootstrap.cc:492] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:25.370993 14038 tablet_bootstrap.cc:492] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:29:25.371511 13622 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34316
I20260709 12:29:25.372049 14038 tablet_bootstrap.cc:492] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap complete.
I20260709 12:29:25.372809 14038 ts_tablet_manager.cc:1403] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.004s
I20260709 12:29:25.375732 14101 heartbeater.cc:499] Master 127.3.89.254:33703 was elected leader, sending a full tablet report...
I20260709 12:29:25.375415 14038 raft_consensus.cc:359] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.376471 14038 raft_consensus.cc:740] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:25.377151 14038 consensus_queue.cc:260] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.379334 14038 ts_tablet_manager.cc:1434] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:29:25.380293 14038 tablet_bootstrap.cc:492] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:25.395133 14038 tablet_bootstrap.cc:492] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:29:25.396274 14038 tablet_bootstrap.cc:492] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap complete.
I20260709 12:29:25.397013 14038 ts_tablet_manager.cc:1403] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 12:29:25.399394 14038 raft_consensus.cc:359] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.400271 14038 raft_consensus.cc:740] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:25.401006 14038 consensus_queue.cc:260] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.402943 14038 ts_tablet_manager.cc:1434] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:29:25.403822 14038 tablet_bootstrap.cc:492] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap starting.
I20260709 12:29:25.416261 14038 tablet_bootstrap.cc:492] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:25.416913 14038 tablet_bootstrap.cc:492] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b: Bootstrap complete.
I20260709 12:29:25.417621 14038 ts_tablet_manager.cc:1403] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 12:29:25.419920 14038 raft_consensus.cc:359] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.420661 14038 raft_consensus.cc:740] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb736605e6460e91c7b8a131e0e84b, State: Initialized, Role: FOLLOWER
I20260709 12:29:25.421430 14038 consensus_queue.cc:260] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.423655 14038 ts_tablet_manager.cc:1434] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:25.570159 14021 raft_consensus.cc:993] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af: : Instructing follower d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:25.570680 13967 raft_consensus.cc:1081] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 2 LEADER]: Signalling peer d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:25.572242 13800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6bf9b18c2d047babed6c7b74b9dbf01"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:59058
I20260709 12:29:25.572933 13800 raft_consensus.cc:493] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:25.573282 13800 raft_consensus.cc:3060] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:25.578469 13800 raft_consensus.cc:515] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.580181 13800 leader_election.cc:290] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 3 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:25.581485 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf9b18c2d047babed6c7b74b9dbf01" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:25.582170 13874 raft_consensus.cc:3055] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:25.582463 13874 raft_consensus.cc:740] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:25.583312 13874 consensus_queue.cc:260] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.584527 13874 raft_consensus.cc:3060] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:25.590842 14074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf9b18c2d047babed6c7b74b9dbf01" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:25.591540 14074 raft_consensus.cc:3060] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:25.591656 13874 raft_consensus.cc:2468] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 3.
I20260709 12:29:25.593087 13765 leader_election.cc:304] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
I20260709 12:29:25.594251 14028 raft_consensus.cc:2804] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:25.595521 14028 raft_consensus.cc:697] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 3 LEADER]: Becoming Leader. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:25.596448 14028 consensus_queue.cc:237] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.600342 14074 raft_consensus.cc:2468] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 3.
I20260709 12:29:25.608278 13622 catalog_manager.cc:5697] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 reported cstate change: term changed from 2 to 3, leader changed from 0a1b5a26007a4097887ffb7b875380af (127.3.89.195) to d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194). New cstate: current_term: 3 leader_uuid: "d6045f6899cf43f7aa7af074983fcae8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:25.642478 14073 raft_consensus.cc:3060] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.647799 14073 pending_rounds.cc:85] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: Aborting all ops after (but not including) 0
I20260709 12:29:25.648015 14073 pending_rounds.cc:107] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:29:25.648185 14073 raft_consensus.cc:2889] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:25.655383 14021 raft_consensus.cc:993] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af: : Instructing follower d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:25.655879 13967 raft_consensus.cc:1081] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Signalling peer d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:25.657521 13800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:59058
I20260709 12:29:25.658078 13800 raft_consensus.cc:493] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:25.658383 13800 raft_consensus.cc:3060] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.664173 13800 raft_consensus.cc:515] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.665927 13800 leader_election.cc:290] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:25.666797 14073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:25.667022 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:25.667577 14073 raft_consensus.cc:3060] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.667649 13874 raft_consensus.cc:3055] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:25.668071 13874 raft_consensus.cc:740] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:25.668824 13874 consensus_queue.cc:260] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.669883 13874 raft_consensus.cc:3060] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.673173 14073 raft_consensus.cc:2468] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 2.
I20260709 12:29:25.674479 13767 leader_election.cc:304] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:25.675237 14028 raft_consensus.cc:2804] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:25.675392 13874 raft_consensus.cc:2468] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 2.
I20260709 12:29:25.675650 14028 raft_consensus.cc:697] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Becoming Leader. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:25.676544 14028 consensus_queue.cc:237] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.683588 13622 catalog_manager.cc:5697] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 reported cstate change: term changed from 1 to 2, leader changed from 0a1b5a26007a4097887ffb7b875380af (127.3.89.195) to d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "d6045f6899cf43f7aa7af074983fcae8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:25.704564 14073 raft_consensus.cc:3060] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.720218 14021 raft_consensus.cc:993] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af: : Instructing follower d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:25.720711 13967 raft_consensus.cc:1081] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Signalling peer d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:25.722323 13800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:59058
I20260709 12:29:25.722988 13800 raft_consensus.cc:493] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:25.723418 13800 raft_consensus.cc:3060] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.729784 13800 raft_consensus.cc:515] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.731540 13800 leader_election.cc:290] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:25.732575 14073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:25.733098 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:25.733327 14073 raft_consensus.cc:3060] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.733886 13874 raft_consensus.cc:3055] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:25.734131 14074 raft_consensus.cc:3060] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.734341 13874 raft_consensus.cc:740] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:25.735415 13874 consensus_queue.cc:260] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.736855 13874 raft_consensus.cc:3060] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.740477 14074 pending_rounds.cc:85] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: Aborting all ops after (but not including) 0
I20260709 12:29:25.740928 14074 pending_rounds.cc:107] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:29:25.741245 14074 raft_consensus.cc:2889] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:25.741997 14073 raft_consensus.cc:2468] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 2.
I20260709 12:29:25.743696 13767 leader_election.cc:304] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:25.744707 14028 raft_consensus.cc:2804] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:25.745219 14028 raft_consensus.cc:697] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Becoming Leader. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:25.745405 13874 raft_consensus.cc:2468] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 2.
I20260709 12:29:25.746181 14028 consensus_queue.cc:237] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:25.751163 14074 raft_consensus.cc:3060] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.754427 13622 catalog_manager.cc:5697] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 reported cstate change: term changed from 1 to 2, leader changed from 0a1b5a26007a4097887ffb7b875380af (127.3.89.195) to d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "d6045f6899cf43f7aa7af074983fcae8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:25.759460 14074 pending_rounds.cc:85] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: Aborting all ops after (but not including) 0
I20260709 12:29:25.759881 14074 pending_rounds.cc:107] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:29:25.760183 14074 raft_consensus.cc:2889] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:25.805855 14074 raft_consensus.cc:3060] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.811805 14074 pending_rounds.cc:85] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: Aborting all ops after (but not including) 0
I20260709 12:29:25.812049 14074 pending_rounds.cc:107] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:29:25.812247 14074 raft_consensus.cc:2889] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:29:25.952260 13656 debug-util.cc:398] Leaking SignalData structure 0x7b08001e6080 after lost signal to thread 3434
W20260709 12:29:25.954151 13656 debug-util.cc:398] Leaking SignalData structure 0x7b0800272ac0 after lost signal to thread 13826
I20260709 12:29:26.046613 13874 raft_consensus.cc:1275] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:26.048280 14028 consensus_queue.cc:1048] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:26.063508 14074 raft_consensus.cc:1275] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:26.065532 14028 consensus_queue.cc:1048] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:26.071584 13673 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:26.137846 14074 raft_consensus.cc:1275] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:26.139462 14117 consensus_queue.cc:1048] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:26.147850 13874 raft_consensus.cc:1275] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:26.150424 14138 consensus_queue.cc:1048] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:26.198078 14074 raft_consensus.cc:1275] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:26.199785 14138 consensus_queue.cc:1048] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:26.208254 13874 raft_consensus.cc:1275] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:26.209973 14139 consensus_queue.cc:1048] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:26.360396  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:26.381350 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
mode: GRACEFUL
new_leader_uuid: "0a1b5a26007a4097887ffb7b875380af"
 from {username='slave'} at 127.0.0.1:35652
I20260709 12:29:26.382153 13800 raft_consensus.cc:606] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Received request to transfer leadership to TS 0a1b5a26007a4097887ffb7b875380af
I20260709 12:29:26.385603 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
mode: GRACEFUL
new_leader_uuid: "0a1b5a26007a4097887ffb7b875380af"
 from {username='slave'} at 127.0.0.1:35652
I20260709 12:29:26.386363 13800 raft_consensus.cc:606] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Received request to transfer leadership to TS 0a1b5a26007a4097887ffb7b875380af
I20260709 12:29:26.387624  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:29:26.388269  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:26.388937  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:26.612675 14138 raft_consensus.cc:993] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8: : Instructing follower 0a1b5a26007a4097887ffb7b875380af to start an election
I20260709 12:29:26.613232 14137 raft_consensus.cc:1081] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Signalling peer 0a1b5a26007a4097887ffb7b875380af to start an election
I20260709 12:29:26.614856 13874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
 from {username='slave'} at 127.0.0.1:43570
I20260709 12:29:26.615487 13874 raft_consensus.cc:493] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:26.615885 13874 raft_consensus.cc:3060] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:26.624320 13874 raft_consensus.cc:515] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:26.626775 13874 leader_election.cc:290] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 3 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:26.627614 14074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:26.628429 14074 raft_consensus.cc:3060] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:26.628342 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:26.629379 13800 raft_consensus.cc:3055] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:26.629817 13800 raft_consensus.cc:740] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:26.630752 13800 consensus_queue.cc:260] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:26.632220 13800 raft_consensus.cc:3060] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:26.637403 14074 raft_consensus.cc:2468] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 3.
I20260709 12:29:26.639075 13841 leader_election.cc:304] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:26.639978 14132 raft_consensus.cc:2804] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:26.640496 14132 raft_consensus.cc:697] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Becoming Leader. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:26.640434 13800 raft_consensus.cc:2468] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 3.
I20260709 12:29:26.641654 14132 consensus_queue.cc:237] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:26.650202 13624 catalog_manager.cc:5697] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af reported cstate change: term changed from 2 to 3, leader changed from d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194) to 0a1b5a26007a4097887ffb7b875380af (127.3.89.195). New cstate: current_term: 3 leader_uuid: "0a1b5a26007a4097887ffb7b875380af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:26.671286 14139 raft_consensus.cc:993] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8: : Instructing follower 0a1b5a26007a4097887ffb7b875380af to start an election
I20260709 12:29:26.671834 14137 raft_consensus.cc:1081] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Signalling peer 0a1b5a26007a4097887ffb7b875380af to start an election
I20260709 12:29:26.673615 13874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
 from {username='slave'} at 127.0.0.1:43570
I20260709 12:29:26.674252 13874 raft_consensus.cc:493] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:26.674618 13874 raft_consensus.cc:3060] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:26.680089 13874 raft_consensus.cc:515] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:26.681985 13874 leader_election.cc:290] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 3 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:26.682869 14074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:26.683157 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:26.683615 14074 raft_consensus.cc:3060] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:26.683868 13800 raft_consensus.cc:3055] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:26.684325 13800 raft_consensus.cc:740] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:26.685338 13800 consensus_queue.cc:260] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:26.686754 13800 raft_consensus.cc:3060] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:26.692198 14074 raft_consensus.cc:2468] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 3.
I20260709 12:29:26.692499 13800 raft_consensus.cc:2468] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 3.
I20260709 12:29:26.693650 13841 leader_election.cc:304] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:26.694561 14132 raft_consensus.cc:2804] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:26.695066 14132 raft_consensus.cc:697] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Becoming Leader. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:26.695902 14132 consensus_queue.cc:237] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:26.702445 13624 catalog_manager.cc:5697] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af reported cstate change: term changed from 2 to 3, leader changed from d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194) to 0a1b5a26007a4097887ffb7b875380af (127.3.89.195). New cstate: current_term: 3 leader_uuid: "0a1b5a26007a4097887ffb7b875380af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: HEALTHY } } }
W20260709 12:29:27.073052 13673 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:27.106664 13800 raft_consensus.cc:1275] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:27.108393 14132 consensus_queue.cc:1048] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:27.119416 14074 raft_consensus.cc:1275] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:27.120764 13967 consensus_queue.cc:1048] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:27.211073 14074 raft_consensus.cc:1275] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:27.211422 13800 raft_consensus.cc:1275] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:27.212572 14132 consensus_queue.cc:1048] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:27.213438 13967 consensus_queue.cc:1048] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:27.396824  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:27.409351 13876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "005c038223964ceb88b90a7698bc658e"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
mode: GRACEFUL
new_leader_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:40438
I20260709 12:29:27.410386 13876 raft_consensus.cc:606] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Received request to transfer leadership to TS d6045f6899cf43f7aa7af074983fcae8
I20260709 12:29:27.414471 13876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
mode: GRACEFUL
new_leader_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:40438
I20260709 12:29:27.415176 13876 raft_consensus.cc:606] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Received request to transfer leadership to TS d6045f6899cf43f7aa7af074983fcae8
I20260709 12:29:27.418481 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c762518c0d2c453a97841958622ffe38"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
mode: GRACEFUL
new_leader_uuid: "0a1b5a26007a4097887ffb7b875380af"
 from {username='slave'} at 127.0.0.1:35652
I20260709 12:29:27.419180 13800 raft_consensus.cc:606] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Received request to transfer leadership to TS 0a1b5a26007a4097887ffb7b875380af
I20260709 12:29:27.421124  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:27.421772  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:27.422696  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:27.522935 14132 raft_consensus.cc:993] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af: : Instructing follower d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:27.523697 14156 raft_consensus.cc:1081] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Signalling peer d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:27.525898 13800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:59058
I20260709 12:29:27.526546 13800 raft_consensus.cc:493] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:27.526981 13800 raft_consensus.cc:3060] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:27.536638 13800 raft_consensus.cc:515] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:27.539655 13800 leader_election.cc:290] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 4 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:27.542091 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:27.542689 14074 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ade9f433393457391d1bc7f19b224de" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:27.543843 14074 raft_consensus.cc:3060] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:27.543136 13874 raft_consensus.cc:3055] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:27.545199 13874 raft_consensus.cc:740] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:27.547290 13874 consensus_queue.cc:260] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:27.550086 13874 raft_consensus.cc:3060] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:27.552467 14132 raft_consensus.cc:993] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af: : Instructing follower d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:27.553329 14155 raft_consensus.cc:1081] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Signalling peer d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:27.555894 13800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "005c038223964ceb88b90a7698bc658e"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:59058
I20260709 12:29:27.556784 13800 raft_consensus.cc:493] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:27.557404 13800 raft_consensus.cc:3060] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.557637 14074 raft_consensus.cc:2468] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 4.
I20260709 12:29:27.559279 13767 leader_election.cc:304] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:27.560541 14139 raft_consensus.cc:2804] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:27.560631 13874 raft_consensus.cc:2468] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 4.
I20260709 12:29:27.562253 14139 raft_consensus.cc:697] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 4 LEADER]: Becoming Leader. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:27.563584 14139 consensus_queue.cc:237] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:27.568094 13800 raft_consensus.cc:515] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:27.571555 13800 leader_election.cc:290] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:27.575364 14073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "005c038223964ceb88b90a7698bc658e" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:27.576196 14073 raft_consensus.cc:3060] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.574882 13622 catalog_manager.cc:5697] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 reported cstate change: term changed from 3 to 4, leader changed from 0a1b5a26007a4097887ffb7b875380af (127.3.89.195) to d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194). New cstate: current_term: 4 leader_uuid: "d6045f6899cf43f7aa7af074983fcae8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.576803 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "005c038223964ceb88b90a7698bc658e" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:27.577411 13874 raft_consensus.cc:3055] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:27.577802 13874 raft_consensus.cc:740] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:27.579237 13874 consensus_queue.cc:260] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:27.580744 13874 raft_consensus.cc:3060] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.587785 14073 raft_consensus.cc:2468] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 2.
I20260709 12:29:27.590148 13767 leader_election.cc:304] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:27.591403 14157 raft_consensus.cc:2804] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.591899 14157 raft_consensus.cc:697] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Becoming Leader. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:27.593103 14157 consensus_queue.cc:237] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:27.596302 13874 raft_consensus.cc:2468] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 2.
I20260709 12:29:27.619364 13622 catalog_manager.cc:5697] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 reported cstate change: term changed from 1 to 2, leader changed from 0a1b5a26007a4097887ffb7b875380af (127.3.89.195) to d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "d6045f6899cf43f7aa7af074983fcae8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.753285 14139 raft_consensus.cc:993] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8: : Instructing follower 0a1b5a26007a4097887ffb7b875380af to start an election
I20260709 12:29:27.754186 14157 raft_consensus.cc:1081] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Signalling peer 0a1b5a26007a4097887ffb7b875380af to start an election
I20260709 12:29:27.755918 13874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c762518c0d2c453a97841958622ffe38"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
 from {username='slave'} at 127.0.0.1:43570
I20260709 12:29:27.756443 13874 raft_consensus.cc:493] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:27.756798 13874 raft_consensus.cc:3060] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:27.763948 13874 raft_consensus.cc:515] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:27.766013 13874 leader_election.cc:290] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 3 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:27.767309 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c762518c0d2c453a97841958622ffe38" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:27.767130 14073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c762518c0d2c453a97841958622ffe38" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:27.768200 13800 raft_consensus.cc:3055] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:27.768601 14073 raft_consensus.cc:3060] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:27.768626 13800 raft_consensus.cc:740] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:27.770169 13800 consensus_queue.cc:260] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:27.771416 13800 raft_consensus.cc:3060] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:27.777100 13800 raft_consensus.cc:2468] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 3.
I20260709 12:29:27.778621 14073 raft_consensus.cc:2468] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 3.
I20260709 12:29:27.778702 13842 leader_election.cc:304] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, d6045f6899cf43f7aa7af074983fcae8; no voters: 
I20260709 12:29:27.779964 14155 raft_consensus.cc:2804] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:27.780514 14155 raft_consensus.cc:697] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Becoming Leader. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:27.781555 14155 consensus_queue.cc:237] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } }
I20260709 12:29:27.791827 13622 catalog_manager.cc:5697] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af reported cstate change: term changed from 2 to 3, leader changed from d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194) to 0a1b5a26007a4097887ffb7b875380af (127.3.89.195). New cstate: current_term: 3 leader_uuid: "0a1b5a26007a4097887ffb7b875380af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.978992 14073 raft_consensus.cc:1275] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 4 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:29:27.980406 14157 consensus_queue.cc:1048] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:27.998687 13874 raft_consensus.cc:1275] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 4 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:29:28.001012 14139 consensus_queue.cc:1048] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:28.012321 13876 raft_consensus.cc:1275] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.014448 14139 consensus_queue.cc:1048] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.023928 14073 raft_consensus.cc:1275] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.026901 14164 consensus_queue.cc:1048] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:29:28.074682 13673 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:28.212687 14073 raft_consensus.cc:1275] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:28.214208 14155 consensus_queue.cc:1048] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.225071 13800 raft_consensus.cc:1275] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:28.227177 14155 consensus_queue.cc:1048] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.431828  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:28.439877 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
mode: GRACEFUL
new_leader_uuid: "0a1b5a26007a4097887ffb7b875380af"
 from {username='slave'} at 127.0.0.1:35652
I20260709 12:29:28.440551 13800 raft_consensus.cc:606] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Received request to transfer leadership to TS 0a1b5a26007a4097887ffb7b875380af
I20260709 12:29:28.443470 13876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d21be6a87739412694ceb7ae4cf3e42d"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
mode: GRACEFUL
new_leader_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:40438
I20260709 12:29:28.444162 13876 raft_consensus.cc:606] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 2 LEADER]: Received request to transfer leadership to TS d6045f6899cf43f7aa7af074983fcae8
I20260709 12:29:28.447031 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f95b4e77affd440fafced08a62b6135d"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
mode: GRACEFUL
new_leader_uuid: "0a1b5a26007a4097887ffb7b875380af"
 from {username='slave'} at 127.0.0.1:35652
I20260709 12:29:28.447595 13800 raft_consensus.cc:606] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Received request to transfer leadership to TS 0a1b5a26007a4097887ffb7b875380af
I20260709 12:29:28.449097  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:28.449684  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:28.450451  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:28.493469 14139 raft_consensus.cc:993] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8: : Instructing follower 0a1b5a26007a4097887ffb7b875380af to start an election
I20260709 12:29:28.494215 14164 raft_consensus.cc:1081] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Signalling peer 0a1b5a26007a4097887ffb7b875380af to start an election
I20260709 12:29:28.495816 13874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f95b4e77affd440fafced08a62b6135d"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
 from {username='slave'} at 127.0.0.1:43570
I20260709 12:29:28.496284 13874 raft_consensus.cc:493] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:28.496573 13874 raft_consensus.cc:3060] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:28.502092 13874 raft_consensus.cc:515] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:28.503674 13874 leader_election.cc:290] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 3 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:28.504699 14073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95b4e77affd440fafced08a62b6135d" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:28.504866 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f95b4e77affd440fafced08a62b6135d" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:28.505334 14073 raft_consensus.cc:3060] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:28.505642 13802 raft_consensus.cc:3055] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:28.506045 13802 raft_consensus.cc:740] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:28.507217 13802 consensus_queue.cc:260] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:28.508548 13802 raft_consensus.cc:3060] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:28.512202 14073 raft_consensus.cc:2468] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 3.
I20260709 12:29:28.513660 13841 leader_election.cc:304] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:28.514416 14155 raft_consensus.cc:2804] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:28.514865 14155 raft_consensus.cc:697] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Becoming Leader. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:28.515829 14155 consensus_queue.cc:237] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:28.517545 13802 raft_consensus.cc:2468] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 3.
I20260709 12:29:28.527215 13624 catalog_manager.cc:5697] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af reported cstate change: term changed from 2 to 3, leader changed from d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194) to 0a1b5a26007a4097887ffb7b875380af (127.3.89.195). New cstate: current_term: 3 leader_uuid: "0a1b5a26007a4097887ffb7b875380af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:28.579005 14166 raft_consensus.cc:993] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8: : Instructing follower 0a1b5a26007a4097887ffb7b875380af to start an election
I20260709 12:29:28.579550 14164 raft_consensus.cc:1081] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Signalling peer 0a1b5a26007a4097887ffb7b875380af to start an election
I20260709 12:29:28.581298 13874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
 from {username='slave'} at 127.0.0.1:43570
I20260709 12:29:28.581890 13874 raft_consensus.cc:493] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:28.582208 13874 raft_consensus.cc:3060] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:28.587973 13874 raft_consensus.cc:515] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:28.589884 13874 leader_election.cc:290] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 3 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:28.590569 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:28.591253 13802 raft_consensus.cc:3055] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:28.591579 13802 raft_consensus.cc:740] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:28.592545 13802 consensus_queue.cc:260] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:28.593815 14073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:28.594161 13802 raft_consensus.cc:3060] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:28.594607 14073 raft_consensus.cc:3060] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:28.600389 14073 raft_consensus.cc:2468] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 3.
I20260709 12:29:28.601508 13841 leader_election.cc:304] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:28.602304 14155 raft_consensus.cc:2804] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:28.602783 14155 raft_consensus.cc:697] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Becoming Leader. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:28.603251 13802 raft_consensus.cc:2468] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 3.
I20260709 12:29:28.603914 14155 consensus_queue.cc:237] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:28.614147 13624 catalog_manager.cc:5697] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af reported cstate change: term changed from 2 to 3, leader changed from d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194) to 0a1b5a26007a4097887ffb7b875380af (127.3.89.195). New cstate: current_term: 3 leader_uuid: "0a1b5a26007a4097887ffb7b875380af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:28.856353 14132 raft_consensus.cc:993] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af: : Instructing follower d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:28.856920 14155 raft_consensus.cc:1081] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 2 LEADER]: Signalling peer d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:28.858588 13802 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d21be6a87739412694ceb7ae4cf3e42d"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:59058
I20260709 12:29:28.859261 13802 raft_consensus.cc:493] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:28.859768 13802 raft_consensus.cc:3060] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:28.868175 13802 raft_consensus.cc:515] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:28.870699 13802 leader_election.cc:290] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 3 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:28.871695 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21be6a87739412694ceb7ae4cf3e42d" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:28.872291 13874 raft_consensus.cc:3055] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:28.872329 14073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21be6a87739412694ceb7ae4cf3e42d" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:28.872684 13874 raft_consensus.cc:740] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:28.873032 14073 raft_consensus.cc:3060] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:28.873711 13874 consensus_queue.cc:260] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:28.875134 13874 raft_consensus.cc:3060] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:28.881089 14073 raft_consensus.cc:2468] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 3.
I20260709 12:29:28.882447 13874 raft_consensus.cc:2468] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 3.
I20260709 12:29:28.882615 13767 leader_election.cc:304] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:28.883549 14164 raft_consensus.cc:2804] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:28.884017 14164 raft_consensus.cc:697] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 3 LEADER]: Becoming Leader. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:28.885048 14164 consensus_queue.cc:237] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:28.893354 13624 catalog_manager.cc:5697] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 reported cstate change: term changed from 2 to 3, leader changed from 0a1b5a26007a4097887ffb7b875380af (127.3.89.195) to d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194). New cstate: current_term: 3 leader_uuid: "d6045f6899cf43f7aa7af074983fcae8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:28.896817 14073 raft_consensus.cc:1275] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:28.898648 14155 consensus_queue.cc:1048] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.920996 13802 raft_consensus.cc:1275] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:28.922585 14172 consensus_queue.cc:1048] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:29.076078 13673 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:29.089092 14073 raft_consensus.cc:1275] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:29.090744 14172 consensus_queue.cc:1048] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:29.101426 13802 raft_consensus.cc:1275] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:29.103487 14132 consensus_queue.cc:1048] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:29.279663 13874 raft_consensus.cc:1275] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:29.281306 14164 consensus_queue.cc:1048] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:29.292169 14073 raft_consensus.cc:1275] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:29.296144 14139 consensus_queue.cc:1048] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:29.457342  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:29.465811 13802 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "005c038223964ceb88b90a7698bc658e"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
mode: GRACEFUL
new_leader_uuid: "0a1b5a26007a4097887ffb7b875380af"
 from {username='slave'} at 127.0.0.1:35652
I20260709 12:29:29.466378 13802 raft_consensus.cc:606] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Received request to transfer leadership to TS 0a1b5a26007a4097887ffb7b875380af
I20260709 12:29:29.469343 13874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
mode: GRACEFUL
new_leader_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:40438
I20260709 12:29:29.469961 13874 raft_consensus.cc:606] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Received request to transfer leadership to TS d6045f6899cf43f7aa7af074983fcae8
I20260709 12:29:29.472720 13874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
mode: GRACEFUL
new_leader_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:40438
I20260709 12:29:29.473285 13874 raft_consensus.cc:606] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Received request to transfer leadership to TS d6045f6899cf43f7aa7af074983fcae8
I20260709 12:29:29.474480  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:29.475142  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:29.475838  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:29.485273 14172 raft_consensus.cc:993] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af: : Instructing follower d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:29.485882 14155 raft_consensus.cc:1081] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Signalling peer d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:29.487751 13802 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:59058
I20260709 12:29:29.488255 13802 raft_consensus.cc:493] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:29.488577 13802 raft_consensus.cc:3060] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:29.494534 13802 raft_consensus.cc:515] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:29.496256 13802 leader_election.cc:290] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 4 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:29.497452 14073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:29.498144 14073 raft_consensus.cc:3060] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:29.498064 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20cf0dad70be47f18b0d83766741777a" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:29.498788 13874 raft_consensus.cc:3055] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:29.499100 13874 raft_consensus.cc:740] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:29.499818 13874 consensus_queue.cc:260] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:29.501014 13874 raft_consensus.cc:3060] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:29.505825 14073 raft_consensus.cc:2468] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 4.
I20260709 12:29:29.507293 13767 leader_election.cc:304] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:29.508268 14190 raft_consensus.cc:2804] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:29.508491 13874 raft_consensus.cc:2468] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 4.
I20260709 12:29:29.508771 14190 raft_consensus.cc:697] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 4 LEADER]: Becoming Leader. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:29.510144 14190 consensus_queue.cc:237] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:29.516582 13624 catalog_manager.cc:5697] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 reported cstate change: term changed from 3 to 4, leader changed from 0a1b5a26007a4097887ffb7b875380af (127.3.89.195) to d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194). New cstate: current_term: 4 leader_uuid: "d6045f6899cf43f7aa7af074983fcae8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:29.836964 14172 raft_consensus.cc:993] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af: : Instructing follower d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:29.837641 14172 raft_consensus.cc:1081] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Signalling peer d6045f6899cf43f7aa7af074983fcae8 to start an election
I20260709 12:29:29.839324 13802 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd"
dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
 from {username='slave'} at 127.0.0.1:59058
I20260709 12:29:29.839789 13802 raft_consensus.cc:493] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:29.840063 13802 raft_consensus.cc:3060] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:29.846764 13802 raft_consensus.cc:515] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:29.849197 13802 leader_election.cc:290] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 4 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), 0a1b5a26007a4097887ffb7b875380af (127.3.89.195:40343)
I20260709 12:29:29.850369 13874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
I20260709 12:29:29.850379 14073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5cea694c7104f96a2431e7015a54edd" candidate_uuid: "d6045f6899cf43f7aa7af074983fcae8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:29.850992 13874 raft_consensus.cc:3055] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:29.851301 14073 raft_consensus.cc:3060] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:29.851341 13874 raft_consensus.cc:740] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:29.852284 13874 consensus_queue.cc:260] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:29.853294 13874 raft_consensus.cc:3060] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:29.857715 14073 raft_consensus.cc:2468] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 4.
I20260709 12:29:29.859225 13767 leader_election.cc:304] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6045f6899cf43f7aa7af074983fcae8, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:29.860199 13874 raft_consensus.cc:2468] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6045f6899cf43f7aa7af074983fcae8 in term 4.
I20260709 12:29:29.860294 14190 raft_consensus.cc:2804] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:29.860956 14190 raft_consensus.cc:697] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 4 LEADER]: Becoming Leader. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:29.861980 14190 consensus_queue.cc:237] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:29.869992 13624 catalog_manager.cc:5697] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 reported cstate change: term changed from 3 to 4, leader changed from 0a1b5a26007a4097887ffb7b875380af (127.3.89.195) to d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194). New cstate: current_term: 4 leader_uuid: "d6045f6899cf43f7aa7af074983fcae8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:29.873893 14164 raft_consensus.cc:993] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8: : Instructing follower 0a1b5a26007a4097887ffb7b875380af to start an election
I20260709 12:29:29.874502 14164 raft_consensus.cc:1081] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Signalling peer 0a1b5a26007a4097887ffb7b875380af to start an election
I20260709 12:29:29.876658 13874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "005c038223964ceb88b90a7698bc658e"
dest_uuid: "0a1b5a26007a4097887ffb7b875380af"
 from {username='slave'} at 127.0.0.1:43570
I20260709 12:29:29.877254 13874 raft_consensus.cc:493] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:29.877720 13874 raft_consensus.cc:3060] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.883358 13874 raft_consensus.cc:515] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:29.885250 13874 leader_election.cc:290] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 3 election: Requested vote from peers f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845), d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121)
I20260709 12:29:29.886724 14073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "005c038223964ceb88b90a7698bc658e" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f6bb736605e6460e91c7b8a131e0e84b"
I20260709 12:29:29.886771 13802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "005c038223964ceb88b90a7698bc658e" candidate_uuid: "0a1b5a26007a4097887ffb7b875380af" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d6045f6899cf43f7aa7af074983fcae8"
I20260709 12:29:29.887549 14073 raft_consensus.cc:3060] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.887828 13802 raft_consensus.cc:3055] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:29.888381 13802 raft_consensus.cc:740] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d6045f6899cf43f7aa7af074983fcae8, State: Running, Role: LEADER
I20260709 12:29:29.889644 13802 consensus_queue.cc:260] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:29.890722 13802 raft_consensus.cc:3060] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:29.894737 14073 raft_consensus.cc:2468] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 3.
I20260709 12:29:29.896117 13841 leader_election.cc:304] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1b5a26007a4097887ffb7b875380af, f6bb736605e6460e91c7b8a131e0e84b; no voters: 
I20260709 12:29:29.896872 14172 raft_consensus.cc:2804] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:29.897393 14172 raft_consensus.cc:697] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Becoming Leader. State: Replica: 0a1b5a26007a4097887ffb7b875380af, State: Running, Role: LEADER
I20260709 12:29:29.897971 13802 raft_consensus.cc:2468] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1b5a26007a4097887ffb7b875380af in term 3.
I20260709 12:29:29.898442 14172 consensus_queue.cc:237] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } }
I20260709 12:29:29.906325 13624 catalog_manager.cc:5697] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af reported cstate change: term changed from 2 to 3, leader changed from d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194) to 0a1b5a26007a4097887ffb7b875380af (127.3.89.195). New cstate: current_term: 3 leader_uuid: "0a1b5a26007a4097887ffb7b875380af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:29.991262 14073 raft_consensus.cc:1275] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 4 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:29:29.992517 14164 consensus_queue.cc:1048] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:30.003553 13874 raft_consensus.cc:1275] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 4 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:29:30.005024 14191 consensus_queue.cc:1048] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:29:30.077271 13673 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:30.280593 14073 raft_consensus.cc:1275] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 4 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:30.282040 14200 consensus_queue.cc:1048] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:30.285672 14074 raft_consensus.cc:1275] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:30.287206 14155 consensus_queue.cc:1048] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6bb736605e6460e91c7b8a131e0e84b" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:30.295024 13876 raft_consensus.cc:1275] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 4 FOLLOWER]: Refusing update from remote peer d6045f6899cf43f7aa7af074983fcae8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:30.296926 14200 consensus_queue.cc:1048] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1b5a26007a4097887ffb7b875380af" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:30.308811 13802 raft_consensus.cc:1275] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Refusing update from remote peer 0a1b5a26007a4097887ffb7b875380af: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:30.314504 14206 consensus_queue.cc:1048] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6045f6899cf43f7aa7af074983fcae8" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:30.483795  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:30.516451  3431 tablet_server.cc:179] TabletServer@127.3.89.193:45845 shutting down...
W20260709 12:29:30.541199 13841 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 97 similar messages]
W20260709 12:29:30.544910 13841 consensus_peers.cc:597] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:30.546082  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:30.546948  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:30.547817  3431 raft_consensus.cc:2243] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.548393  3431 raft_consensus.cc:2272] T 7ade9f433393457391d1bc7f19b224de P f6bb736605e6460e91c7b8a131e0e84b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.551227  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:30.552141  3431 raft_consensus.cc:2243] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.552695  3431 raft_consensus.cc:2272] T c5cea694c7104f96a2431e7015a54edd P f6bb736605e6460e91c7b8a131e0e84b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.557680  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:30.558575  3431 raft_consensus.cc:2243] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.559206  3431 raft_consensus.cc:2272] T 005c038223964ceb88b90a7698bc658e P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:30.560104 13767 consensus_peers.cc:597] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:30.562587  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:30.563282  3431 raft_consensus.cc:2243] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.563714  3431 raft_consensus.cc:2272] T a52eaa3c1df34e6c8a1ccea63bf39b1d P f6bb736605e6460e91c7b8a131e0e84b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.565975  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:30.566514  3431 raft_consensus.cc:2243] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.566985  3431 raft_consensus.cc:2272] T 20cf0dad70be47f18b0d83766741777a P f6bb736605e6460e91c7b8a131e0e84b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.569772  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:30.570363  3431 raft_consensus.cc:2243] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.570820  3431 raft_consensus.cc:2272] T 46731d8225284b3c85730af89cb0a871 P f6bb736605e6460e91c7b8a131e0e84b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.572999  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:30.573611  3431 raft_consensus.cc:2243] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.574237  3431 raft_consensus.cc:2272] T c762518c0d2c453a97841958622ffe38 P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.576422  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:30.576993  3431 raft_consensus.cc:2243] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.577446  3431 raft_consensus.cc:2272] T d21be6a87739412694ceb7ae4cf3e42d P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.579694  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:30.580224  3431 raft_consensus.cc:2243] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.580718  3431 raft_consensus.cc:2272] T c6bf9b18c2d047babed6c7b74b9dbf01 P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.584033  3431 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:30.584640  3431 raft_consensus.cc:2243] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.585132  3431 raft_consensus.cc:2272] T f95b4e77affd440fafced08a62b6135d P f6bb736605e6460e91c7b8a131e0e84b [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:30.588680 13767 consensus_peers.cc:597] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:30.621942  3431 tablet_server.cc:196] TabletServer@127.3.89.193:45845 shutdown complete.
I20260709 12:29:30.650775  3431 tablet_server.cc:179] TabletServer@127.3.89.194:0 shutting down...
W20260709 12:29:30.662626 13841 consensus_peers.cc:597] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af -> Peer f6bb736605e6460e91c7b8a131e0e84b (127.3.89.193:45845): Couldn't send request to peer f6bb736605e6460e91c7b8a131e0e84b. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:30.681103  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:30.682160  3431 tablet_replica.cc:333] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8: stopping tablet replica
I20260709 12:29:30.683322  3431 raft_consensus.cc:2243] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.684144  3431 raft_consensus.cc:2272] T a52eaa3c1df34e6c8a1ccea63bf39b1d P d6045f6899cf43f7aa7af074983fcae8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.687485  3431 tablet_replica.cc:333] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8: stopping tablet replica
I20260709 12:29:30.688738  3431 raft_consensus.cc:2243] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:30.689884  3431 raft_consensus.cc:2272] T 20cf0dad70be47f18b0d83766741777a P d6045f6899cf43f7aa7af074983fcae8 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:30.690750 13842 consensus_peers.cc:597] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af -> Peer d6045f6899cf43f7aa7af074983fcae8 (127.3.89.194:40121): Couldn't send request to peer d6045f6899cf43f7aa7af074983fcae8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:30.693557  3431 tablet_replica.cc:333] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8: stopping tablet replica
I20260709 12:29:30.694406  3431 raft_consensus.cc:2243] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:30.695340  3431 raft_consensus.cc:2272] T c5cea694c7104f96a2431e7015a54edd P d6045f6899cf43f7aa7af074983fcae8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.698432  3431 tablet_replica.cc:333] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8: stopping tablet replica
I20260709 12:29:30.699110  3431 raft_consensus.cc:2243] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.699602  3431 raft_consensus.cc:2272] T f95b4e77affd440fafced08a62b6135d P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.702060  3431 tablet_replica.cc:333] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8: stopping tablet replica
I20260709 12:29:30.702679  3431 raft_consensus.cc:2243] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.703145  3431 raft_consensus.cc:2272] T c762518c0d2c453a97841958622ffe38 P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.705444  3431 tablet_replica.cc:333] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8: stopping tablet replica
I20260709 12:29:30.706152  3431 raft_consensus.cc:2243] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:30.707000  3431 raft_consensus.cc:2272] T d21be6a87739412694ceb7ae4cf3e42d P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.709667  3431 tablet_replica.cc:333] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8: stopping tablet replica
I20260709 12:29:30.710278  3431 raft_consensus.cc:2243] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.710840  3431 raft_consensus.cc:2272] T 46731d8225284b3c85730af89cb0a871 P d6045f6899cf43f7aa7af074983fcae8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.713208  3431 tablet_replica.cc:333] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8: stopping tablet replica
I20260709 12:29:30.713951  3431 raft_consensus.cc:2243] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:30.714888  3431 raft_consensus.cc:2272] T 7ade9f433393457391d1bc7f19b224de P d6045f6899cf43f7aa7af074983fcae8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.717715  3431 tablet_replica.cc:333] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8: stopping tablet replica
I20260709 12:29:30.718354  3431 raft_consensus.cc:2243] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:30.719309  3431 raft_consensus.cc:2272] T c6bf9b18c2d047babed6c7b74b9dbf01 P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.721679  3431 tablet_replica.cc:333] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8: stopping tablet replica
I20260709 12:29:30.722261  3431 raft_consensus.cc:2243] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.722728  3431 raft_consensus.cc:2272] T 005c038223964ceb88b90a7698bc658e P d6045f6899cf43f7aa7af074983fcae8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.757509  3431 tablet_server.cc:196] TabletServer@127.3.89.194:0 shutdown complete.
I20260709 12:29:30.784561  3431 tablet_server.cc:179] TabletServer@127.3.89.195:0 shutting down...
I20260709 12:29:30.816783  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:30.817559  3431 tablet_replica.cc:333] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af: stopping tablet replica
I20260709 12:29:30.818397  3431 raft_consensus.cc:2243] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.818954  3431 raft_consensus.cc:2272] T c6bf9b18c2d047babed6c7b74b9dbf01 P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.821245  3431 tablet_replica.cc:333] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af: stopping tablet replica
I20260709 12:29:30.821930  3431 raft_consensus.cc:2243] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.822444  3431 raft_consensus.cc:2272] T c5cea694c7104f96a2431e7015a54edd P 0a1b5a26007a4097887ffb7b875380af [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.825093  3431 tablet_replica.cc:333] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af: stopping tablet replica
I20260709 12:29:30.825745  3431 raft_consensus.cc:2243] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:30.826613  3431 raft_consensus.cc:2272] T a52eaa3c1df34e6c8a1ccea63bf39b1d P 0a1b5a26007a4097887ffb7b875380af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.828714  3431 tablet_replica.cc:333] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af: stopping tablet replica
I20260709 12:29:30.829288  3431 raft_consensus.cc:2243] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:30.830137  3431 raft_consensus.cc:2272] T 005c038223964ceb88b90a7698bc658e P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.832612  3431 tablet_replica.cc:333] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af: stopping tablet replica
I20260709 12:29:30.833215  3431 raft_consensus.cc:2243] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.833694  3431 raft_consensus.cc:2272] T 20cf0dad70be47f18b0d83766741777a P 0a1b5a26007a4097887ffb7b875380af [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.836313  3431 tablet_replica.cc:333] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af: stopping tablet replica
I20260709 12:29:30.836886  3431 raft_consensus.cc:2243] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.837337  3431 raft_consensus.cc:2272] T 7ade9f433393457391d1bc7f19b224de P 0a1b5a26007a4097887ffb7b875380af [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.839432  3431 tablet_replica.cc:333] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af: stopping tablet replica
I20260709 12:29:30.839952  3431 raft_consensus.cc:2243] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:30.840798  3431 raft_consensus.cc:2272] T 46731d8225284b3c85730af89cb0a871 P 0a1b5a26007a4097887ffb7b875380af [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.843642  3431 tablet_replica.cc:333] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af: stopping tablet replica
I20260709 12:29:30.844376  3431 raft_consensus.cc:2243] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:30.845140  3431 raft_consensus.cc:2272] T c762518c0d2c453a97841958622ffe38 P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.848446  3431 tablet_replica.cc:333] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af: stopping tablet replica
I20260709 12:29:30.849174  3431 raft_consensus.cc:2243] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:30.849665  3431 raft_consensus.cc:2272] T d21be6a87739412694ceb7ae4cf3e42d P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.851868  3431 tablet_replica.cc:333] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af: stopping tablet replica
I20260709 12:29:30.852330  3431 raft_consensus.cc:2243] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:30.853101  3431 raft_consensus.cc:2272] T f95b4e77affd440fafced08a62b6135d P 0a1b5a26007a4097887ffb7b875380af [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.887468  3431 tablet_server.cc:196] TabletServer@127.3.89.195:0 shutdown complete.
I20260709 12:29:30.914963  3431 master.cc:562] Master@127.3.89.254:33703 shutting down...
I20260709 12:29:30.938746  3431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:30.939414  3431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:30.939731  3431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 44291da4ca1944c699c48c997f26cfdc: stopping tablet replica
I20260709 12:29:30.979198  3431 master.cc:584] Master@127.3.89.254:33703 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13114 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:29:31.025919  3431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.3.89.254:36899
I20260709 12:29:31.027056  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:31.033166 14215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:31.033777 14216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:31.035032  3431 server_base.cc:1061] running on GCE node
W20260709 12:29:31.035120 14218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:31.036178  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:31.036448  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:31.036624  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600171036604 us; error 0 us; skew 500 ppm
I20260709 12:29:31.037213  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:31.040063  3431 webserver.cc:533] Webserver started at http://127.3.89.254:35337/ using document root <none> and password file <none>
I20260709 12:29:31.040632  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:31.040850  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:31.041157  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:31.042434  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/master-0-root/instance:
uuid: "c30fdc1bf8e849d38e95cd4130b1aa51"
format_stamp: "Formatted at 2026-07-09 12:29:31 on dist-test-slave-3hrd"
I20260709 12:29:31.047219  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 12:29:31.050985 14223 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:31.052094  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:29:31.052389  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/master-0-root
uuid: "c30fdc1bf8e849d38e95cd4130b1aa51"
format_stamp: "Formatted at 2026-07-09 12:29:31 on dist-test-slave-3hrd"
I20260709 12:29:31.052672  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:31.075711  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:31.077032  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:31.121140  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:36899
I20260709 12:29:31.121256 14274 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.254:36899 every 8 connection(s)
I20260709 12:29:31.125795 14275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.138897 14275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51: Bootstrap starting.
I20260709 12:29:31.144523 14275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.149286 14275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51: No bootstrap required, opened a new log
I20260709 12:29:31.151788 14275 raft_consensus.cc:359] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30fdc1bf8e849d38e95cd4130b1aa51" member_type: VOTER }
I20260709 12:29:31.152248 14275 raft_consensus.cc:385] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.152504 14275 raft_consensus.cc:740] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c30fdc1bf8e849d38e95cd4130b1aa51, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.153262 14275 consensus_queue.cc:260] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30fdc1bf8e849d38e95cd4130b1aa51" member_type: VOTER }
I20260709 12:29:31.153821 14275 raft_consensus.cc:399] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:31.154116 14275 raft_consensus.cc:493] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:31.154505 14275 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.160068 14275 raft_consensus.cc:515] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30fdc1bf8e849d38e95cd4130b1aa51" member_type: VOTER }
I20260709 12:29:31.160728 14275 leader_election.cc:304] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c30fdc1bf8e849d38e95cd4130b1aa51; no voters: 
I20260709 12:29:31.162024 14275 leader_election.cc:290] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:31.162370 14278 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:31.163822 14278 raft_consensus.cc:697] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [term 1 LEADER]: Becoming Leader. State: Replica: c30fdc1bf8e849d38e95cd4130b1aa51, State: Running, Role: LEADER
I20260709 12:29:31.164536 14278 consensus_queue.cc:237] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30fdc1bf8e849d38e95cd4130b1aa51" member_type: VOTER }
I20260709 12:29:31.165468 14275 sys_catalog.cc:565] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:31.169687 14280 sys_catalog.cc:455] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c30fdc1bf8e849d38e95cd4130b1aa51. Latest consensus state: current_term: 1 leader_uuid: "c30fdc1bf8e849d38e95cd4130b1aa51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30fdc1bf8e849d38e95cd4130b1aa51" member_type: VOTER } }
I20260709 12:29:31.170554 14280 sys_catalog.cc:458] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:31.169611 14279 sys_catalog.cc:455] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c30fdc1bf8e849d38e95cd4130b1aa51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30fdc1bf8e849d38e95cd4130b1aa51" member_type: VOTER } }
I20260709 12:29:31.172770 14279 sys_catalog.cc:458] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:31.172791 14285 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:31.178005 14285 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:31.183319  3431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:31.187875 14285 catalog_manager.cc:1383] Generated new cluster ID: 73eb1858ba144b389703146882dadce1
I20260709 12:29:31.188241 14285 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:31.207096 14285 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:31.208561 14285 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:31.229657 14285 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51: Generated new TSK 0
I20260709 12:29:31.230396 14285 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:31.254118  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:31.261624 14296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:31.262831 14297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:31.267374  3431 server_base.cc:1061] running on GCE node
W20260709 12:29:31.267567 14300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:31.274049  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:31.274317  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:31.274497  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600171274479 us; error 0 us; skew 500 ppm
I20260709 12:29:31.275300  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:31.278227  3431 webserver.cc:533] Webserver started at http://127.3.89.193:42545/ using document root <none> and password file <none>
I20260709 12:29:31.278748  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:31.278970  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:31.279258  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:31.280371  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-0-root/instance:
uuid: "e38530b0df45448e97a389793a29d29c"
format_stamp: "Formatted at 2026-07-09 12:29:31 on dist-test-slave-3hrd"
I20260709 12:29:31.284971  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:29:31.288767 14304 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:31.289816  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 12:29:31.290159  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-0-root
uuid: "e38530b0df45448e97a389793a29d29c"
format_stamp: "Formatted at 2026-07-09 12:29:31 on dist-test-slave-3hrd"
I20260709 12:29:31.290472  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:31.308087  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:31.309381  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:31.311007  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:31.313397  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:31.313656  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:29:31.313947  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:31.314185  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:31.363345  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:42879
I20260709 12:29:31.363488 14366 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.193:42879 every 8 connection(s)
I20260709 12:29:31.368925  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:31.377995 14371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:31.379040 14372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:31.382400 14367 heartbeater.cc:344] Connected to a master server at 127.3.89.254:36899
I20260709 12:29:31.382931 14367 heartbeater.cc:461] Registering TS with master...
W20260709 12:29:31.384021 14374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:31.384032 14367 heartbeater.cc:507] Master 127.3.89.254:36899 requested a full tablet report, sending...
I20260709 12:29:31.384632  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:31.385907  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:31.386158  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:31.386365  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600171386346 us; error 0 us; skew 500 ppm
I20260709 12:29:31.386947  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:31.387020 14240 ts_manager.cc:194] Registered new tserver with Master: e38530b0df45448e97a389793a29d29c (127.3.89.193:42879)
I20260709 12:29:31.389410 14240 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35248
I20260709 12:29:31.389708  3431 webserver.cc:533] Webserver started at http://127.3.89.194:41009/ using document root <none> and password file <none>
I20260709 12:29:31.390311  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:31.390477  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:31.390799  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:31.391942  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-1-root/instance:
uuid: "1d4801bb16b1436691a919cb98cead33"
format_stamp: "Formatted at 2026-07-09 12:29:31 on dist-test-slave-3hrd"
I20260709 12:29:31.396764  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:29:31.400570 14379 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:31.401489  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:29:31.401878  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-1-root
uuid: "1d4801bb16b1436691a919cb98cead33"
format_stamp: "Formatted at 2026-07-09 12:29:31 on dist-test-slave-3hrd"
I20260709 12:29:31.402190  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:31.417781  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:31.418959  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:31.420485  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:31.422804  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:31.423043  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:31.423285  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:31.423455  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:31.465161  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:41079
I20260709 12:29:31.465270 14442 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.194:41079 every 8 connection(s)
I20260709 12:29:31.470153  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:31.477253 14446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:31.478472 14447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:31.481045 14449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:31.482306  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:31.483353 14443 heartbeater.cc:344] Connected to a master server at 127.3.89.254:36899
I20260709 12:29:31.483371  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:31.483817  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:31.483839 14443 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:31.484054  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600171484040 us; error 0 us; skew 500 ppm
I20260709 12:29:31.484859  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:31.485000 14443 heartbeater.cc:507] Master 127.3.89.254:36899 requested a full tablet report, sending...
I20260709 12:29:31.487478 14240 ts_manager.cc:194] Registered new tserver with Master: 1d4801bb16b1436691a919cb98cead33 (127.3.89.194:41079)
I20260709 12:29:31.487847  3431 webserver.cc:533] Webserver started at http://127.3.89.195:36243/ using document root <none> and password file <none>
I20260709 12:29:31.488404  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:31.488603  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:31.488879  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:31.489130 14240 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35258
I20260709 12:29:31.490137  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-2-root/instance:
uuid: "6033ddf10fd94a9b92e4777bfa02cc94"
format_stamp: "Formatted at 2026-07-09 12:29:31 on dist-test-slave-3hrd"
I20260709 12:29:31.494983  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:29:31.498404 14454 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:31.499197  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:29:31.499452  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-2-root
uuid: "6033ddf10fd94a9b92e4777bfa02cc94"
format_stamp: "Formatted at 2026-07-09 12:29:31 on dist-test-slave-3hrd"
I20260709 12:29:31.499711  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599794910850-3431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:31.512888  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:31.514125  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:31.515640  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:31.517994  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:31.518226  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:31.518461  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:31.518630  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:31.561182  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:37519
I20260709 12:29:31.561332 14516 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.195:37519 every 8 connection(s)
I20260709 12:29:31.576664 14517 heartbeater.cc:344] Connected to a master server at 127.3.89.254:36899
I20260709 12:29:31.577101 14517 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:31.577905 14517 heartbeater.cc:507] Master 127.3.89.254:36899 requested a full tablet report, sending...
I20260709 12:29:31.579805 14240 ts_manager.cc:194] Registered new tserver with Master: 6033ddf10fd94a9b92e4777bfa02cc94 (127.3.89.195:37519)
I20260709 12:29:31.579939  3431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014772049s
I20260709 12:29:31.581207 14240 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35262
I20260709 12:29:31.610242 14240 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35272:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:31.612448 14240 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:31.662870 14407 tablet_service.cc:1511] Processing CreateTablet for tablet c9deb6b3cccc4caab4337a16eedffcfa (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=493f6845aeb74da58b942f68e5e76f8b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:31.663106 14406 tablet_service.cc:1511] Processing CreateTablet for tablet 87edaccfa2ce49c0bfe6590bcff29293 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=493f6845aeb74da58b942f68e5e76f8b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:31.664664 14407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9deb6b3cccc4caab4337a16eedffcfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.668097 14406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87edaccfa2ce49c0bfe6590bcff29293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.676152 14332 tablet_service.cc:1511] Processing CreateTablet for tablet c9deb6b3cccc4caab4337a16eedffcfa (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=493f6845aeb74da58b942f68e5e76f8b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:31.677682 14332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9deb6b3cccc4caab4337a16eedffcfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.682947 14331 tablet_service.cc:1511] Processing CreateTablet for tablet 87edaccfa2ce49c0bfe6590bcff29293 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=493f6845aeb74da58b942f68e5e76f8b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:31.689118 14331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87edaccfa2ce49c0bfe6590bcff29293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.690591 14532 tablet_bootstrap.cc:492] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33: Bootstrap starting.
I20260709 12:29:31.695281 14481 tablet_service.cc:1511] Processing CreateTablet for tablet 87edaccfa2ce49c0bfe6590bcff29293 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=493f6845aeb74da58b942f68e5e76f8b]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:31.696911 14481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87edaccfa2ce49c0bfe6590bcff29293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.699745 14482 tablet_service.cc:1511] Processing CreateTablet for tablet c9deb6b3cccc4caab4337a16eedffcfa (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=493f6845aeb74da58b942f68e5e76f8b]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:31.701298 14482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9deb6b3cccc4caab4337a16eedffcfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.702605 14532 tablet_bootstrap.cc:654] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.703890 14533 tablet_bootstrap.cc:492] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c: Bootstrap starting.
I20260709 12:29:31.711411 14533 tablet_bootstrap.cc:654] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.723276 14532 tablet_bootstrap.cc:492] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33: No bootstrap required, opened a new log
I20260709 12:29:31.723871 14532 ts_tablet_manager.cc:1403] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.012s
I20260709 12:29:31.729095 14533 tablet_bootstrap.cc:492] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c: No bootstrap required, opened a new log
I20260709 12:29:31.729682 14536 tablet_bootstrap.cc:492] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94: Bootstrap starting.
I20260709 12:29:31.729805 14533 ts_tablet_manager.cc:1403] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c: Time spent bootstrapping tablet: real 0.026s	user 0.023s	sys 0.000s
I20260709 12:29:31.729806 14532 raft_consensus.cc:359] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.730875 14532 raft_consensus.cc:385] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.731216 14532 raft_consensus.cc:740] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d4801bb16b1436691a919cb98cead33, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.732028 14532 consensus_queue.cc:260] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.733740 14533 raft_consensus.cc:359] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.734714 14533 raft_consensus.cc:385] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.734771 14443 heartbeater.cc:499] Master 127.3.89.254:36899 was elected leader, sending a full tablet report...
I20260709 12:29:31.735178 14533 raft_consensus.cc:740] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e38530b0df45448e97a389793a29d29c, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.735740 14532 ts_tablet_manager.cc:1434] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.006s
I20260709 12:29:31.736277 14533 consensus_queue.cc:260] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.737161 14536 tablet_bootstrap.cc:654] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.736860 14532 tablet_bootstrap.cc:492] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33: Bootstrap starting.
I20260709 12:29:31.745258 14532 tablet_bootstrap.cc:654] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.760710 14367 heartbeater.cc:499] Master 127.3.89.254:36899 was elected leader, sending a full tablet report...
I20260709 12:29:31.761057 14533 ts_tablet_manager.cc:1434] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.023s
I20260709 12:29:31.762280 14533 tablet_bootstrap.cc:492] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c: Bootstrap starting.
I20260709 12:29:31.763403 14532 tablet_bootstrap.cc:492] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33: No bootstrap required, opened a new log
I20260709 12:29:31.763830 14536 tablet_bootstrap.cc:492] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94: No bootstrap required, opened a new log
I20260709 12:29:31.764075 14532 ts_tablet_manager.cc:1403] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.007s
I20260709 12:29:31.764411 14536 ts_tablet_manager.cc:1403] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.012s
I20260709 12:29:31.767867 14532 raft_consensus.cc:359] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.768180 14536 raft_consensus.cc:359] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.768828 14532 raft_consensus.cc:385] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.769025 14536 raft_consensus.cc:385] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.769289 14532 raft_consensus.cc:740] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d4801bb16b1436691a919cb98cead33, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.769430 14533 tablet_bootstrap.cc:654] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.769495 14536 raft_consensus.cc:740] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6033ddf10fd94a9b92e4777bfa02cc94, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.770321 14532 consensus_queue.cc:260] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.770532 14536 consensus_queue.cc:260] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.773259 14532 ts_tablet_manager.cc:1434] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:29:31.779610 14533 tablet_bootstrap.cc:492] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c: No bootstrap required, opened a new log
I20260709 12:29:31.780529 14533 ts_tablet_manager.cc:1403] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 12:29:31.782444 14538 raft_consensus.cc:493] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.783241 14538 raft_consensus.cc:515] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.786864 14538 leader_election.cc:290] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d4801bb16b1436691a919cb98cead33 (127.3.89.194:41079), 6033ddf10fd94a9b92e4777bfa02cc94 (127.3.89.195:37519)
I20260709 12:29:31.789108 14517 heartbeater.cc:499] Master 127.3.89.254:36899 was elected leader, sending a full tablet report...
I20260709 12:29:31.790213 14541 raft_consensus.cc:493] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.791118 14536 ts_tablet_manager.cc:1434] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94: Time spent starting tablet: real 0.026s	user 0.018s	sys 0.007s
I20260709 12:29:31.790799 14541 raft_consensus.cc:515] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.792325 14536 tablet_bootstrap.cc:492] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94: Bootstrap starting.
I20260709 12:29:31.797127 14533 raft_consensus.cc:359] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.803783 14533 raft_consensus.cc:385] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.804280 14533 raft_consensus.cc:740] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e38530b0df45448e97a389793a29d29c, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.804347 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9deb6b3cccc4caab4337a16eedffcfa" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d4801bb16b1436691a919cb98cead33" is_pre_election: true
I20260709 12:29:31.805207 14418 raft_consensus.cc:2468] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 0.
I20260709 12:29:31.805347 14533 consensus_queue.cc:260] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.807109 14305 leader_election.cc:304] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d4801bb16b1436691a919cb98cead33, e38530b0df45448e97a389793a29d29c; no voters: 
I20260709 12:29:31.808000 14533 ts_tablet_manager.cc:1434] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 12:29:31.808416 14538 raft_consensus.cc:2804] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:31.799580 14536 tablet_bootstrap.cc:654] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.808825 14538 raft_consensus.cc:493] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:31.809231 14538 raft_consensus.cc:3060] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.816738 14492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9deb6b3cccc4caab4337a16eedffcfa" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" is_pre_election: true
I20260709 12:29:31.817936 14492 raft_consensus.cc:2468] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 0.
I20260709 12:29:31.819821 14538 raft_consensus.cc:515] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.822984 14541 leader_election.cc:290] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d4801bb16b1436691a919cb98cead33 (127.3.89.194:41079), e38530b0df45448e97a389793a29d29c (127.3.89.193:42879)
I20260709 12:29:31.826676 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9deb6b3cccc4caab4337a16eedffcfa" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d4801bb16b1436691a919cb98cead33"
I20260709 12:29:31.827374 14418 raft_consensus.cc:3060] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.834889 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9deb6b3cccc4caab4337a16eedffcfa" candidate_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d4801bb16b1436691a919cb98cead33" is_pre_election: true
I20260709 12:29:31.836607 14492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9deb6b3cccc4caab4337a16eedffcfa" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6033ddf10fd94a9b92e4777bfa02cc94"
I20260709 12:29:31.837625 14492 raft_consensus.cc:3060] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.839705 14418 raft_consensus.cc:2468] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 1.
I20260709 12:29:31.843564 14305 leader_election.cc:304] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d4801bb16b1436691a919cb98cead33, e38530b0df45448e97a389793a29d29c; no voters: 
I20260709 12:29:31.843950 14342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9deb6b3cccc4caab4337a16eedffcfa" candidate_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e38530b0df45448e97a389793a29d29c" is_pre_election: true
I20260709 12:29:31.844652 14538 leader_election.cc:290] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 1 election: Requested vote from peers 1d4801bb16b1436691a919cb98cead33 (127.3.89.194:41079), 6033ddf10fd94a9b92e4777bfa02cc94 (127.3.89.195:37519)
I20260709 12:29:31.845014 14342 raft_consensus.cc:2393] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6033ddf10fd94a9b92e4777bfa02cc94 in current term 1: Already voted for candidate e38530b0df45448e97a389793a29d29c in this term.
I20260709 12:29:31.845600 14538 raft_consensus.cc:2804] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:31.846187 14538 raft_consensus.cc:697] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 1 LEADER]: Becoming Leader. State: Replica: e38530b0df45448e97a389793a29d29c, State: Running, Role: LEADER
I20260709 12:29:31.846820 14457 leader_election.cc:304] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6033ddf10fd94a9b92e4777bfa02cc94; no voters: 1d4801bb16b1436691a919cb98cead33, e38530b0df45448e97a389793a29d29c
I20260709 12:29:31.847118 14538 consensus_queue.cc:237] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.848400 14492 raft_consensus.cc:2468] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 1.
I20260709 12:29:31.848989 14541 raft_consensus.cc:2749] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:31.849241 14536 tablet_bootstrap.cc:492] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94: No bootstrap required, opened a new log
I20260709 12:29:31.849864 14536 ts_tablet_manager.cc:1403] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94: Time spent bootstrapping tablet: real 0.058s	user 0.022s	sys 0.021s
I20260709 12:29:31.852826 14536 raft_consensus.cc:359] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.853415 14536 raft_consensus.cc:385] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:31.853760 14536 raft_consensus.cc:740] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6033ddf10fd94a9b92e4777bfa02cc94, State: Initialized, Role: FOLLOWER
I20260709 12:29:31.854431 14536 consensus_queue.cc:260] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.856324 14536 ts_tablet_manager.cc:1434] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 12:29:31.856819 14239 catalog_manager.cc:5697] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c reported cstate change: term changed from 0 to 1, leader changed from <none> to e38530b0df45448e97a389793a29d29c (127.3.89.193). New cstate: current_term: 1 leader_uuid: "e38530b0df45448e97a389793a29d29c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:31.865592 14538 raft_consensus.cc:493] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:31.866215 14538 raft_consensus.cc:515] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.868217 14538 leader_election.cc:290] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d4801bb16b1436691a919cb98cead33 (127.3.89.194:41079), 6033ddf10fd94a9b92e4777bfa02cc94 (127.3.89.195:37519)
I20260709 12:29:31.868891 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87edaccfa2ce49c0bfe6590bcff29293" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d4801bb16b1436691a919cb98cead33" is_pre_election: true
I20260709 12:29:31.869315 14492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87edaccfa2ce49c0bfe6590bcff29293" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" is_pre_election: true
I20260709 12:29:31.870168 14418 raft_consensus.cc:2468] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 0.
I20260709 12:29:31.870229 14492 raft_consensus.cc:2468] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 0.
I20260709 12:29:31.871511 14305 leader_election.cc:304] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d4801bb16b1436691a919cb98cead33, e38530b0df45448e97a389793a29d29c; no voters: 
I20260709 12:29:31.872172 14538 raft_consensus.cc:2804] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:31.872488 14538 raft_consensus.cc:493] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:31.872777 14538 raft_consensus.cc:3060] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.878132 14538 raft_consensus.cc:515] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.879704 14538 leader_election.cc:290] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 1 election: Requested vote from peers 1d4801bb16b1436691a919cb98cead33 (127.3.89.194:41079), 6033ddf10fd94a9b92e4777bfa02cc94 (127.3.89.195:37519)
I20260709 12:29:31.880541 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87edaccfa2ce49c0bfe6590bcff29293" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d4801bb16b1436691a919cb98cead33"
I20260709 12:29:31.880692 14492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87edaccfa2ce49c0bfe6590bcff29293" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6033ddf10fd94a9b92e4777bfa02cc94"
I20260709 12:29:31.881060 14418 raft_consensus.cc:3060] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.881306 14492 raft_consensus.cc:3060] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:31.886315 14418 raft_consensus.cc:2468] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 1.
I20260709 12:29:31.886338 14492 raft_consensus.cc:2468] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 1.
I20260709 12:29:31.887492 14305 leader_election.cc:304] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d4801bb16b1436691a919cb98cead33, e38530b0df45448e97a389793a29d29c; no voters: 
I20260709 12:29:31.888181 14538 raft_consensus.cc:2804] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:31.888592 14538 raft_consensus.cc:697] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 1 LEADER]: Becoming Leader. State: Replica: e38530b0df45448e97a389793a29d29c, State: Running, Role: LEADER
I20260709 12:29:31.889369 14538 consensus_queue.cc:237] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:31.896725 14238 catalog_manager.cc:5697] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c reported cstate change: term changed from 0 to 1, leader changed from <none> to e38530b0df45448e97a389793a29d29c (127.3.89.193). New cstate: current_term: 1 leader_uuid: "e38530b0df45448e97a389793a29d29c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:31.941541 14238 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35278:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:31.943598 14238 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:31.966804 14331 tablet_service.cc:1511] Processing CreateTablet for tablet 0ca6bd4816684a7db4e6f30b21d12704 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b20682b8cc2a4422a212e620b1f0d1f9]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:31.968183 14331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ca6bd4816684a7db4e6f30b21d12704. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.967959 14407 tablet_service.cc:1511] Processing CreateTablet for tablet 0ca6bd4816684a7db4e6f30b21d12704 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b20682b8cc2a4422a212e620b1f0d1f9]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:31.969238 14407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ca6bd4816684a7db4e6f30b21d12704. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.969753 14482 tablet_service.cc:1511] Processing CreateTablet for tablet 0ca6bd4816684a7db4e6f30b21d12704 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b20682b8cc2a4422a212e620b1f0d1f9]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:31.971026 14482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ca6bd4816684a7db4e6f30b21d12704. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.971482 14332 tablet_service.cc:1511] Processing CreateTablet for tablet 2f200e87cfa24f5296f36b2de1c00922 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b20682b8cc2a4422a212e620b1f0d1f9]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:31.972671 14332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f200e87cfa24f5296f36b2de1c00922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.975339 14406 tablet_service.cc:1511] Processing CreateTablet for tablet 2f200e87cfa24f5296f36b2de1c00922 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b20682b8cc2a4422a212e620b1f0d1f9]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:31.976593 14406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f200e87cfa24f5296f36b2de1c00922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.978572 14481 tablet_service.cc:1511] Processing CreateTablet for tablet 2f200e87cfa24f5296f36b2de1c00922 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b20682b8cc2a4422a212e620b1f0d1f9]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:31.979851 14481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f200e87cfa24f5296f36b2de1c00922. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:31.989488 14536 tablet_bootstrap.cc:492] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94: Bootstrap starting.
I20260709 12:29:31.990903 14532 tablet_bootstrap.cc:492] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33: Bootstrap starting.
I20260709 12:29:31.995086 14533 tablet_bootstrap.cc:492] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c: Bootstrap starting.
I20260709 12:29:31.996413 14536 tablet_bootstrap.cc:654] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:31.998556 14532 tablet_bootstrap.cc:654] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.004559 14533 tablet_bootstrap.cc:654] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.009176 14532 tablet_bootstrap.cc:492] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33: No bootstrap required, opened a new log
I20260709 12:29:32.009716 14532 ts_tablet_manager.cc:1403] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.008s
I20260709 12:29:32.012779 14532 raft_consensus.cc:359] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.013252 14533 tablet_bootstrap.cc:492] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c: No bootstrap required, opened a new log
I20260709 12:29:32.013531 14532 raft_consensus.cc:385] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.013912 14533 ts_tablet_manager.cc:1403] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.000s
I20260709 12:29:32.013957 14532 raft_consensus.cc:740] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d4801bb16b1436691a919cb98cead33, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.016072 14532 consensus_queue.cc:260] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.017022 14536 tablet_bootstrap.cc:492] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94: No bootstrap required, opened a new log
I20260709 12:29:32.017536 14536 ts_tablet_manager.cc:1403] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.000s
I20260709 12:29:32.018515 14533 raft_consensus.cc:359] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.019301 14533 raft_consensus.cc:385] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.019661 14533 raft_consensus.cc:740] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e38530b0df45448e97a389793a29d29c, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.020603 14533 consensus_queue.cc:260] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.020578 14536 raft_consensus.cc:359] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.023767 14533 ts_tablet_manager.cc:1434] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:32.024946 14533 tablet_bootstrap.cc:492] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c: Bootstrap starting.
I20260709 12:29:32.022411 14536 raft_consensus.cc:385] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.025977 14536 raft_consensus.cc:740] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6033ddf10fd94a9b92e4777bfa02cc94, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.027143 14536 consensus_queue.cc:260] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.027890 14532 ts_tablet_manager.cc:1434] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:29:32.029110 14532 tablet_bootstrap.cc:492] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33: Bootstrap starting.
I20260709 12:29:32.032560 14533 tablet_bootstrap.cc:654] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.037194 14532 tablet_bootstrap.cc:654] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.038950 14536 ts_tablet_manager.cc:1434] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94: Time spent starting tablet: real 0.021s	user 0.003s	sys 0.003s
I20260709 12:29:32.040521 14536 tablet_bootstrap.cc:492] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94: Bootstrap starting.
I20260709 12:29:32.044480 14533 tablet_bootstrap.cc:492] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c: No bootstrap required, opened a new log
I20260709 12:29:32.045253 14533 ts_tablet_manager.cc:1403] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.000s
I20260709 12:29:32.047219 14536 tablet_bootstrap.cc:654] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:32.049001 14533 raft_consensus.cc:359] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.049806 14533 raft_consensus.cc:385] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.050133 14533 raft_consensus.cc:740] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e38530b0df45448e97a389793a29d29c, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.051188 14533 consensus_queue.cc:260] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.052664 14532 tablet_bootstrap.cc:492] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33: No bootstrap required, opened a new log
I20260709 12:29:32.053292 14532 ts_tablet_manager.cc:1403] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.000s
I20260709 12:29:32.053429 14533 ts_tablet_manager.cc:1434] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:29:32.053936 14536 tablet_bootstrap.cc:492] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94: No bootstrap required, opened a new log
I20260709 12:29:32.054507 14536 ts_tablet_manager.cc:1403] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 12:29:32.056504 14532 raft_consensus.cc:359] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.057335 14532 raft_consensus.cc:385] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.057688 14532 raft_consensus.cc:740] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d4801bb16b1436691a919cb98cead33, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.057523 14536 raft_consensus.cc:359] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.058363 14536 raft_consensus.cc:385] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:32.058706 14536 raft_consensus.cc:740] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6033ddf10fd94a9b92e4777bfa02cc94, State: Initialized, Role: FOLLOWER
I20260709 12:29:32.058558 14532 consensus_queue.cc:260] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.059559 14536 consensus_queue.cc:260] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.061035 14532 ts_tablet_manager.cc:1434] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:32.061896 14536 ts_tablet_manager.cc:1434] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:32.064109 14537 raft_consensus.cc:493] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.064648 14537 raft_consensus.cc:515] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.070192 14537 leader_election.cc:290] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e38530b0df45448e97a389793a29d29c (127.3.89.193:42879), 6033ddf10fd94a9b92e4777bfa02cc94 (127.3.89.195:37519)
I20260709 12:29:32.081739 14342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f200e87cfa24f5296f36b2de1c00922" candidate_uuid: "1d4801bb16b1436691a919cb98cead33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e38530b0df45448e97a389793a29d29c" is_pre_election: true
I20260709 12:29:32.082542 14342 raft_consensus.cc:2468] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d4801bb16b1436691a919cb98cead33 in term 0.
I20260709 12:29:32.083006 14492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f200e87cfa24f5296f36b2de1c00922" candidate_uuid: "1d4801bb16b1436691a919cb98cead33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" is_pre_election: true
I20260709 12:29:32.083704 14492 raft_consensus.cc:2468] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d4801bb16b1436691a919cb98cead33 in term 0.
I20260709 12:29:32.083619 14382 leader_election.cc:304] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d4801bb16b1436691a919cb98cead33, e38530b0df45448e97a389793a29d29c; no voters: 
I20260709 12:29:32.084748 14537 raft_consensus.cc:2804] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:32.085085 14537 raft_consensus.cc:493] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:32.085446 14537 raft_consensus.cc:3060] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.090987 14537 raft_consensus.cc:515] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.092746 14537 leader_election.cc:290] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [CANDIDATE]: Term 1 election: Requested vote from peers e38530b0df45448e97a389793a29d29c (127.3.89.193:42879), 6033ddf10fd94a9b92e4777bfa02cc94 (127.3.89.195:37519)
I20260709 12:29:32.093497 14342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f200e87cfa24f5296f36b2de1c00922" candidate_uuid: "1d4801bb16b1436691a919cb98cead33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e38530b0df45448e97a389793a29d29c"
I20260709 12:29:32.093829 14492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f200e87cfa24f5296f36b2de1c00922" candidate_uuid: "1d4801bb16b1436691a919cb98cead33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6033ddf10fd94a9b92e4777bfa02cc94"
I20260709 12:29:32.094117 14342 raft_consensus.cc:3060] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.094422 14492 raft_consensus.cc:3060] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.099526 14342 raft_consensus.cc:2468] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d4801bb16b1436691a919cb98cead33 in term 1.
I20260709 12:29:32.099689 14492 raft_consensus.cc:2468] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d4801bb16b1436691a919cb98cead33 in term 1.
I20260709 12:29:32.100725 14382 leader_election.cc:304] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d4801bb16b1436691a919cb98cead33, e38530b0df45448e97a389793a29d29c; no voters: 
I20260709 12:29:32.101428 14537 raft_consensus.cc:2804] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:32.102943 14537 raft_consensus.cc:697] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 1 LEADER]: Becoming Leader. State: Replica: 1d4801bb16b1436691a919cb98cead33, State: Running, Role: LEADER
I20260709 12:29:32.103839 14537 consensus_queue.cc:237] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.110913 14238 catalog_manager.cc:5697] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d4801bb16b1436691a919cb98cead33 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "1d4801bb16b1436691a919cb98cead33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:32.125790 14538 raft_consensus.cc:493] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.126366 14538 raft_consensus.cc:515] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.127976 14538 leader_election.cc:290] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d4801bb16b1436691a919cb98cead33 (127.3.89.194:41079), 6033ddf10fd94a9b92e4777bfa02cc94 (127.3.89.195:37519)
I20260709 12:29:32.128933 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca6bd4816684a7db4e6f30b21d12704" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d4801bb16b1436691a919cb98cead33" is_pre_election: true
I20260709 12:29:32.129280 14492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca6bd4816684a7db4e6f30b21d12704" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" is_pre_election: true
I20260709 12:29:32.129684 14418 raft_consensus.cc:2468] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 0.
I20260709 12:29:32.130002 14492 raft_consensus.cc:2468] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 0.
I20260709 12:29:32.130846 14305 leader_election.cc:304] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d4801bb16b1436691a919cb98cead33, e38530b0df45448e97a389793a29d29c; no voters: 
I20260709 12:29:32.131600 14538 raft_consensus.cc:2804] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:32.131923 14538 raft_consensus.cc:493] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:32.132201 14538 raft_consensus.cc:3060] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.137773 14538 raft_consensus.cc:515] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.139459 14538 leader_election.cc:290] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 1 election: Requested vote from peers 1d4801bb16b1436691a919cb98cead33 (127.3.89.194:41079), 6033ddf10fd94a9b92e4777bfa02cc94 (127.3.89.195:37519)
I20260709 12:29:32.140363 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca6bd4816684a7db4e6f30b21d12704" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d4801bb16b1436691a919cb98cead33"
I20260709 12:29:32.140594 14492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ca6bd4816684a7db4e6f30b21d12704" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6033ddf10fd94a9b92e4777bfa02cc94"
I20260709 12:29:32.140938 14418 raft_consensus.cc:3060] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.141229 14492 raft_consensus.cc:3060] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:32.146436 14492 raft_consensus.cc:2468] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 1.
I20260709 12:29:32.146698 14418 raft_consensus.cc:2468] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 1.
I20260709 12:29:32.147559 14305 leader_election.cc:304] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6033ddf10fd94a9b92e4777bfa02cc94, e38530b0df45448e97a389793a29d29c; no voters: 
I20260709 12:29:32.148341 14538 raft_consensus.cc:2804] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:32.148808 14538 raft_consensus.cc:697] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [term 1 LEADER]: Becoming Leader. State: Replica: e38530b0df45448e97a389793a29d29c, State: Running, Role: LEADER
I20260709 12:29:32.149534 14538 consensus_queue.cc:237] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.157629 14238 catalog_manager.cc:5697] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c reported cstate change: term changed from 0 to 1, leader changed from <none> to e38530b0df45448e97a389793a29d29c (127.3.89.193). New cstate: current_term: 1 leader_uuid: "e38530b0df45448e97a389793a29d29c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:32.220191 14342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87edaccfa2ce49c0bfe6590bcff29293"
dest_uuid: "e38530b0df45448e97a389793a29d29c"
mode: GRACEFUL
new_leader_uuid: "1d4801bb16b1436691a919cb98cead33"
 from {username='slave'} at 127.0.0.1:34470
I20260709 12:29:32.220721 14342 raft_consensus.cc:606] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 1 LEADER]: Received request to transfer leadership to TS 1d4801bb16b1436691a919cb98cead33
I20260709 12:29:32.238401 14418 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f200e87cfa24f5296f36b2de1c00922"
dest_uuid: "1d4801bb16b1436691a919cb98cead33"
mode: GRACEFUL
new_leader_uuid: "e38530b0df45448e97a389793a29d29c"
 from {username='slave'} at 127.0.0.1:60014
I20260709 12:29:32.238905 14418 raft_consensus.cc:606] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 1 LEADER]: Received request to transfer leadership to TS e38530b0df45448e97a389793a29d29c
I20260709 12:29:32.284531 14538 consensus_queue.cc:1048] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.291818 14547 raft_consensus.cc:993] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c: : Instructing follower 1d4801bb16b1436691a919cb98cead33 to start an election
I20260709 12:29:32.292692 14538 raft_consensus.cc:1081] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 1 LEADER]: Signalling peer 1d4801bb16b1436691a919cb98cead33 to start an election
I20260709 12:29:32.295405 14417 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87edaccfa2ce49c0bfe6590bcff29293"
dest_uuid: "1d4801bb16b1436691a919cb98cead33"
 from {username='slave'} at 127.0.0.1:59996
I20260709 12:29:32.296068 14417 raft_consensus.cc:493] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:32.296459 14417 raft_consensus.cc:3060] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.301455 14538 consensus_queue.cc:1048] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:32.305737 14538 consensus_queue.cc:1048] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.313021 14417 raft_consensus.cc:515] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.314225 14418 raft_consensus.cc:1240] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 2 FOLLOWER]: Rejecting Update request from peer e38530b0df45448e97a389793a29d29c for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:32.319458 14417 leader_election.cc:290] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [CANDIDATE]: Term 2 election: Requested vote from peers e38530b0df45448e97a389793a29d29c (127.3.89.193:42879), 6033ddf10fd94a9b92e4777bfa02cc94 (127.3.89.195:37519)
I20260709 12:29:32.319312 14538 consensus_queue.cc:1059] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.320317 14490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87edaccfa2ce49c0bfe6590bcff29293" candidate_uuid: "1d4801bb16b1436691a919cb98cead33" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6033ddf10fd94a9b92e4777bfa02cc94"
I20260709 12:29:32.321062 14547 raft_consensus.cc:3055] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:32.321084 14342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87edaccfa2ce49c0bfe6590bcff29293" candidate_uuid: "1d4801bb16b1436691a919cb98cead33" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e38530b0df45448e97a389793a29d29c"
I20260709 12:29:32.321739 14547 raft_consensus.cc:740] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e38530b0df45448e97a389793a29d29c, State: Running, Role: LEADER
I20260709 12:29:32.322770 14547 consensus_queue.cc:260] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.323998 14547 raft_consensus.cc:3060] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.345065 14538 consensus_queue.cc:1048] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.354040 14342 raft_consensus.cc:2468] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d4801bb16b1436691a919cb98cead33 in term 2.
I20260709 12:29:32.361238 14382 leader_election.cc:304] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1d4801bb16b1436691a919cb98cead33, e38530b0df45448e97a389793a29d29c; no voters: 6033ddf10fd94a9b92e4777bfa02cc94
I20260709 12:29:32.362670 14537 raft_consensus.cc:2804] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:32.363224 14537 raft_consensus.cc:697] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 2 LEADER]: Becoming Leader. State: Replica: 1d4801bb16b1436691a919cb98cead33, State: Running, Role: LEADER
I20260709 12:29:32.364049 14537 consensus_queue.cc:237] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.385558 14240 catalog_manager.cc:5697] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 reported cstate change: term changed from 1 to 2, leader changed from e38530b0df45448e97a389793a29d29c (127.3.89.193) to 1d4801bb16b1436691a919cb98cead33 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "1d4801bb16b1436691a919cb98cead33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:32.550230 14568 consensus_queue.cc:1048] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.563048 14547 consensus_queue.cc:1048] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.595027 14537 consensus_queue.cc:1048] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.605770 14537 consensus_queue.cc:1048] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:32.613083 14563 raft_consensus.cc:993] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33: : Instructing follower e38530b0df45448e97a389793a29d29c to start an election
I20260709 12:29:32.613692 14563 raft_consensus.cc:1081] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 1 LEADER]: Signalling peer e38530b0df45448e97a389793a29d29c to start an election
I20260709 12:29:32.615291 14342 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f200e87cfa24f5296f36b2de1c00922"
dest_uuid: "e38530b0df45448e97a389793a29d29c"
 from {username='slave'} at 127.0.0.1:34458
I20260709 12:29:32.615942 14342 raft_consensus.cc:493] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:32.616376 14342 raft_consensus.cc:3060] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.623315 14342 raft_consensus.cc:515] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.625299 14342 leader_election.cc:290] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 2 election: Requested vote from peers 1d4801bb16b1436691a919cb98cead33 (127.3.89.194:41079), 6033ddf10fd94a9b92e4777bfa02cc94 (127.3.89.195:37519)
I20260709 12:29:32.626050 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f200e87cfa24f5296f36b2de1c00922" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1d4801bb16b1436691a919cb98cead33"
I20260709 12:29:32.626338 14491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f200e87cfa24f5296f36b2de1c00922" candidate_uuid: "e38530b0df45448e97a389793a29d29c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6033ddf10fd94a9b92e4777bfa02cc94"
I20260709 12:29:32.626684 14417 raft_consensus.cc:3055] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:32.627056 14417 raft_consensus.cc:740] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1d4801bb16b1436691a919cb98cead33, State: Running, Role: LEADER
I20260709 12:29:32.627096 14491 raft_consensus.cc:3060] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.627890 14417 consensus_queue.cc:260] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.628798 14417 raft_consensus.cc:3060] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.633976 14491 raft_consensus.cc:2468] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 2.
I20260709 12:29:32.634225 14417 raft_consensus.cc:2468] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38530b0df45448e97a389793a29d29c in term 2.
I20260709 12:29:32.635121 14305 leader_election.cc:304] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6033ddf10fd94a9b92e4777bfa02cc94, e38530b0df45448e97a389793a29d29c; no voters: 
I20260709 12:29:32.635828 14547 raft_consensus.cc:2804] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:32.636241 14547 raft_consensus.cc:697] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 2 LEADER]: Becoming Leader. State: Replica: e38530b0df45448e97a389793a29d29c, State: Running, Role: LEADER
I20260709 12:29:32.637063 14547 consensus_queue.cc:237] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:32.643388 14237 catalog_manager.cc:5697] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c reported cstate change: term changed from 1 to 2, leader changed from 1d4801bb16b1436691a919cb98cead33 (127.3.89.194) to e38530b0df45448e97a389793a29d29c (127.3.89.193). New cstate: current_term: 2 leader_uuid: "e38530b0df45448e97a389793a29d29c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:32.744076  3431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:29:32.745790  3431 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:29:32.746194  3431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:29:32.760553 14342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f200e87cfa24f5296f36b2de1c00922"
dest_uuid: "e38530b0df45448e97a389793a29d29c"
mode: GRACEFUL
new_leader_uuid: "6033ddf10fd94a9b92e4777bfa02cc94"
 from {username='slave'} at 127.0.0.1:46938
I20260709 12:29:32.761262 14342 raft_consensus.cc:606] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 2 LEADER]: Received request to transfer leadership to TS 6033ddf10fd94a9b92e4777bfa02cc94
I20260709 12:29:32.762902  3431 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:29:32.802361 14342 raft_consensus.cc:1275] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 2 FOLLOWER]: Refusing update from remote peer 1d4801bb16b1436691a919cb98cead33: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:32.803835 14537 consensus_queue.cc:1048] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:32.816215 14491 raft_consensus.cc:3060] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:32.825255 14491 raft_consensus.cc:1275] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 2 FOLLOWER]: Refusing update from remote peer 1d4801bb16b1436691a919cb98cead33: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:32.826799 14563 consensus_queue.cc:1048] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:33.039986 14491 raft_consensus.cc:1275] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 2 FOLLOWER]: Refusing update from remote peer e38530b0df45448e97a389793a29d29c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.041385 14538 consensus_queue.cc:1048] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:33.047245 14573 raft_consensus.cc:993] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c: : Instructing follower 6033ddf10fd94a9b92e4777bfa02cc94 to start an election
I20260709 12:29:33.047816 14547 raft_consensus.cc:1081] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 2 LEADER]: Signalling peer 6033ddf10fd94a9b92e4777bfa02cc94 to start an election
I20260709 12:29:33.049885 14492 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f200e87cfa24f5296f36b2de1c00922"
dest_uuid: "6033ddf10fd94a9b92e4777bfa02cc94"
 from {username='slave'} at 127.0.0.1:34026
I20260709 12:29:33.051476 14492 raft_consensus.cc:493] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:33.052052 14492 raft_consensus.cc:3060] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:33.054788 14417 raft_consensus.cc:1275] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 2 FOLLOWER]: Refusing update from remote peer e38530b0df45448e97a389793a29d29c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:33.057475 14547 consensus_queue.cc:1048] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:33.059506 14492 raft_consensus.cc:515] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:33.067802 14342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f200e87cfa24f5296f36b2de1c00922" candidate_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e38530b0df45448e97a389793a29d29c"
I20260709 12:29:33.068476 14342 raft_consensus.cc:3055] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:33.068789 14342 raft_consensus.cc:740] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e38530b0df45448e97a389793a29d29c, State: Running, Role: LEADER
I20260709 12:29:33.069792 14342 consensus_queue.cc:260] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:33.070050 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f200e87cfa24f5296f36b2de1c00922" candidate_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1d4801bb16b1436691a919cb98cead33"
I20260709 12:29:33.070740 14417 raft_consensus.cc:3060] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:33.070801 14342 raft_consensus.cc:3060] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:33.066795 14492 leader_election.cc:290] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [CANDIDATE]: Term 3 election: Requested vote from peers e38530b0df45448e97a389793a29d29c (127.3.89.193:42879), 1d4801bb16b1436691a919cb98cead33 (127.3.89.194:41079)
I20260709 12:29:33.076666 14342 raft_consensus.cc:2468] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6033ddf10fd94a9b92e4777bfa02cc94 in term 3.
I20260709 12:29:33.077824 14457 leader_election.cc:304] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6033ddf10fd94a9b92e4777bfa02cc94, e38530b0df45448e97a389793a29d29c; no voters: 
I20260709 12:29:33.078716 14417 raft_consensus.cc:2468] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6033ddf10fd94a9b92e4777bfa02cc94 in term 3.
I20260709 12:29:33.078756 14541 raft_consensus.cc:2804] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:33.080422 14541 raft_consensus.cc:697] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 3 LEADER]: Becoming Leader. State: Replica: 6033ddf10fd94a9b92e4777bfa02cc94, State: Running, Role: LEADER
I20260709 12:29:33.081284 14541 consensus_queue.cc:237] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } }
I20260709 12:29:33.087283 14240 catalog_manager.cc:5697] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 reported cstate change: term changed from 2 to 3, leader changed from e38530b0df45448e97a389793a29d29c (127.3.89.193) to 6033ddf10fd94a9b92e4777bfa02cc94 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e38530b0df45448e97a389793a29d29c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d4801bb16b1436691a919cb98cead33" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 41079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6033ddf10fd94a9b92e4777bfa02cc94" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 37519 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:33.131117  3431 tablet_server.cc:179] TabletServer@127.3.89.193:0 shutting down...
I20260709 12:29:33.159965  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:33.160725  3431 tablet_replica.cc:333] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c: stopping tablet replica
I20260709 12:29:33.161303  3431 raft_consensus.cc:2243] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:33.162217  3431 raft_consensus.cc:2272] T 0ca6bd4816684a7db4e6f30b21d12704 P e38530b0df45448e97a389793a29d29c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:33.164465  3431 tablet_replica.cc:333] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c: stopping tablet replica
I20260709 12:29:33.164954  3431 raft_consensus.cc:2243] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:33.165347  3431 raft_consensus.cc:2272] T 2f200e87cfa24f5296f36b2de1c00922 P e38530b0df45448e97a389793a29d29c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:33.167758  3431 tablet_replica.cc:333] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c: stopping tablet replica
I20260709 12:29:33.168249  3431 raft_consensus.cc:2243] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:33.168648  3431 raft_consensus.cc:2272] T 87edaccfa2ce49c0bfe6590bcff29293 P e38530b0df45448e97a389793a29d29c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:33.170802  3431 tablet_replica.cc:333] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c: stopping tablet replica
I20260709 12:29:33.171278  3431 raft_consensus.cc:2243] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:33.171993  3431 raft_consensus.cc:2272] T c9deb6b3cccc4caab4337a16eedffcfa P e38530b0df45448e97a389793a29d29c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:33.195982  3431 tablet_server.cc:196] TabletServer@127.3.89.193:0 shutdown complete.
I20260709 12:29:33.208971  3431 tablet_server.cc:179] TabletServer@127.3.89.194:0 shutting down...
I20260709 12:29:33.229215  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:33.229933  3431 tablet_replica.cc:333] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33: stopping tablet replica
I20260709 12:29:33.230532  3431 raft_consensus.cc:2243] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:33.231026  3431 raft_consensus.cc:2272] T 2f200e87cfa24f5296f36b2de1c00922 P 1d4801bb16b1436691a919cb98cead33 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:33.233752  3431 tablet_replica.cc:333] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33: stopping tablet replica
I20260709 12:29:33.234376  3431 raft_consensus.cc:2243] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:33.234876  3431 raft_consensus.cc:2272] T 0ca6bd4816684a7db4e6f30b21d12704 P 1d4801bb16b1436691a919cb98cead33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:33.237453  3431 tablet_replica.cc:333] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33: stopping tablet replica
I20260709 12:29:33.238198  3431 raft_consensus.cc:2243] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:33.239063  3431 raft_consensus.cc:2272] T 87edaccfa2ce49c0bfe6590bcff29293 P 1d4801bb16b1436691a919cb98cead33 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:33.241677  3431 tablet_replica.cc:333] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33: stopping tablet replica
I20260709 12:29:33.242264  3431 raft_consensus.cc:2243] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:33.242722  3431 raft_consensus.cc:2272] T c9deb6b3cccc4caab4337a16eedffcfa P 1d4801bb16b1436691a919cb98cead33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:33.269697  3431 tablet_server.cc:196] TabletServer@127.3.89.194:0 shutdown complete.
I20260709 12:29:33.283150  3431 tablet_server.cc:179] TabletServer@127.3.89.195:0 shutting down...
I20260709 12:29:33.302659  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:33.303373  3431 tablet_replica.cc:333] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94: stopping tablet replica
I20260709 12:29:33.304338  3431 raft_consensus.cc:2243] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:33.305012  3431 pending_rounds.cc:70] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94: Trying to abort 1 pending ops.
I20260709 12:29:33.305233  3431 pending_rounds.cc:77] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305626308948197376 op_type: NO_OP noop_request { }
I20260709 12:29:33.305540  3431 raft_consensus.cc:2889] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:33.305905  3431 raft_consensus.cc:2272] T 2f200e87cfa24f5296f36b2de1c00922 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:33.308490  3431 tablet_replica.cc:333] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94: stopping tablet replica
I20260709 12:29:33.309031  3431 raft_consensus.cc:2243] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:33.309509  3431 raft_consensus.cc:2272] T 0ca6bd4816684a7db4e6f30b21d12704 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:33.311794  3431 tablet_replica.cc:333] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94: stopping tablet replica
I20260709 12:29:33.312445  3431 raft_consensus.cc:2243] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:33.312893  3431 raft_consensus.cc:2272] T 87edaccfa2ce49c0bfe6590bcff29293 P 6033ddf10fd94a9b92e4777bfa02cc94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:33.315408  3431 tablet_replica.cc:333] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94: stopping tablet replica
I20260709 12:29:33.315912  3431 raft_consensus.cc:2243] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:33.316334  3431 raft_consensus.cc:2272] T c9deb6b3cccc4caab4337a16eedffcfa P 6033ddf10fd94a9b92e4777bfa02cc94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:33.330071  3431 tablet_server.cc:196] TabletServer@127.3.89.195:0 shutdown complete.
I20260709 12:29:33.342878  3431 master.cc:562] Master@127.3.89.254:36899 shutting down...
I20260709 12:29:33.360976  3431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:33.361492  3431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:33.361867  3431 tablet_replica.cc:333] T 00000000000000000000000000000000 P c30fdc1bf8e849d38e95cd4130b1aa51: stopping tablet replica
I20260709 12:29:33.381014  3431 master.cc:584] Master@127.3.89.254:36899 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2385 ms)
[----------] 6 tests from LeaderRebalancerTest (378330 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:29:33.411211  3431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.3.89.254:39867
I20260709 12:29:33.412338  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:33.418031 14584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:33.418510 14585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:33.419479 14587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:33.420979  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:33.421870  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:33.422057  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:33.422174  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600173422163 us; error 0 us; skew 500 ppm
I20260709 12:29:33.422652  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:33.425204  3431 webserver.cc:533] Webserver started at http://127.3.89.254:44701/ using document root <none> and password file <none>
I20260709 12:29:33.425722  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:33.425894  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:33.426115  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:33.427181  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/master-0-root/instance:
uuid: "2f9192d6a0f04a41bcd2e2831e205822"
format_stamp: "Formatted at 2026-07-09 12:29:33 on dist-test-slave-3hrd"
I20260709 12:29:33.431814  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:29:33.435736 14592 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:33.436663  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:29:33.437021  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/master-0-root
uuid: "2f9192d6a0f04a41bcd2e2831e205822"
format_stamp: "Formatted at 2026-07-09 12:29:33 on dist-test-slave-3hrd"
I20260709 12:29:33.437343  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:33.447607  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:33.448642  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:33.489895  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:39867
I20260709 12:29:33.490020 14643 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.254:39867 every 8 connection(s)
I20260709 12:29:33.494244 14644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:33.508325 14644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822: Bootstrap starting.
I20260709 12:29:33.513067 14644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:33.517678 14644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822: No bootstrap required, opened a new log
I20260709 12:29:33.520105 14644 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9192d6a0f04a41bcd2e2831e205822" member_type: VOTER }
I20260709 12:29:33.520532 14644 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:33.520732 14644 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9192d6a0f04a41bcd2e2831e205822, State: Initialized, Role: FOLLOWER
I20260709 12:29:33.521278 14644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9192d6a0f04a41bcd2e2831e205822" member_type: VOTER }
I20260709 12:29:33.521852 14644 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:33.522070 14644 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:33.522306 14644 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:33.527575 14644 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9192d6a0f04a41bcd2e2831e205822" member_type: VOTER }
I20260709 12:29:33.528105 14644 leader_election.cc:304] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f9192d6a0f04a41bcd2e2831e205822; no voters: 
I20260709 12:29:33.529279 14644 leader_election.cc:290] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:33.529675 14647 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:33.531145 14647 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9192d6a0f04a41bcd2e2831e205822, State: Running, Role: LEADER
I20260709 12:29:33.531972 14647 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9192d6a0f04a41bcd2e2831e205822" member_type: VOTER }
I20260709 12:29:33.532662 14644 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:33.535044 14648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f9192d6a0f04a41bcd2e2831e205822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9192d6a0f04a41bcd2e2831e205822" member_type: VOTER } }
I20260709 12:29:33.535101 14649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f9192d6a0f04a41bcd2e2831e205822. Latest consensus state: current_term: 1 leader_uuid: "2f9192d6a0f04a41bcd2e2831e205822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9192d6a0f04a41bcd2e2831e205822" member_type: VOTER } }
I20260709 12:29:33.535777 14648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:33.535832 14649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:33.538873 14653 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:33.543581 14653 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:33.549890  3431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:33.553378 14653 catalog_manager.cc:1383] Generated new cluster ID: c1a250b19a4f4e21adece3b15761bb7d
I20260709 12:29:33.553742 14653 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:33.576951 14653 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:33.578444 14653 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:33.602442 14653 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822: Generated new TSK 0
I20260709 12:29:33.603138 14653 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:33.616812  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:33.623769 14665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:33.624881 14666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:33.628252 14668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:33.628307  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:33.629544  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:33.629879  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:33.630074  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600173630056 us; error 0 us; skew 500 ppm
I20260709 12:29:33.630690  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:33.633330  3431 webserver.cc:533] Webserver started at http://127.3.89.193:34431/ using document root <none> and password file <none>
I20260709 12:29:33.633914  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:33.634151  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:33.634505  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:33.635623  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-0-root/instance:
uuid: "21c8774363c141f7868e4fc3ec5854dd"
format_stamp: "Formatted at 2026-07-09 12:29:33 on dist-test-slave-3hrd"
I20260709 12:29:33.640825  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 12:29:33.644965 14673 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:33.646319  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:29:33.646690  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-0-root
uuid: "21c8774363c141f7868e4fc3ec5854dd"
format_stamp: "Formatted at 2026-07-09 12:29:33 on dist-test-slave-3hrd"
I20260709 12:29:33.646986  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:33.661762  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:33.662886  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:33.664319  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:33.667104  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:33.667313  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:33.667629  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:33.667786  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:33.713214  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:43267
I20260709 12:29:33.713304 14735 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.193:43267 every 8 connection(s)
I20260709 12:29:33.719331  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:33.727061 14740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:33.727787 14741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:33.730875  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:33.732288 14736 heartbeater.cc:344] Connected to a master server at 127.3.89.254:39867
W20260709 12:29:33.732302 14744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:33.732776 14736 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:33.733697  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:29:33.733754 14736 heartbeater.cc:507] Master 127.3.89.254:39867 requested a full tablet report, sending...
W20260709 12:29:33.733952  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:33.734417  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600173734391 us; error 0 us; skew 500 ppm
I20260709 12:29:33.735234  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:33.736318 14609 ts_manager.cc:194] Registered new tserver with Master: 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267)
I20260709 12:29:33.738125  3431 webserver.cc:533] Webserver started at http://127.3.89.194:39905/ using document root <none> and password file <none>
I20260709 12:29:33.738683  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:33.738981  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:33.738796 14609 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40422
I20260709 12:29:33.739368  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:33.740633  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-1-root/instance:
uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
format_stamp: "Formatted at 2026-07-09 12:29:33 on dist-test-slave-3hrd"
I20260709 12:29:33.745375  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 12:29:33.750074 14748 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:33.751572  3431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:29:33.752058  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-1-root
uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
format_stamp: "Formatted at 2026-07-09 12:29:33 on dist-test-slave-3hrd"
I20260709 12:29:33.752434  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:33.775635  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:33.776938  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:33.778518  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:33.780915  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:33.781169  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:33.781437  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:33.781649  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:29:33.830664  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:35477
I20260709 12:29:33.830744 14810 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.194:35477 every 8 connection(s)
I20260709 12:29:33.836762  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:33.846791 14814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:33.846480 14815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:33.852943 14811 heartbeater.cc:344] Connected to a master server at 127.3.89.254:39867
I20260709 12:29:33.853682 14811 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:33.854581 14811 heartbeater.cc:507] Master 127.3.89.254:39867 requested a full tablet report, sending...
W20260709 12:29:33.855623 14817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:33.856642  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:33.857709 14609 ts_manager.cc:194] Registered new tserver with Master: 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:33.857903  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:33.858266  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:33.858464  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600173858447 us; error 0 us; skew 500 ppm
I20260709 12:29:33.859129  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:33.859403 14609 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40428
I20260709 12:29:33.862190  3431 webserver.cc:533] Webserver started at http://127.3.89.195:46239/ using document root <none> and password file <none>
I20260709 12:29:33.862797  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:33.863023  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:33.863294  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:33.864420  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-2-root/instance:
uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
format_stamp: "Formatted at 2026-07-09 12:29:33 on dist-test-slave-3hrd"
I20260709 12:29:33.870411  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.003s
I20260709 12:29:33.874161 14822 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:33.875190  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:29:33.875499  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-2-root
uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
format_stamp: "Formatted at 2026-07-09 12:29:33 on dist-test-slave-3hrd"
I20260709 12:29:33.875775  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599794910850-3431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:33.900391  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:33.901469  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:33.903070  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:33.905516  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:33.905777  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:33.906032  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:33.906229  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:33.949926  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:35339
I20260709 12:29:33.950111 14884 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.195:35339 every 8 connection(s)
I20260709 12:29:33.966454 14885 heartbeater.cc:344] Connected to a master server at 127.3.89.254:39867
I20260709 12:29:33.966953 14885 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:33.967700 14885 heartbeater.cc:507] Master 127.3.89.254:39867 requested a full tablet report, sending...
I20260709 12:29:33.970090 14609 ts_manager.cc:194] Registered new tserver with Master: 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:33.970458  3431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016199056s
I20260709 12:29:33.971647 14609 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40436
I20260709 12:29:33.998498 14609 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40442:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:34.001128 14609 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:34.073937 14701 tablet_service.cc:1511] Processing CreateTablet for tablet d168e6a345a94901a3dac027a767123e (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:34.075603 14701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d168e6a345a94901a3dac027a767123e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.079526 14700 tablet_service.cc:1511] Processing CreateTablet for tablet 3928359ec0c2401e98c97cb66daa7126 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:34.081173 14700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3928359ec0c2401e98c97cb66daa7126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.090422 14699 tablet_service.cc:1511] Processing CreateTablet for tablet 170e6f1f2be94ad1afa5b0ab4c556d87 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:34.092120 14699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 170e6f1f2be94ad1afa5b0ab4c556d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.095503 14776 tablet_service.cc:1511] Processing CreateTablet for tablet d168e6a345a94901a3dac027a767123e (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:34.097075 14776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d168e6a345a94901a3dac027a767123e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.098863 14775 tablet_service.cc:1511] Processing CreateTablet for tablet 3928359ec0c2401e98c97cb66daa7126 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:34.099490 14849 tablet_service.cc:1511] Processing CreateTablet for tablet 3928359ec0c2401e98c97cb66daa7126 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:34.100512 14775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3928359ec0c2401e98c97cb66daa7126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.101048 14849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3928359ec0c2401e98c97cb66daa7126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.102530 14772 tablet_service.cc:1511] Processing CreateTablet for tablet 4350e9f3eba145d89b0f53790e46cc50 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:34.102751 14774 tablet_service.cc:1511] Processing CreateTablet for tablet 170e6f1f2be94ad1afa5b0ab4c556d87 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:34.102751 14848 tablet_service.cc:1511] Processing CreateTablet for tablet 170e6f1f2be94ad1afa5b0ab4c556d87 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:34.105146 14773 tablet_service.cc:1511] Processing CreateTablet for tablet af2477a6255446e89e4ee4848bea759b (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:34.105687 14848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 170e6f1f2be94ad1afa5b0ab4c556d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.107936 14850 tablet_service.cc:1511] Processing CreateTablet for tablet d168e6a345a94901a3dac027a767123e (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:34.108624 14771 tablet_service.cc:1511] Processing CreateTablet for tablet 2859ea67269e481f83cebadbe397b8a2 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:34.109272 14850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d168e6a345a94901a3dac027a767123e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.109920 14772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4350e9f3eba145d89b0f53790e46cc50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.113971 14774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 170e6f1f2be94ad1afa5b0ab4c556d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.118412 14773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af2477a6255446e89e4ee4848bea759b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.119578 14697 tablet_service.cc:1511] Processing CreateTablet for tablet 4350e9f3eba145d89b0f53790e46cc50 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:34.120774 14846 tablet_service.cc:1511] Processing CreateTablet for tablet 4350e9f3eba145d89b0f53790e46cc50 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:34.119578 14847 tablet_service.cc:1511] Processing CreateTablet for tablet af2477a6255446e89e4ee4848bea759b (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:34.121776 14698 tablet_service.cc:1511] Processing CreateTablet for tablet af2477a6255446e89e4ee4848bea759b (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:34.122424 14696 tablet_service.cc:1511] Processing CreateTablet for tablet 2859ea67269e481f83cebadbe397b8a2 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:34.123277 14847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af2477a6255446e89e4ee4848bea759b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.124215 14697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4350e9f3eba145d89b0f53790e46cc50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.127089 14698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af2477a6255446e89e4ee4848bea759b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.129070 14846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4350e9f3eba145d89b0f53790e46cc50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.130002 14771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2859ea67269e481f83cebadbe397b8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.134971 14696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2859ea67269e481f83cebadbe397b8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.160377 14845 tablet_service.cc:1511] Processing CreateTablet for tablet 2859ea67269e481f83cebadbe397b8a2 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:34.162372 14845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2859ea67269e481f83cebadbe397b8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.162730 14699 tablet_service.cc:1511] Processing CreateTablet for tablet f63cdd55867645ababeebf041ec58a26 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:34.164533 14699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f63cdd55867645ababeebf041ec58a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.201689 14770 tablet_service.cc:1511] Processing CreateTablet for tablet f63cdd55867645ababeebf041ec58a26 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:34.203118 14770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f63cdd55867645ababeebf041ec58a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.222450 14900 tablet_bootstrap.cc:492] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:34.222584 14902 tablet_bootstrap.cc:492] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:34.235965 14844 tablet_service.cc:1511] Processing CreateTablet for tablet f63cdd55867645ababeebf041ec58a26 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:34.237442 14844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f63cdd55867645ababeebf041ec58a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.241395 14701 tablet_service.cc:1511] Processing CreateTablet for tablet b3b45722450c4ab48fae4b4faa88cfdd (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:34.243448 14701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b45722450c4ab48fae4b4faa88cfdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.244406 14775 tablet_service.cc:1511] Processing CreateTablet for tablet b3b45722450c4ab48fae4b4faa88cfdd (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:34.251631 14775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b45722450c4ab48fae4b4faa88cfdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.245390 14849 tablet_service.cc:1511] Processing CreateTablet for tablet b3b45722450c4ab48fae4b4faa88cfdd (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:34.241545 14902 tablet_bootstrap.cc:654] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.259074 14849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3b45722450c4ab48fae4b4faa88cfdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.259986 14697 tablet_service.cc:1511] Processing CreateTablet for tablet 544e89b953424226ae1c78c218552c43 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:34.261379 14697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544e89b953424226ae1c78c218552c43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.282362 14770 tablet_service.cc:1511] Processing CreateTablet for tablet 544e89b953424226ae1c78c218552c43 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:34.284375 14770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544e89b953424226ae1c78c218552c43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.263501 14900 tablet_bootstrap.cc:654] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.307650 14844 tablet_service.cc:1511] Processing CreateTablet for tablet 544e89b953424226ae1c78c218552c43 (DEFAULT_TABLE table=test-workload [id=7b246d2c2d024fc48ad04a0676469a27]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:34.308975 14844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544e89b953424226ae1c78c218552c43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:34.317965 14902 tablet_bootstrap.cc:492] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:34.318552 14902 ts_tablet_manager.cc:1403] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.096s	user 0.032s	sys 0.014s
I20260709 12:29:34.324270 14901 tablet_bootstrap.cc:492] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:34.327934 14900 tablet_bootstrap.cc:492] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:34.328491 14900 ts_tablet_manager.cc:1403] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.106s	user 0.012s	sys 0.015s
I20260709 12:29:34.328961 14902 raft_consensus.cc:359] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.329854 14902 raft_consensus.cc:385] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.330211 14902 raft_consensus.cc:740] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.330783 14901 tablet_bootstrap.cc:654] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.332801 14902 consensus_queue.cc:260] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.331455 14900 raft_consensus.cc:359] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.334707 14900 raft_consensus.cc:385] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.335050 14900 raft_consensus.cc:740] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.336787 14811 heartbeater.cc:499] Master 127.3.89.254:39867 was elected leader, sending a full tablet report...
I20260709 12:29:34.337374 14902 ts_tablet_manager.cc:1434] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:29:34.336925 14900 consensus_queue.cc:260] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.338421 14902 tablet_bootstrap.cc:492] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:34.344823 14902 tablet_bootstrap.cc:654] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.348806 14736 heartbeater.cc:499] Master 127.3.89.254:39867 was elected leader, sending a full tablet report...
I20260709 12:29:34.349536 14901 tablet_bootstrap.cc:492] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:34.350106 14901 ts_tablet_manager.cc:1403] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.003s
I20260709 12:29:34.354658 14900 ts_tablet_manager.cc:1434] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.005s
I20260709 12:29:34.355672 14900 tablet_bootstrap.cc:492] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:34.361733 14900 tablet_bootstrap.cc:654] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.362962 14901 raft_consensus.cc:359] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.363793 14901 raft_consensus.cc:385] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.364105 14901 raft_consensus.cc:740] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.364761 14902 tablet_bootstrap.cc:492] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:34.365303 14902 ts_tablet_manager.cc:1403] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.015s
I20260709 12:29:34.365065 14901 consensus_queue.cc:260] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.369082 14902 raft_consensus.cc:359] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.369974 14902 raft_consensus.cc:385] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.370265 14902 raft_consensus.cc:740] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.371065 14902 consensus_queue.cc:260] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.371591 14901 ts_tablet_manager.cc:1434] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.002s
I20260709 12:29:34.372335 14885 heartbeater.cc:499] Master 127.3.89.254:39867 was elected leader, sending a full tablet report...
I20260709 12:29:34.373663 14902 ts_tablet_manager.cc:1434] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:29:34.374737 14902 tablet_bootstrap.cc:492] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:34.377616 14901 tablet_bootstrap.cc:492] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:34.380962 14902 tablet_bootstrap.cc:654] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.384740 14901 tablet_bootstrap.cc:654] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.389228 14900 tablet_bootstrap.cc:492] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:34.389925 14900 ts_tablet_manager.cc:1403] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.034s	user 0.026s	sys 0.007s
I20260709 12:29:34.392273 14905 raft_consensus.cc:493] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.393534 14902 tablet_bootstrap.cc:492] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:34.393347 14905 raft_consensus.cc:515] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.394166 14902 ts_tablet_manager.cc:1403] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 12:29:34.395897 14901 tablet_bootstrap.cc:492] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:34.396422 14901 ts_tablet_manager.cc:1403] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.003s
I20260709 12:29:34.397331 14902 raft_consensus.cc:359] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.398159 14902 raft_consensus.cc:385] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.398440 14902 raft_consensus.cc:740] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.399250 14905 leader_election.cc:290] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:34.401484 14900 raft_consensus.cc:359] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.402319 14900 raft_consensus.cc:385] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.402640 14900 raft_consensus.cc:740] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.403421 14900 consensus_queue.cc:260] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.399349 14901 raft_consensus.cc:359] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.404750 14901 raft_consensus.cc:385] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.405081 14901 raft_consensus.cc:740] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.406006 14901 consensus_queue.cc:260] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.399339 14902 consensus_queue.cc:260] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.408221 14901 ts_tablet_manager.cc:1434] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:29:34.408643 14902 ts_tablet_manager.cc:1434] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:29:34.409236 14901 tablet_bootstrap.cc:492] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:34.409713 14902 tablet_bootstrap.cc:492] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:34.414935 14900 ts_tablet_manager.cc:1434] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.025s	user 0.002s	sys 0.004s
I20260709 12:29:34.416009 14900 tablet_bootstrap.cc:492] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:34.416373 14901 tablet_bootstrap.cc:654] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.416689 14910 raft_consensus.cc:493] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.417210 14910 raft_consensus.cc:515] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.420243 14910 leader_election.cc:290] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:34.427629 14902 tablet_bootstrap.cc:654] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.428239 14900 tablet_bootstrap.cc:654] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.451042 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:34.452678 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:34.454840 14901 tablet_bootstrap.cc:492] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:34.455416 14901 ts_tablet_manager.cc:1403] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.018s
W20260709 12:29:34.456336 14749 leader_election.cc:343] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:34.458492 14901 raft_consensus.cc:359] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.459281 14901 raft_consensus.cc:385] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.459599 14901 raft_consensus.cc:740] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
W20260709 12:29:34.460326 14750 leader_election.cc:343] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:34.460768 14902 tablet_bootstrap.cc:492] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:34.461030 14750 leader_election.cc:304] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23; no voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:34.461486 14902 ts_tablet_manager.cc:1403] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.052s	user 0.012s	sys 0.008s
I20260709 12:29:34.461901 14786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4350e9f3eba145d89b0f53790e46cc50" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:34.462653 14786 raft_consensus.cc:2468] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 0.
I20260709 12:29:34.462997 14905 raft_consensus.cc:2749] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:34.464267 14824 leader_election.cc:304] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:34.465162 14910 raft_consensus.cc:2804] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:34.465204 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4350e9f3eba145d89b0f53790e46cc50" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:34.465624 14910 raft_consensus.cc:493] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.466145 14910 raft_consensus.cc:3060] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.465480 14901 consensus_queue.cc:260] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.465194 14902 raft_consensus.cc:359] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.467582 14902 raft_consensus.cc:385] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.467959 14902 raft_consensus.cc:740] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.468382 14900 tablet_bootstrap.cc:492] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:34.469969 14900 ts_tablet_manager.cc:1403] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.054s	user 0.010s	sys 0.008s
I20260709 12:29:34.470165 14902 consensus_queue.cc:260] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.474053 14900 raft_consensus.cc:359] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.475176 14902 ts_tablet_manager.cc:1434] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.003s
I20260709 12:29:34.475088 14900 raft_consensus.cc:385] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.475759 14900 raft_consensus.cc:740] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.476433 14902 tablet_bootstrap.cc:492] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:34.477020 14900 consensus_queue.cc:260] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.483487 14902 tablet_bootstrap.cc:654] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:34.485517 14824 leader_election.cc:343] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:34.486291 14900 ts_tablet_manager.cc:1434] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.004s
I20260709 12:29:34.487001 14910 raft_consensus.cc:515] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.490149 14901 ts_tablet_manager.cc:1434] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.034s	user 0.019s	sys 0.011s
I20260709 12:29:34.490723 14910 leader_election.cc:290] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:34.491313 14901 tablet_bootstrap.cc:492] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:34.492208 14900 tablet_bootstrap.cc:492] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:34.499066 14901 tablet_bootstrap.cc:654] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.500241 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4350e9f3eba145d89b0f53790e46cc50" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:34.501843 14786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4350e9f3eba145d89b0f53790e46cc50" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:34.502497 14786 raft_consensus.cc:3060] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.505055 14902 tablet_bootstrap.cc:492] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:34.505698 14902 ts_tablet_manager.cc:1403] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.030s	user 0.020s	sys 0.007s
I20260709 12:29:34.508879 14902 raft_consensus.cc:359] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.509730 14902 raft_consensus.cc:385] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.510087 14902 raft_consensus.cc:740] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.511029 14902 consensus_queue.cc:260] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.511628 14900 tablet_bootstrap.cc:654] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.513823 14902 ts_tablet_manager.cc:1434] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:34.514950 14902 tablet_bootstrap.cc:492] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:34.520459 14905 raft_consensus.cc:493] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.521620 14902 tablet_bootstrap.cc:654] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.521272 14905 raft_consensus.cc:515] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
W20260709 12:29:34.509225 14824 leader_election.cc:343] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:34.525787 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3928359ec0c2401e98c97cb66daa7126" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:34.526809 14711 raft_consensus.cc:2468] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 0.
I20260709 12:29:34.528695 14750 leader_election.cc:304] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:34.531117 14905 leader_election.cc:290] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:34.531957 14905 raft_consensus.cc:2804] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:34.532501 14905 raft_consensus.cc:493] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.534240 14902 tablet_bootstrap.cc:492] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:34.534857 14902 ts_tablet_manager.cc:1403] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.020s	user 0.005s	sys 0.013s
I20260709 12:29:34.535305 14786 raft_consensus.cc:2468] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 1.
I20260709 12:29:34.535218 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3928359ec0c2401e98c97cb66daa7126" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:34.536310 14860 raft_consensus.cc:2468] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 0.
I20260709 12:29:34.532984 14905 raft_consensus.cc:3060] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.538609 14824 leader_election.cc:304] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 21c8774363c141f7868e4fc3ec5854dd
I20260709 12:29:34.539574 14920 raft_consensus.cc:2804] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:34.540063 14920 raft_consensus.cc:697] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Leader. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:34.541002 14920 consensus_queue.cc:237] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.542155 14902 raft_consensus.cc:359] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.542889 14901 tablet_bootstrap.cc:492] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:34.543222 14902 raft_consensus.cc:385] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.543543 14901 ts_tablet_manager.cc:1403] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.052s	user 0.035s	sys 0.016s
W20260709 12:29:34.543640 14657 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:34.543643 14902 raft_consensus.cc:740] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.544986 14902 consensus_queue.cc:260] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.547518 14901 raft_consensus.cc:359] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.547911 14902 ts_tablet_manager.cc:1434] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 12:29:34.548343 14901 raft_consensus.cc:385] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.548700 14901 raft_consensus.cc:740] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.549751 14901 consensus_queue.cc:260] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.552338 14901 ts_tablet_manager.cc:1434] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:34.552459 14902 tablet_bootstrap.cc:492] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:34.553368 14901 tablet_bootstrap.cc:492] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:34.562429 14902 tablet_bootstrap.cc:654] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.563373 14608 catalog_manager.cc:5697] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:34.563369 14901 tablet_bootstrap.cc:654] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.574146 14920 raft_consensus.cc:493] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.575251 14920 raft_consensus.cc:515] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.581892 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "170e6f1f2be94ad1afa5b0ab4c556d87" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:34.582866 14711 raft_consensus.cc:2468] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 0.
I20260709 12:29:34.584863 14786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "170e6f1f2be94ad1afa5b0ab4c556d87" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:34.585850 14786 raft_consensus.cc:2468] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 0.
I20260709 12:29:34.587671 14824 leader_election.cc:304] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:34.600993 14910 raft_consensus.cc:2804] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:34.601869 14910 raft_consensus.cc:493] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.602371 14910 raft_consensus.cc:3060] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.584491 14920 leader_election.cc:290] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:34.595305 14905 raft_consensus.cc:515] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.608247 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3928359ec0c2401e98c97cb66daa7126" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:34.609083 14711 raft_consensus.cc:3060] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.615794 14900 tablet_bootstrap.cc:492] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:34.621932 14900 ts_tablet_manager.cc:1403] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.130s	user 0.019s	sys 0.006s
I20260709 12:29:34.623088 14910 raft_consensus.cc:515] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.626351 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3928359ec0c2401e98c97cb66daa7126" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:34.631047 14860 raft_consensus.cc:3060] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.631338 14901 tablet_bootstrap.cc:492] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:34.631498 14902 tablet_bootstrap.cc:492] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:34.631915 14901 ts_tablet_manager.cc:1403] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.079s	user 0.037s	sys 0.031s
I20260709 12:29:34.632062 14902 ts_tablet_manager.cc:1403] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.080s	user 0.026s	sys 0.045s
I20260709 12:29:34.633308 14710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "170e6f1f2be94ad1afa5b0ab4c556d87" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:34.634013 14710 raft_consensus.cc:3060] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.635305 14901 raft_consensus.cc:359] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.635491 14902 raft_consensus.cc:359] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.636101 14901 raft_consensus.cc:385] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.636387 14902 raft_consensus.cc:385] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.636448 14901 raft_consensus.cc:740] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.636808 14902 raft_consensus.cc:740] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.637548 14901 consensus_queue.cc:260] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.637751 14902 consensus_queue.cc:260] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.639873 14860 raft_consensus.cc:2468] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 1.
I20260709 12:29:34.626609 14900 raft_consensus.cc:359] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.640996 14900 raft_consensus.cc:385] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.641398 14749 leader_election.cc:304] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:34.641393 14900 raft_consensus.cc:740] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.627061 14905 leader_election.cc:290] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:34.628734 14711 raft_consensus.cc:2468] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 1.
I20260709 12:29:34.642838 14925 raft_consensus.cc:2804] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:34.644933 14905 raft_consensus.cc:493] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.642940 14900 consensus_queue.cc:260] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.645560 14905 raft_consensus.cc:515] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.640019 14901 ts_tablet_manager.cc:1434] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.003s
I20260709 12:29:34.648957 14901 tablet_bootstrap.cc:492] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:34.649091 14902 ts_tablet_manager.cc:1434] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 12:29:34.650082 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63cdd55867645ababeebf041ec58a26" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:34.646626 14710 raft_consensus.cc:2468] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 1.
W20260709 12:29:34.651525 14750 leader_election.cc:343] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:34.652704 14824 leader_election.cc:304] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:34.651511 14902 tablet_bootstrap.cc:492] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:34.658547 14925 raft_consensus.cc:697] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Becoming Leader. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:34.659423 14906 raft_consensus.cc:493] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.657661 14920 raft_consensus.cc:2804] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:34.653643 14900 ts_tablet_manager.cc:1434] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.002s
I20260709 12:29:34.662261 14786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "170e6f1f2be94ad1afa5b0ab4c556d87" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:34.664461 14786 raft_consensus.cc:3060] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.664829 14901 tablet_bootstrap.cc:654] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.664136 14906 raft_consensus.cc:515] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.670608 14920 raft_consensus.cc:697] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Leader. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:34.671200 14935 raft_consensus.cc:493] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.672154 14920 consensus_queue.cc:237] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.673580 14905 leader_election.cc:290] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:34.665310 14910 leader_election.cc:290] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:34.676299 14900 tablet_bootstrap.cc:492] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:34.685545 14925 consensus_queue.cc:237] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.686636 14932 raft_consensus.cc:493] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.688678 14932 raft_consensus.cc:515] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.679172 14906 leader_election.cc:290] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:34.694188 14932 leader_election.cc:290] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:34.678772 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63cdd55867645ababeebf041ec58a26" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:34.700254 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d168e6a345a94901a3dac027a767123e" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:34.695065 14710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d168e6a345a94901a3dac027a767123e" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
W20260709 12:29:34.702499 14749 leader_election.cc:343] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:34.702981 14710 raft_consensus.cc:2468] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 0.
I20260709 12:29:34.702752 14786 raft_consensus.cc:2468] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 1.
I20260709 12:29:34.707624 14750 leader_election.cc:304] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:34.699577 14860 raft_consensus.cc:2468] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 0.
I20260709 12:29:34.710327 14932 raft_consensus.cc:2804] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:34.714773 14749 leader_election.cc:304] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 21c8774363c141f7868e4fc3ec5854dd
I20260709 12:29:34.714846 14932 raft_consensus.cc:493] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.715517 14932 raft_consensus.cc:3060] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.715659 14905 raft_consensus.cc:2804] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:34.716125 14905 raft_consensus.cc:493] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.717021 14905 raft_consensus.cc:3060] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.687693 14935 raft_consensus.cc:515] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.721155 14710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:34.721887 14710 raft_consensus.cc:2468] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 0.
I20260709 12:29:34.723745 14824 leader_election.cc:304] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:34.724884 14786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:34.724762 14905 raft_consensus.cc:515] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.725601 14786 raft_consensus.cc:2468] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 0.
I20260709 12:29:34.727599 14710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63cdd55867645ababeebf041ec58a26" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
W20260709 12:29:34.729031 14750 leader_election.cc:343] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:34.721406 14935 leader_election.cc:290] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:34.734910 14905 leader_election.cc:290] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:34.739285 14786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d168e6a345a94901a3dac027a767123e" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:34.738804 14606 catalog_manager.cc:5697] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:34.741973 14900 tablet_bootstrap.cc:654] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.729772 14920 raft_consensus.cc:2804] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:34.744297 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63cdd55867645ababeebf041ec58a26" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:34.745049 14860 raft_consensus.cc:3060] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.733978 14932 raft_consensus.cc:515] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.750531 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d168e6a345a94901a3dac027a767123e" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:34.751741 14786 raft_consensus.cc:2393] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 21c8774363c141f7868e4fc3ec5854dd in current term 1: Already voted for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in this term.
I20260709 12:29:34.753021 14710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d168e6a345a94901a3dac027a767123e" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:34.753242 14920 raft_consensus.cc:493] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.753679 14710 raft_consensus.cc:3060] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:34.752903 14674 leader_election.cc:343] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:34.755119 14609 catalog_manager.cc:5697] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:34.756915 14675 leader_election.cc:304] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd; no voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:34.753876 14920 raft_consensus.cc:3060] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.759799 14860 raft_consensus.cc:2468] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 1.
I20260709 12:29:34.761009 14749 leader_election.cc:304] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 21c8774363c141f7868e4fc3ec5854dd
I20260709 12:29:34.762249 14905 raft_consensus.cc:2804] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:34.750337 14902 tablet_bootstrap.cc:654] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.762786 14905 raft_consensus.cc:697] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Becoming Leader. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:34.763717 14905 consensus_queue.cc:237] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.766388 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d168e6a345a94901a3dac027a767123e" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
W20260709 12:29:34.768091 14749 leader_election.cc:343] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:34.768637 14710 raft_consensus.cc:2468] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 1.
I20260709 12:29:34.768792 14932 leader_election.cc:290] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:34.769202 14906 raft_consensus.cc:2749] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:34.767084 14920 raft_consensus.cc:515] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.776024 14750 leader_election.cc:304] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:34.777338 14905 raft_consensus.cc:2804] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:34.780149 14905 raft_consensus.cc:697] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Becoming Leader. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:34.780491 14786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:34.781167 14786 raft_consensus.cc:3060] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.781208 14905 consensus_queue.cc:237] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.779362 14920 leader_election.cc:290] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:34.783869 14901 tablet_bootstrap.cc:492] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:34.784514 14901 ts_tablet_manager.cc:1403] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.136s	user 0.018s	sys 0.009s
I20260709 12:29:34.783955 14606 catalog_manager.cc:5697] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:34.786461 14710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:34.787104 14710 raft_consensus.cc:3060] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.787824 14901 raft_consensus.cc:359] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.788548 14901 raft_consensus.cc:385] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.788838 14901 raft_consensus.cc:740] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.789733 14901 consensus_queue.cc:260] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.792167 14901 ts_tablet_manager.cc:1434] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:34.793218 14901 tablet_bootstrap.cc:492] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:34.797883 14786 raft_consensus.cc:2468] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 1.
I20260709 12:29:34.799760 14901 tablet_bootstrap.cc:654] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.801244 14824 leader_election.cc:304] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:34.802244 14920 raft_consensus.cc:2804] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:34.802752 14920 raft_consensus.cc:697] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Leader. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:34.799846 14710 raft_consensus.cc:2468] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 1.
I20260709 12:29:34.803646 14920 consensus_queue.cc:237] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.814229 14901 tablet_bootstrap.cc:492] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:34.814997 14901 ts_tablet_manager.cc:1403] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.003s
I20260709 12:29:34.819543 14900 tablet_bootstrap.cc:492] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:34.819204 14901 raft_consensus.cc:359] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.820246 14901 raft_consensus.cc:385] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.820376 14900 ts_tablet_manager.cc:1403] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.144s	user 0.019s	sys 0.000s
I20260709 12:29:34.820732 14901 raft_consensus.cc:740] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.821905 14901 consensus_queue.cc:260] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.824981 14901 ts_tablet_manager.cc:1434] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 12:29:34.824445 14900 raft_consensus.cc:359] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.825690 14900 raft_consensus.cc:385] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.826128 14900 raft_consensus.cc:740] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.826469 14901 tablet_bootstrap.cc:492] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:34.827159 14900 consensus_queue.cc:260] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.830400 14902 tablet_bootstrap.cc:492] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:34.830884 14902 ts_tablet_manager.cc:1403] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.180s	user 0.016s	sys 0.003s
I20260709 12:29:34.833595 14902 raft_consensus.cc:359] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.834542 14902 raft_consensus.cc:385] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.834873 14902 raft_consensus.cc:740] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.835729 14902 consensus_queue.cc:260] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.837395 14901 tablet_bootstrap.cc:654] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.838299 14902 ts_tablet_manager.cc:1434] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:34.839408 14902 tablet_bootstrap.cc:492] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:34.844319 14900 ts_tablet_manager.cc:1434] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.023s	user 0.012s	sys 0.000s
I20260709 12:29:34.851867 14902 tablet_bootstrap.cc:654] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.854950 14901 tablet_bootstrap.cc:492] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:34.855528 14901 ts_tablet_manager.cc:1403] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.016s
I20260709 12:29:34.859041 14901 raft_consensus.cc:359] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.859989 14901 raft_consensus.cc:385] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.860329 14901 raft_consensus.cc:740] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.861465 14901 consensus_queue.cc:260] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.864603 14900 tablet_bootstrap.cc:492] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:34.864711 14901 ts_tablet_manager.cc:1434] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 12:29:34.865300 14902 tablet_bootstrap.cc:492] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:34.865849 14902 ts_tablet_manager.cc:1403] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.006s
I20260709 12:29:34.866068 14901 tablet_bootstrap.cc:492] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:34.861937 14609 catalog_manager.cc:5697] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:34.868769 14902 raft_consensus.cc:359] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.869696 14902 raft_consensus.cc:385] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.870042 14902 raft_consensus.cc:740] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.871003 14902 consensus_queue.cc:260] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.875559 14901 tablet_bootstrap.cc:654] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.876941 14606 catalog_manager.cc:5697] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:34.877988 14900 tablet_bootstrap.cc:654] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.881558 14902 ts_tablet_manager.cc:1434] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.009s
I20260709 12:29:34.887750 14901 tablet_bootstrap.cc:492] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:34.888154 14900 tablet_bootstrap.cc:492] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:34.888302 14901 ts_tablet_manager.cc:1403] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.003s
I20260709 12:29:34.888674 14900 ts_tablet_manager.cc:1403] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.006s
I20260709 12:29:34.891153 14901 raft_consensus.cc:359] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.891690 14900 raft_consensus.cc:359] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.892359 14901 raft_consensus.cc:385] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.892402 14900 raft_consensus.cc:385] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.892763 14901 raft_consensus.cc:740] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.892799 14900 raft_consensus.cc:740] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.893641 14901 consensus_queue.cc:260] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.893759 14900 consensus_queue.cc:260] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.895994 14901 ts_tablet_manager.cc:1434] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:29:34.896136 14900 ts_tablet_manager.cc:1434] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:34.898226 14900 tablet_bootstrap.cc:492] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:34.906534 14900 tablet_bootstrap.cc:654] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.910854 14900 tablet_bootstrap.cc:492] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:34.911304 14900 ts_tablet_manager.cc:1403] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:29:34.913470 14900 raft_consensus.cc:359] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.914139 14900 raft_consensus.cc:385] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.914371 14900 raft_consensus.cc:740] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.914968 14900 consensus_queue.cc:260] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.916554 14900 ts_tablet_manager.cc:1434] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:29:34.917445 14900 tablet_bootstrap.cc:492] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:34.923592 14900 tablet_bootstrap.cc:654] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.928339 14900 tablet_bootstrap.cc:492] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:34.928754 14900 ts_tablet_manager.cc:1403] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:29:34.930275 14905 raft_consensus.cc:493] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.930773 14905 raft_consensus.cc:515] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.930876 14900 raft_consensus.cc:359] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.931507 14900 raft_consensus.cc:385] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.931728 14900 raft_consensus.cc:740] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.932344 14900 consensus_queue.cc:260] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.932610 14905 leader_election.cc:290] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:34.933305 14710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2859ea67269e481f83cebadbe397b8a2" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:34.933766 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2859ea67269e481f83cebadbe397b8a2" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:34.934036 14710 raft_consensus.cc:2468] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 0.
I20260709 12:29:34.934463 14860 raft_consensus.cc:2468] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 0.
I20260709 12:29:34.935103 14900 ts_tablet_manager.cc:1434] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:29:34.935395 14750 leader_election.cc:304] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:34.936372 14900 tablet_bootstrap.cc:492] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:34.936548 14905 raft_consensus.cc:2804] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:34.937009 14905 raft_consensus.cc:493] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.937310 14905 raft_consensus.cc:3060] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.942835 14900 tablet_bootstrap.cc:654] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:34.944625 14905 raft_consensus.cc:515] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.945791 14920 consensus_queue.cc:1048] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:34.947925 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2859ea67269e481f83cebadbe397b8a2" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:34.948575 14860 raft_consensus.cc:3060] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.949400 14710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2859ea67269e481f83cebadbe397b8a2" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:34.950347 14710 raft_consensus.cc:3060] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.956422 14860 raft_consensus.cc:2468] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 1.
I20260709 12:29:34.958158 14749 leader_election.cc:304] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:34.959085 14905 leader_election.cc:290] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:34.959097 14932 raft_consensus.cc:2804] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:34.959762 14932 raft_consensus.cc:697] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Becoming Leader. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:34.960697 14932 consensus_queue.cc:237] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.962857 14900 tablet_bootstrap.cc:492] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:34.963464 14900 ts_tablet_manager.cc:1403] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.012s
I20260709 12:29:34.967346 14900 raft_consensus.cc:359] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.968483 14900 raft_consensus.cc:385] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:34.968875 14900 raft_consensus.cc:740] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:34.969892 14900 consensus_queue.cc:260] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.972891 14900 ts_tablet_manager.cc:1434] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 12:29:34.973340 14934 raft_consensus.cc:493] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.974013 14934 raft_consensus.cc:515] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.976090 14934 leader_election.cc:290] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:34.976675 14609 catalog_manager.cc:5697] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:34.978207 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b45722450c4ab48fae4b4faa88cfdd" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:34.979809 14711 raft_consensus.cc:2468] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 0.
I20260709 12:29:34.981768 14784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b45722450c4ab48fae4b4faa88cfdd" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:34.982690 14784 raft_consensus.cc:2468] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 0.
I20260709 12:29:34.984338 14824 leader_election.cc:304] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:34.986136 14934 raft_consensus.cc:2804] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:34.986573 14934 raft_consensus.cc:493] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:34.986913 14934 raft_consensus.cc:3060] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.985210 14710 raft_consensus.cc:2468] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 1.
I20260709 12:29:34.987221 14906 raft_consensus.cc:493] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:34.987821 14906 raft_consensus.cc:515] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:34.990252 14906 leader_election.cc:290] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:34.992071 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544e89b953424226ae1c78c218552c43" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:34.992861 14860 raft_consensus.cc:2468] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 0.
I20260709 12:29:34.993460 14710 raft_consensus.cc:3060] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:34.994256 14785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544e89b953424226ae1c78c218552c43" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:34.994942 14785 raft_consensus.cc:2468] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 0.
I20260709 12:29:34.995126 14674 leader_election.cc:304] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:34.998903 14906 raft_consensus.cc:2804] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:34.999357 14906 raft_consensus.cc:493] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:35.000946 14906 raft_consensus.cc:3060] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:35.004055 14935 consensus_queue.cc:1048] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:29:35.001756 14934 raft_consensus.cc:515] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.002177 14920 raft_consensus.cc:493] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.008275 14920 raft_consensus.cc:515] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.011214 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d168e6a345a94901a3dac027a767123e" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:35.010645 14906 raft_consensus.cc:515] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.012128 14711 raft_consensus.cc:2393] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in current term 1: Already voted for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in this term.
I20260709 12:29:35.012566 14920 leader_election.cc:290] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:35.014391 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544e89b953424226ae1c78c218552c43" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:35.015045 14860 raft_consensus.cc:3060] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:35.015206 14934 leader_election.cc:290] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:35.016193 14784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d168e6a345a94901a3dac027a767123e" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:35.017867 14906 leader_election.cc:290] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 election: Requested vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:35.017875 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b45722450c4ab48fae4b4faa88cfdd" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:35.019006 14784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b45722450c4ab48fae4b4faa88cfdd" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:35.019551 14711 raft_consensus.cc:3060] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:35.020493 14824 leader_election.cc:304] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd
I20260709 12:29:35.019649 14784 raft_consensus.cc:3060] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:35.022533 14934 raft_consensus.cc:3060] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:35.022831 14785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544e89b953424226ae1c78c218552c43" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:35.023142 14860 raft_consensus.cc:2468] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 1.
I20260709 12:29:35.023746 14785 raft_consensus.cc:3060] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:35.024459 14674 leader_election.cc:304] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:35.025525 14906 raft_consensus.cc:2804] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:35.026027 14906 raft_consensus.cc:697] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Becoming Leader. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:35.027333 14906 consensus_queue.cc:237] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.038836 14934 raft_consensus.cc:2749] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:35.042065 14784 raft_consensus.cc:2468] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 1.
I20260709 12:29:35.043996 14824 leader_election.cc:304] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:35.045238 14934 raft_consensus.cc:2804] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:35.045539 14711 raft_consensus.cc:2468] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 1.
I20260709 12:29:35.048278 14934 raft_consensus.cc:697] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Leader. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:35.048872 14785 raft_consensus.cc:2468] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 1.
I20260709 12:29:35.049800 14934 consensus_queue.cc:237] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.062496 14609 catalog_manager.cc:5697] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:35.067621 14607 catalog_manager.cc:5697] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193). New cstate: current_term: 1 leader_uuid: "21c8774363c141f7868e4fc3ec5854dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:35.078204 14932 consensus_queue.cc:1048] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:35.088239 14905 consensus_queue.cc:1048] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:35.173617 14906 raft_consensus.cc:493] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:35.173929 14905 consensus_queue.cc:1048] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:35.174155 14906 raft_consensus.cc:515] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.176365 14906 leader_election.cc:290] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:35.176950 14784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63cdd55867645ababeebf041ec58a26" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:35.177521 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63cdd55867645ababeebf041ec58a26" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:35.179544 14674 leader_election.cc:304] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd; no voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:35.180428 14906 raft_consensus.cc:2749] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:35.184387 14710 raft_consensus.cc:3060] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:35.193661 14933 consensus_queue.cc:1048] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:35.235913 14784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d168e6a345a94901a3dac027a767123e"
dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
mode: GRACEFUL
new_leader_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:58672
I20260709 12:29:35.236433 14784 raft_consensus.cc:606] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Received request to transfer leadership to TS 21c8774363c141f7868e4fc3ec5854dd
I20260709 12:29:35.242347 14784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3928359ec0c2401e98c97cb66daa7126"
dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
mode: GRACEFUL
new_leader_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:58672
I20260709 12:29:35.242975 14784 raft_consensus.cc:606] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Received request to transfer leadership to TS 21c8774363c141f7868e4fc3ec5854dd
I20260709 12:29:35.258383 14933 consensus_queue.cc:1048] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:35.258926 14860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "170e6f1f2be94ad1afa5b0ab4c556d87"
dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
mode: GRACEFUL
new_leader_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:33830
I20260709 12:29:35.259573 14860 raft_consensus.cc:606] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Received request to transfer leadership to TS 21c8774363c141f7868e4fc3ec5854dd
I20260709 12:29:35.266847 14934 consensus_queue.cc:1048] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:35.268115 14859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b"
dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
mode: GRACEFUL
new_leader_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:33830
I20260709 12:29:35.268800 14859 raft_consensus.cc:606] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Received request to transfer leadership to TS 21c8774363c141f7868e4fc3ec5854dd
I20260709 12:29:35.269714 14905 consensus_queue.cc:1048] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:35.283114 14925 raft_consensus.cc:993] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23: : Instructing follower 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:35.283643 14932 raft_consensus.cc:1081] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Signalling peer 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:35.287542 14710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d168e6a345a94901a3dac027a767123e"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:51076
I20260709 12:29:35.288236 14710 raft_consensus.cc:493] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:35.288708 14710 raft_consensus.cc:3060] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.290171 14934 consensus_queue.cc:1048] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:35.301043 14859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4350e9f3eba145d89b0f53790e46cc50"
dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
mode: GRACEFUL
new_leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
 from {username='slave'} at 127.0.0.1:33830
I20260709 12:29:35.300726 14710 raft_consensus.cc:515] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.303675 14934 consensus_queue.cc:1048] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:35.304188 14785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d168e6a345a94901a3dac027a767123e" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:35.304857 14785 raft_consensus.cc:3055] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:35.304553 14920 consensus_queue.cc:1048] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:35.305986 14710 leader_election.cc:290] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Requested vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:35.305114 14785 raft_consensus.cc:740] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:35.307473 14785 consensus_queue.cc:260] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.309487 14785 raft_consensus.cc:3060] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.313813 14920 raft_consensus.cc:993] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50: : Instructing follower 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:35.314507 14920 raft_consensus.cc:1081] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Signalling peer 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:35.318001 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d168e6a345a94901a3dac027a767123e" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:35.318692 14860 raft_consensus.cc:3060] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.323772 14709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "170e6f1f2be94ad1afa5b0ab4c556d87"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:51080
I20260709 12:29:35.327203 14709 raft_consensus.cc:493] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:35.327641 14709 raft_consensus.cc:3060] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.327800 14859 raft_consensus.cc:606] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Received request to transfer leadership to TS 1a85a9ae358f49448d7287ce9a1b0f23
I20260709 12:29:35.327891 14860 raft_consensus.cc:2468] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:35.329368 14674 leader_election.cc:304] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:35.330353 14906 raft_consensus.cc:2804] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:35.330828 14906 raft_consensus.cc:697] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Becoming Leader. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:35.331692 14906 consensus_queue.cc:237] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.343148 14859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3b45722450c4ab48fae4b4faa88cfdd"
dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
mode: GRACEFUL
new_leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
 from {username='slave'} at 127.0.0.1:33830
I20260709 12:29:35.343766 14859 raft_consensus.cc:606] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Received request to transfer leadership to TS 1a85a9ae358f49448d7287ce9a1b0f23
I20260709 12:29:35.347330 14709 raft_consensus.cc:515] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.354882 14784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "170e6f1f2be94ad1afa5b0ab4c556d87" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:35.355758 14784 raft_consensus.cc:3060] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.359262 14609 catalog_manager.cc:5697] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd reported cstate change: term changed from 1 to 2, leader changed from 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194) to 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193). New cstate: current_term: 2 leader_uuid: "21c8774363c141f7868e4fc3ec5854dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:35.361560 14709 leader_election.cc:290] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Requested vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:35.362908 14785 raft_consensus.cc:2468] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:35.363215 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "170e6f1f2be94ad1afa5b0ab4c556d87" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:35.363837 14859 raft_consensus.cc:3055] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:35.364185 14859 raft_consensus.cc:740] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:35.365000 14859 consensus_queue.cc:260] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.366176 14859 raft_consensus.cc:3060] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.366564 14910 raft_consensus.cc:993] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50: : Instructing follower 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:35.367074 14920 raft_consensus.cc:1081] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Signalling peer 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:35.369338 14710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:51080
I20260709 12:29:35.369957 14710 raft_consensus.cc:493] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:35.370329 14710 raft_consensus.cc:3060] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.371641 14784 raft_consensus.cc:2468] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:35.372926 14675 leader_election.cc:304] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:35.374166 14906 raft_consensus.cc:2804] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:35.374647 14906 raft_consensus.cc:697] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Becoming Leader. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:35.375464 14906 consensus_queue.cc:237] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.378180 14710 raft_consensus.cc:515] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.379292 14709 raft_consensus.cc:1240] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Rejecting Update request from peer 8d1e4a4db8eb4df4bed6a3aa27230d50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:35.380999 14710 leader_election.cc:290] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Requested vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:35.381824 14785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:35.382476 14785 raft_consensus.cc:3060] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.383940 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:35.379201 14859 raft_consensus.cc:2468] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:35.384580 14860 raft_consensus.cc:3055] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:35.385840 14860 raft_consensus.cc:740] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:35.386749 14860 consensus_queue.cc:260] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.387908 14860 raft_consensus.cc:3060] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.389035 14606 catalog_manager.cc:5697] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd reported cstate change: term changed from 1 to 2, leader changed from 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195) to 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193). New cstate: current_term: 2 leader_uuid: "21c8774363c141f7868e4fc3ec5854dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:35.392661 14785 raft_consensus.cc:2468] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:35.396886 14860 raft_consensus.cc:2468] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:35.398468 14674 leader_election.cc:304] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:35.399417 14906 raft_consensus.cc:2804] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:35.399897 14906 raft_consensus.cc:697] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Becoming Leader. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:35.400795 14906 consensus_queue.cc:237] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.410534 14709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "544e89b953424226ae1c78c218552c43"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
mode: GRACEFUL
new_leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
 from {username='slave'} at 127.0.0.1:51092
I20260709 12:29:35.411201 14709 raft_consensus.cc:606] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Received request to transfer leadership to TS 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:35.411453 14609 catalog_manager.cc:5697] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd reported cstate change: term changed from 1 to 2, leader changed from 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195) to 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193). New cstate: current_term: 2 leader_uuid: "21c8774363c141f7868e4fc3ec5854dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:35.420979 14934 raft_consensus.cc:993] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50: : Instructing follower 1a85a9ae358f49448d7287ce9a1b0f23 to start an election
I20260709 12:29:35.421553 14910 raft_consensus.cc:1081] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Signalling peer 1a85a9ae358f49448d7287ce9a1b0f23 to start an election
I20260709 12:29:35.423331 14785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4350e9f3eba145d89b0f53790e46cc50"
dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
 from {username='slave'} at 127.0.0.1:58658
I20260709 12:29:35.423992 14785 raft_consensus.cc:493] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:35.424348 14785 raft_consensus.cc:3060] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.429385 14925 consensus_queue.cc:1048] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:35.430917 14785 raft_consensus.cc:515] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.433313 14785 leader_election.cc:290] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 2 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:35.434016 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4350e9f3eba145d89b0f53790e46cc50" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:35.434857 14859 raft_consensus.cc:3055] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:35.435249 14859 raft_consensus.cc:740] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:35.436290 14859 consensus_queue.cc:260] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.437994 14859 raft_consensus.cc:3060] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.434042 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4350e9f3eba145d89b0f53790e46cc50" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:35.442217 14709 raft_consensus.cc:3060] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.446013 14905 consensus_queue.cc:1048] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:35.449538 14910 consensus_queue.cc:1048] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:35.451503 14709 raft_consensus.cc:2468] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 2.
I20260709 12:29:35.452778 14750 leader_election.cc:304] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:35.453748 14925 raft_consensus.cc:2804] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:35.454285 14925 raft_consensus.cc:697] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Becoming Leader. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:35.455159 14925 consensus_queue.cc:237] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.457324 14859 raft_consensus.cc:2468] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 2.
I20260709 12:29:35.466266 14935 raft_consensus.cc:993] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50: : Instructing follower 1a85a9ae358f49448d7287ce9a1b0f23 to start an election
I20260709 12:29:35.466768 14935 raft_consensus.cc:1081] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Signalling peer 1a85a9ae358f49448d7287ce9a1b0f23 to start an election
I20260709 12:29:35.469878 14932 raft_consensus.cc:993] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23: : Instructing follower 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:35.470396 14932 raft_consensus.cc:1081] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Signalling peer 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:35.475675 14710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3928359ec0c2401e98c97cb66daa7126"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:51076
I20260709 12:29:35.476346 14710 raft_consensus.cc:493] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:35.476835 14710 raft_consensus.cc:3060] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.477321 14910 consensus_queue.cc:1048] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:35.477721 14786 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3b45722450c4ab48fae4b4faa88cfdd"
dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
 from {username='slave'} at 127.0.0.1:58658
I20260709 12:29:35.478467 14786 raft_consensus.cc:493] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:35.478888 14786 raft_consensus.cc:3060] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.484707 14710 raft_consensus.cc:515] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.488233 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3928359ec0c2401e98c97cb66daa7126" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:35.488355 14710 leader_election.cc:290] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Requested vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:35.488936 14783 raft_consensus.cc:3055] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:35.489322 14783 raft_consensus.cc:740] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:35.490239 14783 consensus_queue.cc:260] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.491746 14783 raft_consensus.cc:3060] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.491883 14786 raft_consensus.cc:515] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.493201 14784 raft_consensus.cc:1240] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Rejecting Update request from peer 8d1e4a4db8eb4df4bed6a3aa27230d50 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:35.492627 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3928359ec0c2401e98c97cb66daa7126" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:35.495302 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b45722450c4ab48fae4b4faa88cfdd" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:35.495985 14709 raft_consensus.cc:3060] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.495927 14910 consensus_queue.cc:1059] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:35.495569 14859 raft_consensus.cc:3060] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.496973 14934 raft_consensus.cc:3055] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:35.497324 14934 raft_consensus.cc:740] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:35.498211 14934 consensus_queue.cc:260] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.499344 14934 raft_consensus.cc:3060] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.501360 14783 raft_consensus.cc:2468] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:35.502763 14675 leader_election.cc:304] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:35.503024 14786 leader_election.cc:290] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 2 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:35.503749 14906 raft_consensus.cc:2804] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:35.504159 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b45722450c4ab48fae4b4faa88cfdd" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:35.504279 14906 raft_consensus.cc:697] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Becoming Leader. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:35.505846 14709 raft_consensus.cc:2468] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 2.
I20260709 12:29:35.505960 14906 consensus_queue.cc:237] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.507293 14750 leader_election.cc:304] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:35.508168 14925 raft_consensus.cc:2804] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:35.508668 14925 raft_consensus.cc:697] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Becoming Leader. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:35.509559 14859 raft_consensus.cc:2468] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:35.509519 14925 consensus_queue.cc:237] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.512156 14906 consensus_queue.cc:1048] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:35.514552 14606 catalog_manager.cc:5697] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 reported cstate change: term changed from 1 to 2, leader changed from 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195) to 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:35.527789 14906 consensus_queue.cc:1048] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:35.528126 14860 raft_consensus.cc:2468] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 2.
I20260709 12:29:35.533632 14609 catalog_manager.cc:5697] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd reported cstate change: term changed from 1 to 2, leader changed from 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194) to 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193). New cstate: current_term: 2 leader_uuid: "21c8774363c141f7868e4fc3ec5854dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:35.537544 14981 raft_consensus.cc:993] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd: : Instructing follower 8d1e4a4db8eb4df4bed6a3aa27230d50 to start an election
I20260709 12:29:35.538097 14958 raft_consensus.cc:1081] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Signalling peer 8d1e4a4db8eb4df4bed6a3aa27230d50 to start an election
I20260709 12:29:35.540036 14859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "544e89b953424226ae1c78c218552c43"
dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
 from {username='slave'} at 127.0.0.1:33826
I20260709 12:29:35.540727 14859 raft_consensus.cc:493] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:35.541199 14859 raft_consensus.cc:3060] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.546432 14606 catalog_manager.cc:5697] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 reported cstate change: term changed from 1 to 2, leader changed from 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195) to 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:35.549598 14859 raft_consensus.cc:515] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.552148 14859 leader_election.cc:290] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 2 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:35.554703 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544e89b953424226ae1c78c218552c43" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:35.554805 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544e89b953424226ae1c78c218552c43" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:35.555373 14709 raft_consensus.cc:3055] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:35.555557 14783 raft_consensus.cc:3060] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.555790 14709 raft_consensus.cc:740] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:35.556742 14709 consensus_queue.cc:260] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.558095 14709 raft_consensus.cc:3060] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:35.564242 14783 raft_consensus.cc:2468] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 2.
I20260709 12:29:35.565621 14824 leader_election.cc:304] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:35.566161 14709 raft_consensus.cc:2468] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 2.
I20260709 12:29:35.566469 14934 raft_consensus.cc:2804] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:35.567147 14934 raft_consensus.cc:697] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 LEADER]: Becoming Leader. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:35.568084 14934 consensus_queue.cc:237] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:35.575443 14607 catalog_manager.cc:5697] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 reported cstate change: term changed from 1 to 2, leader changed from 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193) to 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:35.756044 14783 raft_consensus.cc:1275] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:35.757248 14981 consensus_queue.cc:1048] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:35.770179 14859 raft_consensus.cc:1275] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:35.773358 14859 raft_consensus.cc:1275] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:35.773449 14983 consensus_queue.cc:1048] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:35.774827 14981 consensus_queue.cc:1048] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:35.785306 14783 raft_consensus.cc:1275] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:35.786749 14958 consensus_queue.cc:1048] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:35.853266 14709 raft_consensus.cc:1275] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Refusing update from remote peer 1a85a9ae358f49448d7287ce9a1b0f23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:35.854741 14932 consensus_queue.cc:1048] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:35.862682 14860 raft_consensus.cc:1275] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Refusing update from remote peer 1a85a9ae358f49448d7287ce9a1b0f23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:35.864375 14905 consensus_queue.cc:1048] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:35.869280 14783 raft_consensus.cc:1275] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:35.870857 14958 consensus_queue.cc:1048] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:35.881438 14860 raft_consensus.cc:1275] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:35.882951 14981 consensus_queue.cc:1048] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:35.945654 14860 raft_consensus.cc:1275] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Refusing update from remote peer 1a85a9ae358f49448d7287ce9a1b0f23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:35.946952 14905 consensus_queue.cc:1048] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:35.955272 14709 raft_consensus.cc:1275] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Refusing update from remote peer 1a85a9ae358f49448d7287ce9a1b0f23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:35.956754 14905 consensus_queue.cc:1048] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:36.088478 14783 raft_consensus.cc:1275] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:36.089701 14860 raft_consensus.cc:1275] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:36.090013 14981 consensus_queue.cc:1048] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:36.091300 14983 consensus_queue.cc:1048] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:36.111457 14783 raft_consensus.cc:1275] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Refusing update from remote peer 8d1e4a4db8eb4df4bed6a3aa27230d50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:36.113399 14934 consensus_queue.cc:1048] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:36.123023 14709 raft_consensus.cc:1275] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Refusing update from remote peer 8d1e4a4db8eb4df4bed6a3aa27230d50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:36.124585 14934 consensus_queue.cc:1048] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:38.452656 14607 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40446:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:38.455695 14607 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:38.521034 14697 tablet_service.cc:1511] Processing CreateTablet for tablet 5b3e04ad84844c8db14f8e8604c7a6c3 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:38.523151 14697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b3e04ad84844c8db14f8e8604c7a6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.526878 14775 tablet_service.cc:1511] Processing CreateTablet for tablet 5b3e04ad84844c8db14f8e8604c7a6c3 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:38.527477 14844 tablet_service.cc:1511] Processing CreateTablet for tablet 5b3e04ad84844c8db14f8e8604c7a6c3 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:38.527774 14701 tablet_service.cc:1511] Processing CreateTablet for tablet 756b1fe131b441af8553d5b5b3315e69 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:38.528951 14775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b3e04ad84844c8db14f8e8604c7a6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.529542 14701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 756b1fe131b441af8553d5b5b3315e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.529692 14844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b3e04ad84844c8db14f8e8604c7a6c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.533892 14770 tablet_service.cc:1511] Processing CreateTablet for tablet 756b1fe131b441af8553d5b5b3315e69 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:38.535565 14770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 756b1fe131b441af8553d5b5b3315e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.546409 14849 tablet_service.cc:1511] Processing CreateTablet for tablet 756b1fe131b441af8553d5b5b3315e69 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:38.547999 14849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 756b1fe131b441af8553d5b5b3315e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.548838 14696 tablet_service.cc:1511] Processing CreateTablet for tablet f0aead1fd17c4a36b444c90e977c77a0 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:38.550555 14696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0aead1fd17c4a36b444c90e977c77a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.552239 14848 tablet_service.cc:1511] Processing CreateTablet for tablet f0aead1fd17c4a36b444c90e977c77a0 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:38.556159 14771 tablet_service.cc:1511] Processing CreateTablet for tablet f0aead1fd17c4a36b444c90e977c77a0 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:38.557922 14771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0aead1fd17c4a36b444c90e977c77a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.558132 14848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0aead1fd17c4a36b444c90e977c77a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.573678 14699 tablet_service.cc:1511] Processing CreateTablet for tablet 974bd7457c2e4665985fc72b109f6bb8 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:38.575256 14699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 974bd7457c2e4665985fc72b109f6bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.581751 14774 tablet_service.cc:1511] Processing CreateTablet for tablet 974bd7457c2e4665985fc72b109f6bb8 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:38.583261 14774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 974bd7457c2e4665985fc72b109f6bb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.584164 14698 tablet_service.cc:1511] Processing CreateTablet for tablet b347919507c445e094f7521885abfdd2 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:38.585683 14698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b347919507c445e094f7521885abfdd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.586825 14845 tablet_service.cc:1511] Processing CreateTablet for tablet 974bd7457c2e4665985fc72b109f6bb8 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:38.588495 14845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 974bd7457c2e4665985fc72b109f6bb8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:29:38.594821 14657 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:38.599166 14776 tablet_service.cc:1511] Processing CreateTablet for tablet b347919507c445e094f7521885abfdd2 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:38.600713 14776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b347919507c445e094f7521885abfdd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.603101 15003 tablet_bootstrap.cc:492] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:38.612821 15003 tablet_bootstrap.cc:654] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.618381 14850 tablet_service.cc:1511] Processing CreateTablet for tablet b347919507c445e094f7521885abfdd2 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:38.622630 14850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b347919507c445e094f7521885abfdd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.643088 14700 tablet_service.cc:1511] Processing CreateTablet for tablet 4c531a556c3746688ed6ee760df080ae (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:38.645104 14700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c531a556c3746688ed6ee760df080ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.660718 15004 tablet_bootstrap.cc:492] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:38.685806 15004 tablet_bootstrap.cc:654] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.688282 15003 tablet_bootstrap.cc:492] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:38.692190 15005 tablet_bootstrap.cc:492] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:38.693984 14701 tablet_service.cc:1511] Processing CreateTablet for tablet e044c801fa47414e82f7ce946e0ed749 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:38.695360 14701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e044c801fa47414e82f7ce946e0ed749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.697815 15003 ts_tablet_manager.cc:1403] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.095s	user 0.013s	sys 0.013s
I20260709 12:29:38.706194 14774 tablet_service.cc:1511] Processing CreateTablet for tablet 4c531a556c3746688ed6ee760df080ae (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:38.707652 14774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c531a556c3746688ed6ee760df080ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.701470 15003 raft_consensus.cc:359] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.715770 15003 raft_consensus.cc:385] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.715507 14845 tablet_service.cc:1511] Processing CreateTablet for tablet 4c531a556c3746688ed6ee760df080ae (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:38.716140 15003 raft_consensus.cc:740] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.716995 14845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c531a556c3746688ed6ee760df080ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.727842 15004 tablet_bootstrap.cc:492] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:38.728462 15004 ts_tablet_manager.cc:1403] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.068s	user 0.000s	sys 0.014s
I20260709 12:29:38.723474 14776 tablet_service.cc:1511] Processing CreateTablet for tablet e044c801fa47414e82f7ce946e0ed749 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:38.731242 14774 tablet_service.cc:1511] Processing CreateTablet for tablet 29e2aafd2afb4ac394d856a620b50c18 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:38.717058 15003 consensus_queue.cc:260] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.732779 14700 tablet_service.cc:1511] Processing CreateTablet for tablet 29e2aafd2afb4ac394d856a620b50c18 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:38.733796 14696 tablet_service.cc:1511] Processing CreateTablet for tablet 03e8a2e7da7e423baa445ad77abe8ca0 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:38.735441 14850 tablet_service.cc:1511] Processing CreateTablet for tablet 29e2aafd2afb4ac394d856a620b50c18 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:38.731732 14776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e044c801fa47414e82f7ce946e0ed749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.739830 14700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29e2aafd2afb4ac394d856a620b50c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.736804 15003 ts_tablet_manager.cc:1434] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.001s
I20260709 12:29:38.724170 15005 tablet_bootstrap.cc:654] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.741545 15003 tablet_bootstrap.cc:492] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:38.724888 14849 tablet_service.cc:1511] Processing CreateTablet for tablet e044c801fa47414e82f7ce946e0ed749 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:38.738533 14771 tablet_service.cc:1511] Processing CreateTablet for tablet 03e8a2e7da7e423baa445ad77abe8ca0 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:38.739154 14844 tablet_service.cc:1511] Processing CreateTablet for tablet 03e8a2e7da7e423baa445ad77abe8ca0 (DEFAULT_TABLE table=soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:38.742657 14849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e044c801fa47414e82f7ce946e0ed749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.731760 15004 raft_consensus.cc:359] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.744966 14844 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03e8a2e7da7e423baa445ad77abe8ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.745380 15004 raft_consensus.cc:385] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.748592 14850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29e2aafd2afb4ac394d856a620b50c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.748016 15004 raft_consensus.cc:740] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.754662 15004 consensus_queue.cc:260] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.740387 14774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29e2aafd2afb4ac394d856a620b50c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.756820 14771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03e8a2e7da7e423baa445ad77abe8ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.764173 14696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03e8a2e7da7e423baa445ad77abe8ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.766849 15004 ts_tablet_manager.cc:1434] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.038s	user 0.001s	sys 0.007s
I20260709 12:29:38.772349 15005 tablet_bootstrap.cc:492] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:38.772934 15005 ts_tablet_manager.cc:1403] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.081s	user 0.007s	sys 0.011s
I20260709 12:29:38.784785 15003 tablet_bootstrap.cc:654] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.784786 15004 tablet_bootstrap.cc:492] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:38.776407 15005 raft_consensus.cc:359] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.789744 15005 raft_consensus.cc:385] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.790107 15005 raft_consensus.cc:740] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.791177 15005 consensus_queue.cc:260] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.792829 15004 tablet_bootstrap.cc:654] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.794368 15005 ts_tablet_manager.cc:1434] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.021s	user 0.001s	sys 0.009s
I20260709 12:29:38.795483 15005 tablet_bootstrap.cc:492] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:38.806757 15005 tablet_bootstrap.cc:654] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.819196 15003 tablet_bootstrap.cc:492] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:38.819785 15003 ts_tablet_manager.cc:1403] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.079s	user 0.008s	sys 0.004s
I20260709 12:29:38.823247 15003 raft_consensus.cc:359] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.823999 15003 raft_consensus.cc:385] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.824265 15005 tablet_bootstrap.cc:492] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:38.824836 15005 ts_tablet_manager.cc:1403] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.030s	user 0.005s	sys 0.008s
I20260709 12:29:38.826005 15004 tablet_bootstrap.cc:492] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:38.826490 15004 ts_tablet_manager.cc:1403] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.009s
I20260709 12:29:38.827930 15005 raft_consensus.cc:359] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.828730 15005 raft_consensus.cc:385] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.829071 15005 raft_consensus.cc:740] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.828863 15004 raft_consensus.cc:359] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.824324 15003 raft_consensus.cc:740] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.829795 15004 raft_consensus.cc:385] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.830138 15004 raft_consensus.cc:740] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.830248 15005 consensus_queue.cc:260] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.830585 15003 consensus_queue.cc:260] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.831034 15004 consensus_queue.cc:260] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.833364 15003 ts_tablet_manager.cc:1434] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:29:38.834491 15003 tablet_bootstrap.cc:492] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:38.837409 15005 ts_tablet_manager.cc:1434] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 12:29:38.838510 15005 tablet_bootstrap.cc:492] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:38.838799 15004 ts_tablet_manager.cc:1434] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 12:29:38.839897 15004 tablet_bootstrap.cc:492] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:38.842263 15003 tablet_bootstrap.cc:654] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.847534 15004 tablet_bootstrap.cc:654] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.850191 15005 tablet_bootstrap.cc:654] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.863283 15005 tablet_bootstrap.cc:492] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:38.863870 15005 ts_tablet_manager.cc:1403] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.007s
I20260709 12:29:38.867288 15005 raft_consensus.cc:359] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.868088 15005 raft_consensus.cc:385] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.868404 15005 raft_consensus.cc:740] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.871184 15005 consensus_queue.cc:260] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.873183 15004 tablet_bootstrap.cc:492] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:38.874024 15004 ts_tablet_manager.cc:1403] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.008s
I20260709 12:29:38.877064 15005 ts_tablet_manager.cc:1434] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.002s
I20260709 12:29:38.878335 15005 tablet_bootstrap.cc:492] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:38.884439 15004 raft_consensus.cc:359] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.885058 15004 raft_consensus.cc:385] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.885371 15004 raft_consensus.cc:740] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.886319 15004 consensus_queue.cc:260] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.887635 15003 tablet_bootstrap.cc:492] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:38.888229 15003 ts_tablet_manager.cc:1403] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.035s
I20260709 12:29:38.888484 15005 tablet_bootstrap.cc:654] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.891214 15004 ts_tablet_manager.cc:1434] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.003s
I20260709 12:29:38.891414 15003 raft_consensus.cc:359] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.892212 15003 raft_consensus.cc:385] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.892510 15003 raft_consensus.cc:740] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.893143 15004 tablet_bootstrap.cc:492] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:38.893368 15003 consensus_queue.cc:260] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.895875 15003 ts_tablet_manager.cc:1434] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:38.902565 15003 tablet_bootstrap.cc:492] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:38.906145 15004 tablet_bootstrap.cc:654] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.908710 14992 raft_consensus.cc:493] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.909346 14992 raft_consensus.cc:515] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.911753 14992 leader_election.cc:290] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:38.914044 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756b1fe131b441af8553d5b5b3315e69" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:38.914259 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756b1fe131b441af8553d5b5b3315e69" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:38.914911 14709 raft_consensus.cc:2468] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 0.
I20260709 12:29:38.915304 15004 tablet_bootstrap.cc:492] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:38.915249 14783 raft_consensus.cc:2468] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 0.
I20260709 12:29:38.916030 15004 ts_tablet_manager.cc:1403] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.001s
I20260709 12:29:38.916621 14824 leader_election.cc:304] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:38.917676 14932 raft_consensus.cc:493] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.918269 14992 raft_consensus.cc:2804] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:38.918776 14992 raft_consensus.cc:493] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:38.918345 14932 raft_consensus.cc:515] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.919243 14992 raft_consensus.cc:3060] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:38.919651 15004 raft_consensus.cc:359] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.920552 15004 raft_consensus.cc:385] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.920876 15004 raft_consensus.cc:740] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.921777 15004 consensus_queue.cc:260] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.928310 15003 tablet_bootstrap.cc:654] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.928977 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b3e04ad84844c8db14f8e8604c7a6c3" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:38.929996 14709 raft_consensus.cc:2468] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 0.
I20260709 12:29:38.932060 14750 leader_election.cc:304] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:38.937266 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b3e04ad84844c8db14f8e8604c7a6c3" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:38.937942 14932 leader_election.cc:290] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:38.938022 14860 raft_consensus.cc:2468] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 0.
I20260709 12:29:38.938735 14932 raft_consensus.cc:2804] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:38.939177 14932 raft_consensus.cc:493] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:38.939586 14932 raft_consensus.cc:3060] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:38.945227 15004 ts_tablet_manager.cc:1434] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.020s
I20260709 12:29:38.946587 15004 tablet_bootstrap.cc:492] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:38.947862 15005 tablet_bootstrap.cc:492] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:38.948472 15005 ts_tablet_manager.cc:1403] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.070s	user 0.019s	sys 0.037s
I20260709 12:29:38.951220 14932 raft_consensus.cc:515] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.954285 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b3e04ad84844c8db14f8e8604c7a6c3" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:38.954898 14709 raft_consensus.cc:3060] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:38.955618 14992 raft_consensus.cc:515] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.957899 15004 tablet_bootstrap.cc:654] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.959089 14710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756b1fe131b441af8553d5b5b3315e69" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:38.959791 14710 raft_consensus.cc:3060] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:38.960071 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756b1fe131b441af8553d5b5b3315e69" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:38.961037 14783 raft_consensus.cc:3060] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:38.961176 15005 raft_consensus.cc:359] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.962301 15005 raft_consensus.cc:385] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.962677 15005 raft_consensus.cc:740] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.964171 15005 consensus_queue.cc:260] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.967679 15005 ts_tablet_manager.cc:1434] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:29:38.969125 15005 tablet_bootstrap.cc:492] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:38.972316 14783 raft_consensus.cc:2468] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 1.
I20260709 12:29:38.974047 14824 leader_election.cc:304] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:38.975351 15017 raft_consensus.cc:2804] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:38.979079 15005 tablet_bootstrap.cc:654] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.983947 14992 leader_election.cc:290] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:38.985548 15017 raft_consensus.cc:697] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Leader. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:38.987056 15017 consensus_queue.cc:237] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:38.995994 15005 tablet_bootstrap.cc:492] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:38.996735 14710 raft_consensus.cc:2468] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 1.
I20260709 12:29:38.996860 15005 ts_tablet_manager.cc:1403] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.012s
I20260709 12:29:39.002642 15005 raft_consensus.cc:359] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.003793 15005 raft_consensus.cc:385] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.003554 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b3e04ad84844c8db14f8e8604c7a6c3" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:39.004271 15005 raft_consensus.cc:740] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.004550 14860 raft_consensus.cc:3060] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.005383 15005 consensus_queue.cc:260] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.009224 15005 ts_tablet_manager.cc:1434] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 12:29:39.010800 15005 tablet_bootstrap.cc:492] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:39.000888 14932 leader_election.cc:290] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.014604 14709 raft_consensus.cc:2468] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 1.
I20260709 12:29:39.022611 14860 raft_consensus.cc:2468] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 1.
I20260709 12:29:39.026777 15005 tablet_bootstrap.cc:654] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.036788 14991 raft_consensus.cc:493] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.037436 14991 raft_consensus.cc:515] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.040858 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b347919507c445e094f7521885abfdd2" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
W20260709 12:29:39.042685 14675 leader_election.cc:343] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.043946 14991 leader_election.cc:290] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.044691 14750 leader_election.cc:304] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:39.045815 14932 raft_consensus.cc:2804] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.046749 14932 raft_consensus.cc:697] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Becoming Leader. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:39.047695 14932 consensus_queue.cc:237] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.052727 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b347919507c445e094f7521885abfdd2" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
W20260709 12:29:39.054921 14674 leader_election.cc:343] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.055539 14674 leader_election.cc:304] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd; no voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:39.056692 15026 raft_consensus.cc:2749] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:29:39.062469 14749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:35339, user_credentials={real_user=slave}} blocked reactor thread for 62227.6us
I20260709 12:29:39.070397 14607 catalog_manager.cc:5697] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.078826 15018 raft_consensus.cc:493] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.079599 15018 raft_consensus.cc:515] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.084240 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974bd7457c2e4665985fc72b109f6bb8" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:39.085372 14711 raft_consensus.cc:2468] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 0.
I20260709 12:29:39.087095 14750 leader_election.cc:304] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:39.089242 15018 leader_election.cc:290] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.089948 15018 raft_consensus.cc:2804] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.090399 15018 raft_consensus.cc:493] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.091001 15018 raft_consensus.cc:3060] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.105247 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974bd7457c2e4665985fc72b109f6bb8" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:39.106238 14860 raft_consensus.cc:2468] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 0.
I20260709 12:29:39.116498 15003 tablet_bootstrap.cc:492] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:39.117141 15003 ts_tablet_manager.cc:1403] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.215s	user 0.035s	sys 0.024s
I20260709 12:29:39.120694 15004 tablet_bootstrap.cc:492] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:39.121227 15005 tablet_bootstrap.cc:492] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:39.121327 15004 ts_tablet_manager.cc:1403] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.175s	user 0.051s	sys 0.090s
I20260709 12:29:39.122004 15005 ts_tablet_manager.cc:1403] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.112s	user 0.026s	sys 0.038s
I20260709 12:29:39.124812 15004 raft_consensus.cc:359] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.125684 15004 raft_consensus.cc:385] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.125268 15005 raft_consensus.cc:359] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.126149 15004 raft_consensus.cc:740] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.126348 15005 raft_consensus.cc:385] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.126785 15005 raft_consensus.cc:740] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.127077 15004 consensus_queue.cc:260] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.127628 15005 consensus_queue.cc:260] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.130105 15004 ts_tablet_manager.cc:1434] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:29:39.131102 15004 tablet_bootstrap.cc:492] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:39.134009 15003 raft_consensus.cc:359] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.134898 15005 ts_tablet_manager.cc:1434] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 12:29:39.140314 15017 raft_consensus.cc:493] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.141261 15004 tablet_bootstrap.cc:654] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.141422 15017 raft_consensus.cc:515] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.142098 15005 tablet_bootstrap.cc:492] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:39.143940 15017 leader_election.cc:290] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:39.146816 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0aead1fd17c4a36b444c90e977c77a0" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:39.147707 14709 raft_consensus.cc:2468] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 0.
I20260709 12:29:39.148939 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0aead1fd17c4a36b444c90e977c77a0" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:39.150633 15005 tablet_bootstrap.cc:654] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.150453 14824 leader_election.cc:304] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:39.151557 15017 raft_consensus.cc:2804] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.151989 15017 raft_consensus.cc:493] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.152369 15017 raft_consensus.cc:3060] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.153142 15004 tablet_bootstrap.cc:492] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:39.153687 15004 ts_tablet_manager.cc:1403] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.003s
I20260709 12:29:39.135083 15003 raft_consensus.cc:385] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.155112 15003 raft_consensus.cc:740] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.156766 15004 raft_consensus.cc:359] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.157861 15004 raft_consensus.cc:385] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.158205 15004 raft_consensus.cc:740] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.159061 15004 consensus_queue.cc:260] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.160490 14608 catalog_manager.cc:5697] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:39.161888 15004 ts_tablet_manager.cc:1434] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:29:39.162226 15003 consensus_queue.cc:260] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.163563 14783 raft_consensus.cc:2468] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 0.
I20260709 12:29:39.163748 15004 tablet_bootstrap.cc:492] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:39.138248 15018 raft_consensus.cc:515] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.169134 15018 leader_election.cc:290] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.169302 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974bd7457c2e4665985fc72b109f6bb8" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:39.169989 14709 raft_consensus.cc:3060] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.172077 15004 tablet_bootstrap.cc:654] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.173522 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974bd7457c2e4665985fc72b109f6bb8" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:39.174611 14859 raft_consensus.cc:3060] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.174634 15003 ts_tablet_manager.cc:1434] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.000s
I20260709 12:29:39.176061 15005 tablet_bootstrap.cc:492] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:39.176819 15005 ts_tablet_manager.cc:1403] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.004s
I20260709 12:29:39.180399 15005 raft_consensus.cc:359] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.181186 15005 raft_consensus.cc:385] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.181520 15005 raft_consensus.cc:740] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.182310 15005 consensus_queue.cc:260] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.183671 14709 raft_consensus.cc:2468] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 1.
I20260709 12:29:39.176152 15003 tablet_bootstrap.cc:492] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:39.184986 14859 raft_consensus.cc:2468] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 1.
I20260709 12:29:39.188068 14749 leader_election.cc:304] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:39.183705 15017 raft_consensus.cc:515] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.190776 15017 leader_election.cc:290] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:39.191355 15018 raft_consensus.cc:2804] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.193034 15003 tablet_bootstrap.cc:654] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.193338 15005 ts_tablet_manager.cc:1434] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:29:39.193181 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0aead1fd17c4a36b444c90e977c77a0" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:39.194108 14783 raft_consensus.cc:3060] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.193181 15018 raft_consensus.cc:697] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Becoming Leader. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:39.196419 15026 raft_consensus.cc:493] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.197227 15018 consensus_queue.cc:237] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.196986 15026 raft_consensus.cc:515] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.202793 15026 leader_election.cc:290] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.202867 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0aead1fd17c4a36b444c90e977c77a0" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:39.203817 14709 raft_consensus.cc:3060] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.203931 14786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c531a556c3746688ed6ee760df080ae" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:39.205006 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c531a556c3746688ed6ee760df080ae" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:39.205767 14859 raft_consensus.cc:2468] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 0.
W20260709 12:29:39.205917 14675 leader_election.cc:343] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.207105 14674 leader_election.cc:304] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 1a85a9ae358f49448d7287ce9a1b0f23
I20260709 12:29:39.207504 14783 raft_consensus.cc:2468] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 1.
I20260709 12:29:39.208758 15005 tablet_bootstrap.cc:492] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:39.208923 14824 leader_election.cc:304] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:39.209291 15026 raft_consensus.cc:2804] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.209512 15003 tablet_bootstrap.cc:492] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:39.209766 15026 raft_consensus.cc:493] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.210129 15003 ts_tablet_manager.cc:1403] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.008s
I20260709 12:29:39.210281 15026 raft_consensus.cc:3060] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.210180 15017 raft_consensus.cc:2804] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.211403 15017 raft_consensus.cc:697] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Leader. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:39.212548 15017 consensus_queue.cc:237] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.220130 15004 tablet_bootstrap.cc:492] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:39.220713 15004 ts_tablet_manager.cc:1403] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.057s	user 0.003s	sys 0.017s
I20260709 12:29:39.224253 15004 raft_consensus.cc:359] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.224803 15005 tablet_bootstrap.cc:654] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.225064 15004 raft_consensus.cc:385] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.225385 15004 raft_consensus.cc:740] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.226292 15004 consensus_queue.cc:260] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.228816 15004 ts_tablet_manager.cc:1434] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:39.229712 15003 raft_consensus.cc:359] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.230521 15003 raft_consensus.cc:385] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.230818 15003 raft_consensus.cc:740] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.231190 14709 raft_consensus.cc:2468] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 1.
I20260709 12:29:39.231642 15003 consensus_queue.cc:260] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.235039 15003 ts_tablet_manager.cc:1434] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.005s
I20260709 12:29:39.236210 15003 tablet_bootstrap.cc:492] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:39.237877 15005 tablet_bootstrap.cc:492] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:39.238490 15005 ts_tablet_manager.cc:1403] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.002s
I20260709 12:29:39.239809 15004 tablet_bootstrap.cc:492] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:39.240244 15026 raft_consensus.cc:515] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.242002 15005 raft_consensus.cc:359] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.242760 15005 raft_consensus.cc:385] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.243052 15005 raft_consensus.cc:740] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.243971 15005 consensus_queue.cc:260] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.245353 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c531a556c3746688ed6ee760df080ae" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:39.246075 14859 raft_consensus.cc:3060] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.246986 15003 tablet_bootstrap.cc:654] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.248108 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c531a556c3746688ed6ee760df080ae" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
W20260709 12:29:39.252597 14675 leader_election.cc:343] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.253690 15004 tablet_bootstrap.cc:654] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.255643 15005 ts_tablet_manager.cc:1434] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:29:39.256762 15005 tablet_bootstrap.cc:492] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd: Bootstrap starting.
I20260709 12:29:39.262357 14607 catalog_manager.cc:5697] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:39.263571 15005 tablet_bootstrap.cc:654] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.262633 14608 catalog_manager.cc:5697] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.265831 14859 raft_consensus.cc:2468] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 1.
I20260709 12:29:39.268837 15003 tablet_bootstrap.cc:492] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:39.270750 15003 ts_tablet_manager.cc:1403] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.000s
I20260709 12:29:39.273100 14674 leader_election.cc:304] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 1a85a9ae358f49448d7287ce9a1b0f23
I20260709 12:29:39.274737 15003 raft_consensus.cc:359] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.275593 15003 raft_consensus.cc:385] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.275885 15005 tablet_bootstrap.cc:492] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd: No bootstrap required, opened a new log
I20260709 12:29:39.275965 15003 raft_consensus.cc:740] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.276553 15005 ts_tablet_manager.cc:1403] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.003s
I20260709 12:29:39.277402 15003 consensus_queue.cc:260] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.278642 15038 raft_consensus.cc:2804] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.279197 15038 raft_consensus.cc:697] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Becoming Leader. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:39.280129 15003 ts_tablet_manager.cc:1434] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:29:39.280077 15038 consensus_queue.cc:237] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.281371 15003 tablet_bootstrap.cc:492] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:39.287282 15005 raft_consensus.cc:359] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.287999 15005 raft_consensus.cc:385] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.288276 15005 raft_consensus.cc:740] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.289027 15003 tablet_bootstrap.cc:654] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.289122 15005 consensus_queue.cc:260] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.290354 14991 raft_consensus.cc:493] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.290897 14991 raft_consensus.cc:515] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.294976 15004 tablet_bootstrap.cc:492] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:39.295581 15004 ts_tablet_manager.cc:1403] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.056s	user 0.011s	sys 0.000s
I20260709 12:29:39.298727 15004 raft_consensus.cc:359] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.299500 15004 raft_consensus.cc:385] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.299813 15004 raft_consensus.cc:740] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.300704 15004 consensus_queue.cc:260] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.302908 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:39.303004 15003 tablet_bootstrap.cc:492] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:39.303422 15004 ts_tablet_manager.cc:1434] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:29:39.304056 15003 ts_tablet_manager.cc:1403] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.023s	user 0.020s	sys 0.000s
I20260709 12:29:39.304641 15004 tablet_bootstrap.cc:492] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Bootstrap starting.
I20260709 12:29:39.308414 15003 raft_consensus.cc:359] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.309468 15003 raft_consensus.cc:385] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.309943 15003 raft_consensus.cc:740] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
W20260709 12:29:39.309127 14675 leader_election.cc:343] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.310983 15003 consensus_queue.cc:260] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.312028 15004 tablet_bootstrap.cc:654] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.311914 14991 leader_election.cc:290] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.320781 15003 ts_tablet_manager.cc:1434] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:29:39.322250 15003 tablet_bootstrap.cc:492] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:39.323491 15026 leader_election.cc:290] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 election: Requested vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.326506 15018 raft_consensus.cc:493] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.328447 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:39.330235 15005 ts_tablet_manager.cc:1434] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.020s
I20260709 12:29:39.332181 15003 tablet_bootstrap.cc:654] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.334995 14859 raft_consensus.cc:2468] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 0.
I20260709 12:29:39.334841 15018 raft_consensus.cc:515] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.337512 14674 leader_election.cc:304] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 1a85a9ae358f49448d7287ce9a1b0f23
I20260709 12:29:39.338335 15018 leader_election.cc:290] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.338752 14991 raft_consensus.cc:2804] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.339205 14991 raft_consensus.cc:493] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.339735 14991 raft_consensus.cc:3060] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.341676 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e044c801fa47414e82f7ce946e0ed749" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:39.344640 15004 tablet_bootstrap.cc:492] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: No bootstrap required, opened a new log
I20260709 12:29:39.345178 15004 ts_tablet_manager.cc:1403] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent bootstrapping tablet: real 0.041s	user 0.022s	sys 0.002s
I20260709 12:29:39.346494 14709 raft_consensus.cc:2468] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 0.
I20260709 12:29:39.348069 14750 leader_election.cc:304] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:39.349017 15018 raft_consensus.cc:2804] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.349363 15018 raft_consensus.cc:493] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.348440 15004 raft_consensus.cc:359] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.349730 15018 raft_consensus.cc:3060] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.349809 15004 raft_consensus.cc:385] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.350180 15004 raft_consensus.cc:740] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.350942 15004 consensus_queue.cc:260] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.353420 15004 ts_tablet_manager.cc:1434] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:29:39.358397 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e044c801fa47414e82f7ce946e0ed749" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:39.358768 15003 tablet_bootstrap.cc:492] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:39.359355 15003 ts_tablet_manager.cc:1403] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.011s
I20260709 12:29:39.359653 14991 raft_consensus.cc:515] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.360136 14859 raft_consensus.cc:2468] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 0.
I20260709 12:29:39.357954 14609 catalog_manager.cc:5697] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193). New cstate: current_term: 1 leader_uuid: "21c8774363c141f7868e4fc3ec5854dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.362509 15003 raft_consensus.cc:359] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.363258 15003 raft_consensus.cc:385] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.363530 15003 raft_consensus.cc:740] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.367218 14991 leader_election.cc:290] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 election: Requested vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.368198 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
W20260709 12:29:39.369763 14675 leader_election.cc:343] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.370431 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:39.364351 15003 consensus_queue.cc:260] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.371443 15018 raft_consensus.cc:515] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.377529 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e044c801fa47414e82f7ce946e0ed749" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:39.378254 14711 raft_consensus.cc:3060] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.371577 14859 raft_consensus.cc:3060] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.373974 15003 ts_tablet_manager.cc:1434] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.005s
I20260709 12:29:39.386229 14711 raft_consensus.cc:2468] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 1.
I20260709 12:29:39.386839 15017 raft_consensus.cc:493] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.387442 15017 raft_consensus.cc:515] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.387625 14750 leader_election.cc:304] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:39.388629 14932 raft_consensus.cc:2804] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.389148 14932 raft_consensus.cc:697] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Becoming Leader. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:39.389731 15017 leader_election.cc:290] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:39.390075 14932 consensus_queue.cc:237] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.390712 15003 tablet_bootstrap.cc:492] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23: Bootstrap starting.
I20260709 12:29:39.391105 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b347919507c445e094f7521885abfdd2" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:39.391861 14783 raft_consensus.cc:2468] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 0.
I20260709 12:29:39.393110 14824 leader_election.cc:304] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:39.393823 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b347919507c445e094f7521885abfdd2" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:39.394043 15017 raft_consensus.cc:2804] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.394446 15017 raft_consensus.cc:493] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.394572 14709 raft_consensus.cc:2468] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 0.
I20260709 12:29:39.394824 15017 raft_consensus.cc:3060] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.401129 14859 raft_consensus.cc:2468] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 1.
I20260709 12:29:39.403268 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e044c801fa47414e82f7ce946e0ed749" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:39.403564 14674 leader_election.cc:304] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 1a85a9ae358f49448d7287ce9a1b0f23
I20260709 12:29:39.403999 14860 raft_consensus.cc:3060] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.404552 15026 raft_consensus.cc:2804] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.405085 15026 raft_consensus.cc:697] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Becoming Leader. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:39.405751 15017 raft_consensus.cc:515] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.406308 15026 consensus_queue.cc:237] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.408032 15017 leader_election.cc:290] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:39.402076 15018 leader_election.cc:290] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.410447 14608 catalog_manager.cc:5697] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.412362 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b347919507c445e094f7521885abfdd2" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:39.412806 15003 tablet_bootstrap.cc:654] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.413324 14709 raft_consensus.cc:3060] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.414927 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b347919507c445e094f7521885abfdd2" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:39.417141 14783 raft_consensus.cc:3060] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.420744 15003 tablet_bootstrap.cc:492] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23: No bootstrap required, opened a new log
I20260709 12:29:39.421325 15003 ts_tablet_manager.cc:1403] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent bootstrapping tablet: real 0.031s	user 0.001s	sys 0.011s
I20260709 12:29:39.424777 15003 raft_consensus.cc:359] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.425521 15003 raft_consensus.cc:385] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.425877 15003 raft_consensus.cc:740] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.425956 14783 raft_consensus.cc:2468] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 1.
I20260709 12:29:39.426805 15003 consensus_queue.cc:260] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.427254 14824 leader_election.cc:304] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:39.428215 15017 raft_consensus.cc:2804] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.428740 15017 raft_consensus.cc:697] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Leader. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:39.429486 15003 ts_tablet_manager.cc:1434] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:39.429618 15017 consensus_queue.cc:237] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.433332 14606 catalog_manager.cc:5697] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193). New cstate: current_term: 1 leader_uuid: "21c8774363c141f7868e4fc3ec5854dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.436713 14860 raft_consensus.cc:2468] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 1.
I20260709 12:29:39.439283 14709 raft_consensus.cc:2468] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 1.
I20260709 12:29:39.447571 14606 catalog_manager.cc:5697] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:39.480221 15018 raft_consensus.cc:493] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.480715 15018 raft_consensus.cc:515] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.482338 15018 leader_election.cc:290] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.483363 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:39.483217 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:39.484153 14860 raft_consensus.cc:2393] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1a85a9ae358f49448d7287ce9a1b0f23 in current term 1: Already voted for candidate 21c8774363c141f7868e4fc3ec5854dd in this term.
I20260709 12:29:39.484999 15026 raft_consensus.cc:493] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.485097 14749 leader_election.cc:304] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23; no voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:39.485606 15026 raft_consensus.cc:515] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.486138 15018 raft_consensus.cc:3060] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.488150 15026 leader_election.cc:290] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.489240 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e8a2e7da7e423baa445ad77abe8ca0" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" is_pre_election: true
I20260709 12:29:39.489280 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e8a2e7da7e423baa445ad77abe8ca0" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:39.490253 14860 raft_consensus.cc:2468] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 0.
I20260709 12:29:39.490167 14783 raft_consensus.cc:2468] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 0.
I20260709 12:29:39.491343 14674 leader_election.cc:304] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:39.492141 15026 raft_consensus.cc:2804] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.492548 15026 raft_consensus.cc:493] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.492929 15026 raft_consensus.cc:3060] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.495015 15018 raft_consensus.cc:2749] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:39.499225 15026 raft_consensus.cc:515] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.500871 15026 leader_election.cc:290] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 election: Requested vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.501951 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e8a2e7da7e423baa445ad77abe8ca0" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:39.502080 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e8a2e7da7e423baa445ad77abe8ca0" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:39.502547 14783 raft_consensus.cc:3060] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.502686 14860 raft_consensus.cc:3060] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.508827 14783 raft_consensus.cc:2468] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 1.
I20260709 12:29:39.509229 14860 raft_consensus.cc:2468] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 1.
I20260709 12:29:39.510218 14675 leader_election.cc:304] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:39.510982 15026 raft_consensus.cc:2804] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.511478 15026 raft_consensus.cc:697] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Becoming Leader. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:39.512243 15026 consensus_queue.cc:237] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.520090 14606 catalog_manager.cc:5697] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193). New cstate: current_term: 1 leader_uuid: "21c8774363c141f7868e4fc3ec5854dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.522060 15017 consensus_queue.cc:1048] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:39.544234 14992 consensus_queue.cc:1048] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:39.567968 15018 consensus_queue.cc:1048] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:39.579913 14932 consensus_queue.cc:1048] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:39.591306 14932 raft_consensus.cc:493] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.591815 14932 raft_consensus.cc:515] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.593971 14932 leader_election.cc:290] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.594602 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c531a556c3746688ed6ee760df080ae" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:39.595165 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c531a556c3746688ed6ee760df080ae" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:39.595992 14860 raft_consensus.cc:2393] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1a85a9ae358f49448d7287ce9a1b0f23 in current term 1: Already voted for candidate 21c8774363c141f7868e4fc3ec5854dd in this term.
I20260709 12:29:39.597023 14749 leader_election.cc:304] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23; no voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:39.597970 14932 raft_consensus.cc:3060] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.604363 14992 consensus_queue.cc:1048] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:39.606374 14932 raft_consensus.cc:2749] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:39.620428 15017 consensus_queue.cc:1048] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:39.645315 14783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b3e04ad84844c8db14f8e8604c7a6c3"
dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
mode: GRACEFUL
new_leader_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:58672
I20260709 12:29:39.646148 14783 raft_consensus.cc:606] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Received request to transfer leadership to TS 21c8774363c141f7868e4fc3ec5854dd
I20260709 12:29:39.652273 14860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "756b1fe131b441af8553d5b5b3315e69"
dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
mode: GRACEFUL
new_leader_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:33830
I20260709 12:29:39.652874 14860 raft_consensus.cc:606] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Received request to transfer leadership to TS 21c8774363c141f7868e4fc3ec5854dd
I20260709 12:29:39.658900 14860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f0aead1fd17c4a36b444c90e977c77a0"
dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
mode: GRACEFUL
new_leader_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:33830
I20260709 12:29:39.659628 14860 raft_consensus.cc:606] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Received request to transfer leadership to TS 21c8774363c141f7868e4fc3ec5854dd
I20260709 12:29:39.665743 14783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "974bd7457c2e4665985fc72b109f6bb8"
dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
mode: GRACEFUL
new_leader_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:58672
I20260709 12:29:39.666301 14783 raft_consensus.cc:606] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Received request to transfer leadership to TS 21c8774363c141f7868e4fc3ec5854dd
I20260709 12:29:39.672528 14860 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b347919507c445e094f7521885abfdd2"
dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
mode: GRACEFUL
new_leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
 from {username='slave'} at 127.0.0.1:33830
I20260709 12:29:39.673120 14860 raft_consensus.cc:606] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Received request to transfer leadership to TS 1a85a9ae358f49448d7287ce9a1b0f23
I20260709 12:29:39.679190 14709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c531a556c3746688ed6ee760df080ae"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
mode: GRACEFUL
new_leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
 from {username='slave'} at 127.0.0.1:51092
I20260709 12:29:39.679715 14709 raft_consensus.cc:606] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Received request to transfer leadership to TS 1a85a9ae358f49448d7287ce9a1b0f23
I20260709 12:29:39.688851 14709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
mode: GRACEFUL
new_leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
 from {username='slave'} at 127.0.0.1:51092
I20260709 12:29:39.689373 14709 raft_consensus.cc:606] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Received request to transfer leadership to TS 1a85a9ae358f49448d7287ce9a1b0f23
I20260709 12:29:39.694396 14709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03e8a2e7da7e423baa445ad77abe8ca0"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
mode: GRACEFUL
new_leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
 from {username='slave'} at 127.0.0.1:51092
I20260709 12:29:39.694873 14709 raft_consensus.cc:606] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Received request to transfer leadership to TS 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:39.779979 14932 consensus_queue.cc:1048] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:39.787091 15024 raft_consensus.cc:993] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23: : Instructing follower 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:39.787674 15018 raft_consensus.cc:1081] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Signalling peer 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:39.791420 14711 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "974bd7457c2e4665985fc72b109f6bb8"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:51076
I20260709 12:29:39.792135 14711 raft_consensus.cc:493] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:39.792567 14711 raft_consensus.cc:3060] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.793085 15018 consensus_queue.cc:1048] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:39.803324 14711 raft_consensus.cc:515] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.806403 14711 leader_election.cc:290] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Requested vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.807240 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974bd7457c2e4665985fc72b109f6bb8" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:39.807965 14783 raft_consensus.cc:3055] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:39.808359 14783 raft_consensus.cc:740] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:39.808499 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974bd7457c2e4665985fc72b109f6bb8" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:39.809159 14860 raft_consensus.cc:3060] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.809289 14783 consensus_queue.cc:260] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.810457 14783 raft_consensus.cc:3060] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.816610 14783 raft_consensus.cc:2468] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:39.817067 14860 raft_consensus.cc:2468] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:39.817951 14675 leader_election.cc:304] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:39.819082 15026 raft_consensus.cc:2804] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:39.819648 15026 raft_consensus.cc:697] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Becoming Leader. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:39.820442 15026 consensus_queue.cc:237] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.825662 15026 consensus_queue.cc:1048] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:39.825815 14991 consensus_queue.cc:1048] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:39.831472 14608 catalog_manager.cc:5697] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd reported cstate change: term changed from 1 to 2, leader changed from 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194) to 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193). New cstate: current_term: 2 leader_uuid: "21c8774363c141f7868e4fc3ec5854dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.838626 15026 consensus_queue.cc:1048] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:39.847832 14991 raft_consensus.cc:993] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd: : Instructing follower 1a85a9ae358f49448d7287ce9a1b0f23 to start an election
I20260709 12:29:39.848393 14991 raft_consensus.cc:1081] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Signalling peer 1a85a9ae358f49448d7287ce9a1b0f23 to start an election
I20260709 12:29:39.856364 15017 consensus_queue.cc:1048] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:39.868474 14786 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18"
dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
 from {username='slave'} at 127.0.0.1:58666
I20260709 12:29:39.870000 15045 consensus_queue.cc:1048] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:39.870635 14991 raft_consensus.cc:993] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd: : Instructing follower 1a85a9ae358f49448d7287ce9a1b0f23 to start an election
I20260709 12:29:39.871837 15070 raft_consensus.cc:1081] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Signalling peer 1a85a9ae358f49448d7287ce9a1b0f23 to start an election
I20260709 12:29:39.870832 14786 raft_consensus.cc:493] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:39.872380 14786 raft_consensus.cc:3060] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.873975 14783 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c531a556c3746688ed6ee760df080ae"
dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
 from {username='slave'} at 127.0.0.1:58666
I20260709 12:29:39.874759 14783 raft_consensus.cc:493] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:39.875214 14783 raft_consensus.cc:3060] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.887627 14786 raft_consensus.cc:515] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.889736 15055 consensus_queue.cc:1048] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:39.895793 14783 raft_consensus.cc:515] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.897362 14785 raft_consensus.cc:1240] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Rejecting Update request from peer 21c8774363c141f7868e4fc3ec5854dd for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:39.898994 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c531a556c3746688ed6ee760df080ae" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:39.899806 14711 raft_consensus.cc:3055] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:39.900190 14711 raft_consensus.cc:740] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:39.901211 14711 consensus_queue.cc:260] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.904739 15068 consensus_queue.cc:1059] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:39.905879 14711 raft_consensus.cc:3060] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.911794 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c531a556c3746688ed6ee760df080ae" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:39.912557 14859 raft_consensus.cc:3060] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.913386 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:39.914105 14709 raft_consensus.cc:3055] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:39.914453 14709 raft_consensus.cc:740] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:39.915386 14709 consensus_queue.cc:260] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.916337 14709 raft_consensus.cc:3060] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.924584 14709 raft_consensus.cc:2468] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 2.
I20260709 12:29:39.925910 14750 leader_election.cc:304] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:39.926899 15018 raft_consensus.cc:2804] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:39.927474 15018 raft_consensus.cc:697] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Becoming Leader. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:39.928473 15018 consensus_queue.cc:237] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.924345 15068 consensus_queue.cc:1048] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:39.931844 14786 leader_election.cc:290] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 2 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.936192 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:39.937044 14860 raft_consensus.cc:3060] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.934301 14783 leader_election.cc:290] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 2 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.939621 15055 raft_consensus.cc:993] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: : Instructing follower 1a85a9ae358f49448d7287ce9a1b0f23 to start an election
I20260709 12:29:39.939941 14859 raft_consensus.cc:2468] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 2.
I20260709 12:29:39.940145 15055 raft_consensus.cc:1081] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Signalling peer 1a85a9ae358f49448d7287ce9a1b0f23 to start an election
I20260709 12:29:39.941146 14711 raft_consensus.cc:2468] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 2.
I20260709 12:29:39.943105 14750 leader_election.cc:304] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:39.943614 14783 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b347919507c445e094f7521885abfdd2"
dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
 from {username='slave'} at 127.0.0.1:58658
I20260709 12:29:39.943977 15018 raft_consensus.cc:2804] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:39.944200 14783 raft_consensus.cc:493] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:39.944730 14783 raft_consensus.cc:3060] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.945051 15018 raft_consensus.cc:697] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Becoming Leader. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:39.946171 15018 consensus_queue.cc:237] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.950553 15024 consensus_queue.cc:1048] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:39.958226 14860 raft_consensus.cc:2468] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 2.
I20260709 12:29:39.960100 14783 raft_consensus.cc:515] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.967963 14711 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b347919507c445e094f7521885abfdd2" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:39.968605 14711 raft_consensus.cc:3060] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.970678 14783 leader_election.cc:290] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 2 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:39.972162 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b347919507c445e094f7521885abfdd2" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:39.972962 14608 catalog_manager.cc:5697] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 reported cstate change: term changed from 1 to 2, leader changed from 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193) to 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.976075 14608 catalog_manager.cc:5697] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 reported cstate change: term changed from 1 to 2, leader changed from 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193) to 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.977427 15026 raft_consensus.cc:993] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd: : Instructing follower 8d1e4a4db8eb4df4bed6a3aa27230d50 to start an election
I20260709 12:29:39.978030 15026 raft_consensus.cc:1081] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Signalling peer 8d1e4a4db8eb4df4bed6a3aa27230d50 to start an election
I20260709 12:29:39.979166 14859 raft_consensus.cc:3055] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:39.979751 14859 raft_consensus.cc:740] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:39.980827 14859 consensus_queue.cc:260] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:39.982985 14859 raft_consensus.cc:3060] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:39.985599 14711 raft_consensus.cc:2468] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 2.
I20260709 12:29:39.987003 14750 leader_election.cc:304] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:39.987949 15018 raft_consensus.cc:2804] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:39.991922 14859 raft_consensus.cc:2468] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a85a9ae358f49448d7287ce9a1b0f23 in term 2.
I20260709 12:29:39.978011 14932 consensus_queue.cc:1048] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:39.994722 15026 consensus_queue.cc:1048] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:39.994161 14858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03e8a2e7da7e423baa445ad77abe8ca0"
dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
 from {username='slave'} at 127.0.0.1:33826
I20260709 12:29:40.002743 14858 raft_consensus.cc:493] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:40.003167 14858 raft_consensus.cc:3060] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.011739 15017 raft_consensus.cc:993] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50: : Instructing follower 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:40.012290 15017 raft_consensus.cc:1081] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Signalling peer 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:40.012029 14858 raft_consensus.cc:515] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:40.014494 14858 leader_election.cc:290] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 2 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:40.014683 14709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "756b1fe131b441af8553d5b5b3315e69"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:51080
I20260709 12:29:40.015417 14709 raft_consensus.cc:493] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:40.015761 14709 raft_consensus.cc:3060] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.017815 14784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e8a2e7da7e423baa445ad77abe8ca0" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:40.020007 14710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e8a2e7da7e423baa445ad77abe8ca0" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:40.020596 14710 raft_consensus.cc:3055] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:40.020879 14710 raft_consensus.cc:740] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:40.021704 14710 consensus_queue.cc:260] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:40.022859 14710 raft_consensus.cc:3060] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.023190 15018 raft_consensus.cc:697] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Becoming Leader. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:40.024590 15018 consensus_queue.cc:237] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:40.030575 14710 raft_consensus.cc:2468] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 2.
I20260709 12:29:40.032083 14824 leader_election.cc:304] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 1a85a9ae358f49448d7287ce9a1b0f23
I20260709 12:29:40.023207 14709 raft_consensus.cc:515] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:40.033440 15017 raft_consensus.cc:2804] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:40.034030 15017 raft_consensus.cc:697] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 LEADER]: Becoming Leader. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:40.035048 14709 leader_election.cc:290] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Requested vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:40.035151 15017 consensus_queue.cc:237] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:40.036626 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756b1fe131b441af8553d5b5b3315e69" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:40.037153 14783 raft_consensus.cc:3060] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.037675 14858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756b1fe131b441af8553d5b5b3315e69" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:40.042850 14858 raft_consensus.cc:3055] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:40.043208 14858 raft_consensus.cc:740] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:40.044040 14858 consensus_queue.cc:260] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:40.045356 14858 raft_consensus.cc:3060] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.045660 14608 catalog_manager.cc:5697] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 reported cstate change: term changed from 1 to 2, leader changed from 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195) to 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194). New cstate: current_term: 2 leader_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:40.050072 14783 raft_consensus.cc:2468] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:40.051669 14675 leader_election.cc:304] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:40.052759 15026 raft_consensus.cc:2804] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:40.053229 15026 raft_consensus.cc:697] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Becoming Leader. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:40.054097 15026 consensus_queue.cc:237] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:40.054526 14858 raft_consensus.cc:2468] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:40.060066 14609 catalog_manager.cc:5697] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 reported cstate change: term changed from 1 to 2, leader changed from 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193) to 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:40.067953 14606 catalog_manager.cc:5697] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd reported cstate change: term changed from 1 to 2, leader changed from 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195) to 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193). New cstate: current_term: 2 leader_uuid: "21c8774363c141f7868e4fc3ec5854dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:40.091225 15074 raft_consensus.cc:993] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: : Instructing follower 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:40.091776 15017 raft_consensus.cc:1081] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Signalling peer 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:40.093322 14710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f0aead1fd17c4a36b444c90e977c77a0"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:51080
I20260709 12:29:40.093847 14710 raft_consensus.cc:493] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:40.094182 14710 raft_consensus.cc:3060] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.099720 14710 raft_consensus.cc:515] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:40.102298 14784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0aead1fd17c4a36b444c90e977c77a0" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:40.102880 14858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0aead1fd17c4a36b444c90e977c77a0" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:40.102921 14784 raft_consensus.cc:3060] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.102526 14710 leader_election.cc:290] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Requested vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:40.103927 14858 raft_consensus.cc:3055] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:40.104377 14858 raft_consensus.cc:740] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:40.105326 14858 consensus_queue.cc:260] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:40.106460 14858 raft_consensus.cc:3060] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.110251 14784 raft_consensus.cc:2468] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:40.111325 14675 leader_election.cc:304] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 21c8774363c141f7868e4fc3ec5854dd; no voters: 
I20260709 12:29:40.112151 15026 raft_consensus.cc:2804] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:40.112690 15026 raft_consensus.cc:697] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Becoming Leader. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:40.113176 14858 raft_consensus.cc:2468] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:40.113718 15026 consensus_queue.cc:237] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:40.120954 14606 catalog_manager.cc:5697] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd reported cstate change: term changed from 1 to 2, leader changed from 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195) to 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193). New cstate: current_term: 2 leader_uuid: "21c8774363c141f7868e4fc3ec5854dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:40.151790 14932 raft_consensus.cc:993] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23: : Instructing follower 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:40.152570 15078 raft_consensus.cc:1081] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Signalling peer 21c8774363c141f7868e4fc3ec5854dd to start an election
I20260709 12:29:40.154143 14710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b3e04ad84844c8db14f8e8604c7a6c3"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
 from {username='slave'} at 127.0.0.1:51076
I20260709 12:29:40.154662 14710 raft_consensus.cc:493] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:40.155061 14710 raft_consensus.cc:3060] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.160339 14710 raft_consensus.cc:515] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:40.162185 14710 leader_election.cc:290] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Requested vote from peers 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:40.163363 14859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b3e04ad84844c8db14f8e8604c7a6c3" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
I20260709 12:29:40.163395 14784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b3e04ad84844c8db14f8e8604c7a6c3" candidate_uuid: "21c8774363c141f7868e4fc3ec5854dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:40.163986 14859 raft_consensus.cc:3060] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.164163 14784 raft_consensus.cc:3055] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:40.164565 14784 raft_consensus.cc:740] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:40.165277 14784 consensus_queue.cc:260] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:40.166323 14784 raft_consensus.cc:3060] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.169524 14859 raft_consensus.cc:2468] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:40.170866 14674 leader_election.cc:304] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:40.171617 15026 raft_consensus.cc:2804] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:40.171732 14784 raft_consensus.cc:2468] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c8774363c141f7868e4fc3ec5854dd in term 2.
I20260709 12:29:40.172055 15026 raft_consensus.cc:697] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Becoming Leader. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:40.173007 15026 consensus_queue.cc:237] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:40.179666 14606 catalog_manager.cc:5697] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd reported cstate change: term changed from 1 to 2, leader changed from 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194) to 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193). New cstate: current_term: 2 leader_uuid: "21c8774363c141f7868e4fc3ec5854dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:40.258538 14784 raft_consensus.cc:1275] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.260031 15069 consensus_queue.cc:1048] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.269560 14859 raft_consensus.cc:1275] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.271298 15071 consensus_queue.cc:1048] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:40.392247 14710 raft_consensus.cc:1275] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Refusing update from remote peer 1a85a9ae358f49448d7287ce9a1b0f23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.393638 15078 consensus_queue.cc:1048] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.402990 14859 raft_consensus.cc:1275] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Refusing update from remote peer 1a85a9ae358f49448d7287ce9a1b0f23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.404433 15078 consensus_queue.cc:1048] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.426878 14710 raft_consensus.cc:1275] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Refusing update from remote peer 1a85a9ae358f49448d7287ce9a1b0f23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.428148 15078 consensus_queue.cc:1048] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.437820 14859 raft_consensus.cc:1275] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Refusing update from remote peer 1a85a9ae358f49448d7287ce9a1b0f23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.439167 15078 consensus_queue.cc:1048] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.442245 14710 raft_consensus.cc:1275] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Refusing update from remote peer 8d1e4a4db8eb4df4bed6a3aa27230d50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.443918 15055 consensus_queue.cc:1048] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.456514 14784 raft_consensus.cc:3060] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:40.464121 14784 raft_consensus.cc:1275] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Refusing update from remote peer 8d1e4a4db8eb4df4bed6a3aa27230d50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.464310 14783 raft_consensus.cc:1275] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.465932 15017 consensus_queue.cc:1048] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.466432 15026 consensus_queue.cc:1048] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.479172 14859 raft_consensus.cc:1275] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.480793 15069 consensus_queue.cc:1048] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.503909 14710 raft_consensus.cc:1275] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Refusing update from remote peer 1a85a9ae358f49448d7287ce9a1b0f23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.505019 15078 consensus_queue.cc:1048] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.512872 14859 raft_consensus.cc:1275] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Refusing update from remote peer 1a85a9ae358f49448d7287ce9a1b0f23: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.515829 15078 consensus_queue.cc:1048] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.609673 14859 raft_consensus.cc:1275] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.611088 15071 consensus_queue.cc:1048] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.620973 14784 raft_consensus.cc:1275] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.623677 15071 consensus_queue.cc:1048] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.679705 14784 raft_consensus.cc:1275] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.681285 15071 consensus_queue.cc:1048] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:40.692059 14858 raft_consensus.cc:1275] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Refusing update from remote peer 21c8774363c141f7868e4fc3ec5854dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:40.693498 15026 consensus_queue.cc:1048] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:42.698275 14608 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40446:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:29:42.719901 14608 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:40446:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:29:42.751206  3431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:42.945235 15123 tablet.cc:1724] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:42.993290 15097 log.cc:927] Time spent T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:29:42.987941 15138 tablet.cc:1724] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:42.973171 15124 tablet.cc:1724] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:43.008126 15122 log.cc:927] Time spent T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 12:29:43.030387 14750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:43267, user_credentials={real_user=slave}} blocked reactor thread for 123181us
W20260709 12:29:43.032289 14675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:35477, user_credentials={real_user=slave}} blocked reactor thread for 81573.2us
I20260709 12:29:43.070554 15142 tablet.cc:1724] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:29:43.073279 14812 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:4c531a556c3746688ed6ee760df080ae)
W20260709 12:29:43.093418 14749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.195:35339, user_credentials={real_user=slave}} blocked reactor thread for 47081.9us
W20260709 12:29:43.092923 14812 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:4c531a556c3746688ed6ee760df080ae)
W20260709 12:29:43.125065 14812 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:4c531a556c3746688ed6ee760df080ae)
W20260709 12:29:43.174229 14675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:35477, user_credentials={real_user=slave}} blocked reactor thread for 121273us
I20260709 12:29:43.141181 15150 tablet.cc:1724] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.170377 15155 tablet.cc:1724] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.177070 15149 tablet.cc:1724] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.153554 15152 tablet.cc:1724] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.214010 14783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18"
dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
mode: GRACEFUL
new_leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
 from {username='slave'} at 127.0.0.1:42038
I20260709 12:29:43.214717 14783 raft_consensus.cc:606] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Received request to transfer leadership to TS 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:43.219172 15160 tablet.cc:1724] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.200656 15138 tablet.cc:1724] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.261315 15152 tablet.cc:1724] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.259443 15150 tablet.cc:1724] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.264046 15142 tablet.cc:1724] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.266639 15161 tablet.cc:1724] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.273452 15139 raft_consensus.cc:993] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23: : Instructing follower 8d1e4a4db8eb4df4bed6a3aa27230d50 to start an election
I20260709 12:29:43.274080 15139 raft_consensus.cc:1081] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Signalling peer 8d1e4a4db8eb4df4bed6a3aa27230d50 to start an election
I20260709 12:29:43.276137 14858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18"
dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
 from {username='slave'} at 127.0.0.1:33822
I20260709 12:29:43.276753 14858 raft_consensus.cc:493] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:43.277163 14858 raft_consensus.cc:3060] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:43.279424 15163 tablet.cc:1724] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.281736 15123 tablet.cc:1724] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.278342 15164 tablet.cc:1724] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.307209 15149 tablet.cc:1724] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.314643 15161 tablet.cc:1724] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.311483 15124 tablet.cc:1724] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.342105 15138 tablet.cc:1724] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.341948 15163 tablet.cc:1724] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.348546 15123 tablet.cc:1724] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.358117 15155 tablet.cc:1724] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.350242 15160 tablet.cc:1724] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.361778 14709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "974bd7457c2e4665985fc72b109f6bb8"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
mode: GRACEFUL
new_leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
 from {username='slave'} at 127.0.0.1:40746
I20260709 12:29:43.362453 14709 raft_consensus.cc:606] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Received request to transfer leadership to TS 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:43.364041  3431 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:29:43.364751  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:43.361114 15150 tablet.cc:1724] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.372848 15152 tablet.cc:1724] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:29:43.388070 14709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b"
dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
mode: GRACEFUL
new_leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
 from {username='slave'} at 127.0.0.1:40746
I20260709 12:29:43.388684 14709 raft_consensus.cc:606] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Received request to transfer leadership to TS 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:43.391343 14858 raft_consensus.cc:515] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:43.398474 14783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3b45722450c4ab48fae4b4faa88cfdd"
dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
mode: GRACEFUL
new_leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
 from {username='slave'} at 127.0.0.1:42038
I20260709 12:29:43.399253 14783 raft_consensus.cc:606] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Received request to transfer leadership to TS 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:43.401808  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:29:43.407513  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:43.409461 14858 leader_election.cc:290] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 3 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:43.411079 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:43.411787 14783 raft_consensus.cc:3055] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:43.412106 14783 raft_consensus.cc:740] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:43.412891 14783 consensus_queue.cc:260] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:43.414238 14783 raft_consensus.cc:3060] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:43.418057 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29e2aafd2afb4ac394d856a620b50c18" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:43.418766 14709 raft_consensus.cc:3060] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:43.426518 14783 raft_consensus.cc:2468] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 3.
I20260709 12:29:43.427220 14709 raft_consensus.cc:2468] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 3.
I20260709 12:29:43.428714 14824 leader_election.cc:304] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:43.430419 15083 raft_consensus.cc:2804] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:43.430970 15083 raft_consensus.cc:697] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 LEADER]: Becoming Leader. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:43.432226 15083 consensus_queue.cc:237] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:43.444725 14608 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822: soft_deleted_table [id=881ce0ef64774dab8291c49c0c1be7a0] alter complete (version 1)
I20260709 12:29:43.454221 14606 catalog_manager.cc:5697] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 reported cstate change: term changed from 2 to 3, leader changed from 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194) to 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:43.557226 15162 raft_consensus.cc:993] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd: : Instructing follower 8d1e4a4db8eb4df4bed6a3aa27230d50 to start an election
I20260709 12:29:43.557868 15129 raft_consensus.cc:1081] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Signalling peer 8d1e4a4db8eb4df4bed6a3aa27230d50 to start an election
I20260709 12:29:43.559592 14860 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "974bd7457c2e4665985fc72b109f6bb8"
dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
 from {username='slave'} at 127.0.0.1:33826
I20260709 12:29:43.560174 14860 raft_consensus.cc:493] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:43.560422 14860 raft_consensus.cc:3060] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:43.565454 14860 raft_consensus.cc:515] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:43.567196 14860 leader_election.cc:290] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 3 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:43.568056 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974bd7457c2e4665985fc72b109f6bb8" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:43.568311 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "974bd7457c2e4665985fc72b109f6bb8" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:43.568722 14709 raft_consensus.cc:3055] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:43.568813 14783 raft_consensus.cc:3060] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:43.569090 14709 raft_consensus.cc:740] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:43.569926 14709 consensus_queue.cc:260] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:43.571081 14709 raft_consensus.cc:3060] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:43.576073 14783 raft_consensus.cc:2468] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 3.
I20260709 12:29:43.577415 14824 leader_election.cc:304] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:43.577888 14709 raft_consensus.cc:2468] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 3.
I20260709 12:29:43.578336 15083 raft_consensus.cc:2804] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:43.578913 15083 raft_consensus.cc:697] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 LEADER]: Becoming Leader. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:43.579753 15083 consensus_queue.cc:237] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:43.587209 14606 catalog_manager.cc:5697] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 reported cstate change: term changed from 2 to 3, leader changed from 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193) to 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:43.609782 15137 raft_consensus.cc:993] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23: : Instructing follower 8d1e4a4db8eb4df4bed6a3aa27230d50 to start an election
I20260709 12:29:43.610450 15153 raft_consensus.cc:1081] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Signalling peer 8d1e4a4db8eb4df4bed6a3aa27230d50 to start an election
I20260709 12:29:43.612174 14860 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3b45722450c4ab48fae4b4faa88cfdd"
dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
 from {username='slave'} at 127.0.0.1:33822
I20260709 12:29:43.612723 14860 raft_consensus.cc:493] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:43.613008 14860 raft_consensus.cc:3060] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:43.618309 14860 raft_consensus.cc:515] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:43.619943 14860 leader_election.cc:290] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 3 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:43.620901 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b45722450c4ab48fae4b4faa88cfdd" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:43.621165 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3b45722450c4ab48fae4b4faa88cfdd" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:43.621636 14709 raft_consensus.cc:3060] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:43.621930 14783 raft_consensus.cc:3055] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:43.622044 15162 raft_consensus.cc:993] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd: : Instructing follower 8d1e4a4db8eb4df4bed6a3aa27230d50 to start an election
I20260709 12:29:43.622323 14783 raft_consensus.cc:740] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1a85a9ae358f49448d7287ce9a1b0f23, State: Running, Role: LEADER
I20260709 12:29:43.622735 15129 raft_consensus.cc:1081] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Signalling peer 8d1e4a4db8eb4df4bed6a3aa27230d50 to start an election
I20260709 12:29:43.623198 14783 consensus_queue.cc:260] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:43.624420 14783 raft_consensus.cc:3060] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:43.624689 14860 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b"
dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50"
 from {username='slave'} at 127.0.0.1:33826
I20260709 12:29:43.625291 14860 raft_consensus.cc:493] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:43.625715 14860 raft_consensus.cc:3060] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:43.628509 14709 raft_consensus.cc:2468] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 3.
I20260709 12:29:43.629935 14824 leader_election.cc:304] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:43.630784 15083 raft_consensus.cc:2804] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:43.631305 15083 raft_consensus.cc:697] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 LEADER]: Becoming Leader. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:43.632238 15083 consensus_queue.cc:237] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:43.632817 14783 raft_consensus.cc:2468] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 3.
I20260709 12:29:43.633086 14860 raft_consensus.cc:515] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:43.635344 14860 leader_election.cc:290] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 3 election: Requested vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477)
I20260709 12:29:43.636628 14783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1a85a9ae358f49448d7287ce9a1b0f23"
I20260709 12:29:43.636802 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af2477a6255446e89e4ee4848bea759b" candidate_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "21c8774363c141f7868e4fc3ec5854dd"
I20260709 12:29:43.637253 14783 raft_consensus.cc:3060] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:43.637727 14709 raft_consensus.cc:3055] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:43.638137 14709 raft_consensus.cc:740] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 21c8774363c141f7868e4fc3ec5854dd, State: Running, Role: LEADER
I20260709 12:29:43.638870 14709 consensus_queue.cc:260] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:43.640179 14709 raft_consensus.cc:3060] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:43.644646 14783 raft_consensus.cc:2468] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 3.
I20260709 12:29:43.643884 14606 catalog_manager.cc:5697] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 reported cstate change: term changed from 2 to 3, leader changed from 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194) to 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:43.647570 14709 raft_consensus.cc:2468] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1e4a4db8eb4df4bed6a3aa27230d50 in term 3.
I20260709 12:29:43.648543 14824 leader_election.cc:304] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23, 8d1e4a4db8eb4df4bed6a3aa27230d50; no voters: 
I20260709 12:29:43.649622 15083 raft_consensus.cc:2804] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:43.650100 15083 raft_consensus.cc:697] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 LEADER]: Becoming Leader. State: Replica: 8d1e4a4db8eb4df4bed6a3aa27230d50, State: Running, Role: LEADER
I20260709 12:29:43.650827 15083 consensus_queue.cc:237] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:43.660663 14606 catalog_manager.cc:5697] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 reported cstate change: term changed from 2 to 3, leader changed from 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193) to 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:43.863683 14783 raft_consensus.cc:1275] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Refusing update from remote peer 8d1e4a4db8eb4df4bed6a3aa27230d50: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:29:43.864979 15083 consensus_queue.cc:1048] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:43.874102 14709 raft_consensus.cc:1275] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Refusing update from remote peer 8d1e4a4db8eb4df4bed6a3aa27230d50: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:29:43.877266 15173 consensus_queue.cc:1048] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:29:43.965597 14709 raft_consensus.cc:1275] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Refusing update from remote peer 8d1e4a4db8eb4df4bed6a3aa27230d50: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:29:43.966934 15173 consensus_queue.cc:1048] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:43.976720 14783 raft_consensus.cc:1275] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Refusing update from remote peer 8d1e4a4db8eb4df4bed6a3aa27230d50: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:29:43.978278 15173 consensus_queue.cc:1048] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:44.050549 14709 raft_consensus.cc:1275] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Refusing update from remote peer 8d1e4a4db8eb4df4bed6a3aa27230d50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:44.052016 15173 consensus_queue.cc:1048] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:44.065815 14783 raft_consensus.cc:1275] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Refusing update from remote peer 8d1e4a4db8eb4df4bed6a3aa27230d50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:44.067579 15177 consensus_queue.cc:1048] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:44.118108 14709 raft_consensus.cc:1275] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Refusing update from remote peer 8d1e4a4db8eb4df4bed6a3aa27230d50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:44.119485 15177 consensus_queue.cc:1048] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:44.129312 14783 raft_consensus.cc:1275] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Refusing update from remote peer 8d1e4a4db8eb4df4bed6a3aa27230d50: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:44.130957 15083 consensus_queue.cc:1048] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:44.408722  3431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:44.414587  3431 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:44.415062  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:44.421116  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:44.421619  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:45.422663  3431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:45.428532  3431 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:45.429088  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:45.434103  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:45.434562  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:46.435577  3431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:46.441176  3431 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:46.441733  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:46.447217  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:46.447638  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:47.450820  3431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:47.464727  3431 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:47.465200  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:47.474504  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:47.475059  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:48.476465  3431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:48.482153  3431 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:48.482867  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:48.487761  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:48.488211  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:49.489162  3431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:49.494140  3431 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:49.494660  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:49.499372  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:49.499804  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:50.825726  3431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:50.835592  3431 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:50.836421  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:51.159238  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:51.159847  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:51.492084 15202 raft_consensus.cc:493] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 21c8774363c141f7868e4fc3ec5854dd)
I20260709 12:29:51.497530 15202 raft_consensus.cc:515] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21c8774363c141f7868e4fc3ec5854dd" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43267 } } peers { permanent_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 35477 } } peers { permanent_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 35339 } }
I20260709 12:29:51.503315 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756b1fe131b441af8553d5b5b3315e69" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8d1e4a4db8eb4df4bed6a3aa27230d50" is_pre_election: true
I20260709 12:29:51.505800 15202 leader_election.cc:290] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267), 8d1e4a4db8eb4df4bed6a3aa27230d50 (127.3.89.195:35339)
I20260709 12:29:51.514708 14709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "756b1fe131b441af8553d5b5b3315e69" candidate_uuid: "1a85a9ae358f49448d7287ce9a1b0f23" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "21c8774363c141f7868e4fc3ec5854dd" is_pre_election: true
I20260709 12:29:51.516687 14750 leader_election.cc:304] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a85a9ae358f49448d7287ce9a1b0f23; no voters: 21c8774363c141f7868e4fc3ec5854dd, 8d1e4a4db8eb4df4bed6a3aa27230d50
I20260709 12:29:51.517774 15202 raft_consensus.cc:2749] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:29:51.823138 14732 debug-util.cc:398] Leaking SignalData structure 0x7b08001ea1e0 after lost signal to thread 3434
W20260709 12:29:51.824280 14732 debug-util.cc:398] Leaking SignalData structure 0x7b08002aafe0 after lost signal to thread 14643
W20260709 12:29:51.825377 14732 debug-util.cc:398] Leaking SignalData structure 0x7b0800304de0 after lost signal to thread 14735
W20260709 12:29:51.826583 14732 debug-util.cc:398] Leaking SignalData structure 0x7b080004cb60 after lost signal to thread 14810
W20260709 12:29:51.827579 14732 debug-util.cc:398] Leaking SignalData structure 0x7b08000321a0 after lost signal to thread 14884
I20260709 12:29:52.161134  3431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:52.172132  3431 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:52.172892  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:52.186815  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:52.187644  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:53.188915  3431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:29:53.194495  3431 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:29:53.195115  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:53.202826  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:53.203334  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:54.270112  3431 tablet_server.cc:179] TabletServer@127.3.89.193:0 shutting down...
W20260709 12:29:54.299398 14750 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260709 12:29:54.304909 14750 consensus_peers.cc:597] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 -> Peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Couldn't send request to peer 21c8774363c141f7868e4fc3ec5854dd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:54.322397  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:54.323745  3431 tablet_replica.cc:333] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.324906  3431 raft_consensus.cc:2243] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.325904  3431 raft_consensus.cc:2272] T 03e8a2e7da7e423baa445ad77abe8ca0 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.329710  3431 tablet_replica.cc:333] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.330443  3431 raft_consensus.cc:2243] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.331163  3431 raft_consensus.cc:2272] T e044c801fa47414e82f7ce946e0ed749 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.334158  3431 tablet_replica.cc:333] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.334749  3431 raft_consensus.cc:2243] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:54.335939  3431 raft_consensus.cc:2272] T f0aead1fd17c4a36b444c90e977c77a0 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.339640  3431 tablet_replica.cc:333] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.340583  3431 raft_consensus.cc:2243] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.341547  3431 raft_consensus.cc:2272] T b347919507c445e094f7521885abfdd2 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:54.342060 14750 consensus_peers.cc:597] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 -> Peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Couldn't send request to peer 21c8774363c141f7868e4fc3ec5854dd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:54.345173  3431 tablet_replica.cc:333] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.346673  3431 raft_consensus.cc:2243] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.347824  3431 raft_consensus.cc:2272] T 544e89b953424226ae1c78c218552c43 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.351225  3431 tablet_replica.cc:333] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.351953  3431 raft_consensus.cc:2243] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.352579  3431 raft_consensus.cc:2272] T b3b45722450c4ab48fae4b4faa88cfdd P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.356099  3431 tablet_replica.cc:333] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.356824  3431 raft_consensus.cc:2243] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.357435  3431 raft_consensus.cc:2272] T 4350e9f3eba145d89b0f53790e46cc50 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.360394  3431 tablet_replica.cc:333] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.361135  3431 raft_consensus.cc:2243] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:54.362282  3431 raft_consensus.cc:2272] T d168e6a345a94901a3dac027a767123e P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.365291  3431 tablet_replica.cc:333] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.365978  3431 raft_consensus.cc:2243] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:54.367000  3431 raft_consensus.cc:2272] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.369884  3431 tablet_replica.cc:333] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.370656  3431 raft_consensus.cc:2243] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.371225  3431 raft_consensus.cc:2272] T 29e2aafd2afb4ac394d856a620b50c18 P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.374074  3431 tablet_replica.cc:333] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.374665  3431 raft_consensus.cc:2243] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:54.375763  3431 raft_consensus.cc:2272] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.378801  3431 tablet_replica.cc:333] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.379405  3431 raft_consensus.cc:2243] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.380064  3431 raft_consensus.cc:2272] T 4c531a556c3746688ed6ee760df080ae P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.382786  3431 tablet_replica.cc:333] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.383383  3431 raft_consensus.cc:2243] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.383934  3431 raft_consensus.cc:2272] T af2477a6255446e89e4ee4848bea759b P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.386729  3431 tablet_replica.cc:333] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.387370  3431 raft_consensus.cc:2243] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:54.388360  3431 raft_consensus.cc:2272] T 756b1fe131b441af8553d5b5b3315e69 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.391865  3431 tablet_replica.cc:333] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.392851  3431 raft_consensus.cc:2243] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.393898  3431 raft_consensus.cc:2272] T 2859ea67269e481f83cebadbe397b8a2 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:54.396473 14824 consensus_peers.cc:597] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 -> Peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Couldn't send request to peer 21c8774363c141f7868e4fc3ec5854dd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:54.397889  3431 tablet_replica.cc:333] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.398856  3431 raft_consensus.cc:2243] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:54.400131  3431 raft_consensus.cc:2272] T 3928359ec0c2401e98c97cb66daa7126 P 21c8774363c141f7868e4fc3ec5854dd [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:54.401750 14750 consensus_peers.cc:597] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 -> Peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Couldn't send request to peer 21c8774363c141f7868e4fc3ec5854dd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:54.403438  3431 tablet_replica.cc:333] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.404111  3431 raft_consensus.cc:2243] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.404706  3431 raft_consensus.cc:2272] T 974bd7457c2e4665985fc72b109f6bb8 P 21c8774363c141f7868e4fc3ec5854dd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:54.407792  3431 tablet_replica.cc:333] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd: stopping tablet replica
I20260709 12:29:54.408327  3431 raft_consensus.cc:2243] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:54.408870  3431 raft_consensus.cc:2272] T f63cdd55867645ababeebf041ec58a26 P 21c8774363c141f7868e4fc3ec5854dd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:54.457741 14824 consensus_peers.cc:597] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 -> Peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Couldn't send request to peer 21c8774363c141f7868e4fc3ec5854dd. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:43267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:54.460884 14750 consensus_peers.cc:597] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 -> Peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Couldn't send request to peer 21c8774363c141f7868e4fc3ec5854dd. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:43267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:54.572841 14750 consensus_peers.cc:597] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 -> Peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Couldn't send request to peer 21c8774363c141f7868e4fc3ec5854dd. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:43267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:54.578375 14824 consensus_peers.cc:597] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 -> Peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Couldn't send request to peer 21c8774363c141f7868e4fc3ec5854dd. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:43267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:54.612236 14824 consensus_peers.cc:597] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 -> Peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Couldn't send request to peer 21c8774363c141f7868e4fc3ec5854dd. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:43267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:54.636456 14750 consensus_peers.cc:597] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 -> Peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Couldn't send request to peer 21c8774363c141f7868e4fc3ec5854dd. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:43267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:54.734949 14824 consensus_peers.cc:597] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 -> Peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Couldn't send request to peer 21c8774363c141f7868e4fc3ec5854dd. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:43267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:54.762900 14824 consensus_peers.cc:597] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 -> Peer 21c8774363c141f7868e4fc3ec5854dd (127.3.89.193:43267): Couldn't send request to peer 21c8774363c141f7868e4fc3ec5854dd. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:43267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:55.024544  3431 tablet_server.cc:196] TabletServer@127.3.89.193:0 shutdown complete.
I20260709 12:29:55.081787  3431 tablet_server.cc:179] TabletServer@127.3.89.194:0 shutting down...
W20260709 12:29:55.102131 14824 consensus_peers.cc:597] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 -> Peer 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477): Couldn't send request to peer 1a85a9ae358f49448d7287ce9a1b0f23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:55.115947  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:55.116786  3431 tablet_replica.cc:333] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.117679  3431 raft_consensus.cc:2243] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.118290  3431 raft_consensus.cc:2272] T 03e8a2e7da7e423baa445ad77abe8ca0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.120692  3431 tablet_replica.cc:333] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.121351  3431 raft_consensus.cc:2243] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:55.122289  3431 raft_consensus.cc:2272] T e044c801fa47414e82f7ce946e0ed749 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.124569  3431 tablet_replica.cc:333] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.125228  3431 raft_consensus.cc:2243] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:55.126508  3431 raft_consensus.cc:2272] T b347919507c445e094f7521885abfdd2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.129244  3431 tablet_replica.cc:333] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.129830  3431 raft_consensus.cc:2243] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.130295  3431 raft_consensus.cc:2272] T f0aead1fd17c4a36b444c90e977c77a0 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.132431  3431 tablet_replica.cc:333] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.132942  3431 raft_consensus.cc:2243] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.133358  3431 raft_consensus.cc:2272] T b3b45722450c4ab48fae4b4faa88cfdd P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.135522  3431 tablet_replica.cc:333] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.136144  3431 raft_consensus.cc:2243] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.136566  3431 raft_consensus.cc:2272] T 544e89b953424226ae1c78c218552c43 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.139048  3431 tablet_replica.cc:333] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.139704  3431 raft_consensus.cc:2243] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.140326  3431 raft_consensus.cc:2272] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:55.141835 14824 consensus_peers.cc:597] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 -> Peer 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477): Couldn't send request to peer 1a85a9ae358f49448d7287ce9a1b0f23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:55.142851  3431 tablet_replica.cc:333] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.143443  3431 raft_consensus.cc:2243] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.143886  3431 raft_consensus.cc:2272] T d168e6a345a94901a3dac027a767123e P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.146462  3431 tablet_replica.cc:333] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.147147  3431 raft_consensus.cc:2243] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.147658  3431 raft_consensus.cc:2272] T 974bd7457c2e4665985fc72b109f6bb8 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.150331  3431 tablet_replica.cc:333] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.150862  3431 raft_consensus.cc:2243] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:55.151624  3431 raft_consensus.cc:2272] T f63cdd55867645ababeebf041ec58a26 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.153555  3431 tablet_replica.cc:333] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.154187  3431 raft_consensus.cc:2243] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:55.154934  3431 raft_consensus.cc:2272] T 4350e9f3eba145d89b0f53790e46cc50 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.157099  3431 tablet_replica.cc:333] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.157788  3431 raft_consensus.cc:2243] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.158250  3431 raft_consensus.cc:2272] T 756b1fe131b441af8553d5b5b3315e69 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.160390  3431 tablet_replica.cc:333] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.160890  3431 raft_consensus.cc:2243] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:55.161661  3431 raft_consensus.cc:2272] T 2859ea67269e481f83cebadbe397b8a2 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.164170  3431 tablet_replica.cc:333] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.164659  3431 raft_consensus.cc:2243] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.165060  3431 raft_consensus.cc:2272] T 3928359ec0c2401e98c97cb66daa7126 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.167224  3431 tablet_replica.cc:333] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.167695  3431 raft_consensus.cc:2243] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.168160  3431 raft_consensus.cc:2272] T 29e2aafd2afb4ac394d856a620b50c18 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.170296  3431 tablet_replica.cc:333] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.170780  3431 raft_consensus.cc:2243] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.171269  3431 raft_consensus.cc:2272] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.173446  3431 tablet_replica.cc:333] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.173902  3431 raft_consensus.cc:2243] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:55.174710  3431 raft_consensus.cc:2272] T 4c531a556c3746688ed6ee760df080ae P 1a85a9ae358f49448d7287ce9a1b0f23 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.176738  3431 tablet_replica.cc:333] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23: stopping tablet replica
I20260709 12:29:55.177178  3431 raft_consensus.cc:2243] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.177671  3431 raft_consensus.cc:2272] T af2477a6255446e89e4ee4848bea759b P 1a85a9ae358f49448d7287ce9a1b0f23 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.221871  3431 tablet_server.cc:196] TabletServer@127.3.89.194:0 shutdown complete.
I20260709 12:29:55.268617  3431 tablet_server.cc:179] TabletServer@127.3.89.195:0 shutting down...
I20260709 12:29:55.301373  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:55.302260  3431 tablet_replica.cc:333] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.303301  3431 raft_consensus.cc:2243] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:55.304468  3431 raft_consensus.cc:2272] T 03e8a2e7da7e423baa445ad77abe8ca0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.306895  3431 tablet_replica.cc:333] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.307451  3431 raft_consensus.cc:2243] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.307863  3431 raft_consensus.cc:2272] T e044c801fa47414e82f7ce946e0ed749 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.310089  3431 tablet_replica.cc:333] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.311071  3431 raft_consensus.cc:2243] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.311808  3431 raft_consensus.cc:2272] T b347919507c445e094f7521885abfdd2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.315056  3431 tablet_replica.cc:333] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
W20260709 12:29:55.315889 14824 consensus_peers.cc:597] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 -> Peer 1a85a9ae358f49448d7287ce9a1b0f23 (127.3.89.194:35477): Couldn't send request to peer 1a85a9ae358f49448d7287ce9a1b0f23. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:35477: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:55.316025  3431 raft_consensus.cc:2243] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:55.316970  3431 raft_consensus.cc:2272] T 29e2aafd2afb4ac394d856a620b50c18 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.319236  3431 tablet_replica.cc:333] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.319900  3431 raft_consensus.cc:2243] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.320420  3431 raft_consensus.cc:2272] T 5b3e04ad84844c8db14f8e8604c7a6c3 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.322608  3431 tablet_replica.cc:333] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.323167  3431 raft_consensus.cc:2243] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.323701  3431 raft_consensus.cc:2272] T f0aead1fd17c4a36b444c90e977c77a0 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.325911  3431 tablet_replica.cc:333] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.326545  3431 raft_consensus.cc:2243] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:55.327395  3431 raft_consensus.cc:2272] T 544e89b953424226ae1c78c218552c43 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.329739  3431 tablet_replica.cc:333] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.330327  3431 raft_consensus.cc:2243] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:55.331182  3431 raft_consensus.cc:2272] T b3b45722450c4ab48fae4b4faa88cfdd P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.333793  3431 tablet_replica.cc:333] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.334452  3431 raft_consensus.cc:2243] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.334993  3431 raft_consensus.cc:2272] T 170e6f1f2be94ad1afa5b0ab4c556d87 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.337406  3431 tablet_replica.cc:333] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.338023  3431 raft_consensus.cc:2243] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.338613  3431 raft_consensus.cc:2272] T d168e6a345a94901a3dac027a767123e P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.341075  3431 tablet_replica.cc:333] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.341753  3431 raft_consensus.cc:2243] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.342262  3431 raft_consensus.cc:2272] T f63cdd55867645ababeebf041ec58a26 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.344601  3431 tablet_replica.cc:333] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.345270  3431 raft_consensus.cc:2243] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:55.346241  3431 raft_consensus.cc:2272] T 974bd7457c2e4665985fc72b109f6bb8 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.348608  3431 tablet_replica.cc:333] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.349260  3431 raft_consensus.cc:2243] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.349874  3431 raft_consensus.cc:2272] T 4c531a556c3746688ed6ee760df080ae P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.352222  3431 tablet_replica.cc:333] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.352809  3431 raft_consensus.cc:2243] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:55.353762  3431 raft_consensus.cc:2272] T af2477a6255446e89e4ee4848bea759b P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.356058  3431 tablet_replica.cc:333] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.356657  3431 raft_consensus.cc:2243] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.357282  3431 raft_consensus.cc:2272] T 756b1fe131b441af8553d5b5b3315e69 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.359652  3431 tablet_replica.cc:333] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.360237  3431 raft_consensus.cc:2243] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.360739  3431 raft_consensus.cc:2272] T 2859ea67269e481f83cebadbe397b8a2 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.362999  3431 tablet_replica.cc:333] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.363548  3431 raft_consensus.cc:2243] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.364089  3431 raft_consensus.cc:2272] T 3928359ec0c2401e98c97cb66daa7126 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.366359  3431 tablet_replica.cc:333] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50: stopping tablet replica
I20260709 12:29:55.366881  3431 raft_consensus.cc:2243] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:55.367377  3431 raft_consensus.cc:2272] T 4350e9f3eba145d89b0f53790e46cc50 P 8d1e4a4db8eb4df4bed6a3aa27230d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.416548  3431 tablet_server.cc:196] TabletServer@127.3.89.195:0 shutdown complete.
I20260709 12:29:55.452611  3431 master.cc:562] Master@127.3.89.254:39867 shutting down...
I20260709 12:29:55.476234  3431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:55.476857  3431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:55.477135  3431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2f9192d6a0f04a41bcd2e2831e205822: stopping tablet replica
I20260709 12:29:55.498311  3431 master.cc:584] Master@127.3.89.254:39867 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (22143 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:29:55.555563  3431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.3.89.254:40851
I20260709 12:29:55.556835  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:55.564157 15223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:55.564785 15225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:55.563832 15222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:55.565889  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:55.567008  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:55.567248  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:55.567421  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600195567401 us; error 0 us; skew 500 ppm
I20260709 12:29:55.567984  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:55.570595  3431 webserver.cc:533] Webserver started at http://127.3.89.254:38357/ using document root <none> and password file <none>
I20260709 12:29:55.571108  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:55.571314  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:55.571614  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:55.572822  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/master-0-root/instance:
uuid: "56d6fdfd5599486f8a9770f7ce97bfa4"
format_stamp: "Formatted at 2026-07-09 12:29:55 on dist-test-slave-3hrd"
I20260709 12:29:55.577720  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:29:55.581284 15230 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:55.582299  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:29:55.582661  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/master-0-root
uuid: "56d6fdfd5599486f8a9770f7ce97bfa4"
format_stamp: "Formatted at 2026-07-09 12:29:55 on dist-test-slave-3hrd"
I20260709 12:29:55.582974  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:55.593478  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:55.594851  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:55.636103  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:40851
I20260709 12:29:55.636246 15281 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.254:40851 every 8 connection(s)
I20260709 12:29:55.640313 15282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:55.653702 15282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4: Bootstrap starting.
I20260709 12:29:55.659355 15282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:55.664521 15282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4: No bootstrap required, opened a new log
I20260709 12:29:55.666814 15282 raft_consensus.cc:359] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6fdfd5599486f8a9770f7ce97bfa4" member_type: VOTER }
I20260709 12:29:55.667208 15282 raft_consensus.cc:385] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:55.667402 15282 raft_consensus.cc:740] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56d6fdfd5599486f8a9770f7ce97bfa4, State: Initialized, Role: FOLLOWER
I20260709 12:29:55.667899 15282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6fdfd5599486f8a9770f7ce97bfa4" member_type: VOTER }
I20260709 12:29:55.668339 15282 raft_consensus.cc:399] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:55.668551 15282 raft_consensus.cc:493] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:55.668763 15282 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:55.673785 15282 raft_consensus.cc:515] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6fdfd5599486f8a9770f7ce97bfa4" member_type: VOTER }
I20260709 12:29:55.674288 15282 leader_election.cc:304] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56d6fdfd5599486f8a9770f7ce97bfa4; no voters: 
I20260709 12:29:55.675400 15282 leader_election.cc:290] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:55.675755 15285 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:55.677222 15285 raft_consensus.cc:697] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [term 1 LEADER]: Becoming Leader. State: Replica: 56d6fdfd5599486f8a9770f7ce97bfa4, State: Running, Role: LEADER
I20260709 12:29:55.677901 15285 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6fdfd5599486f8a9770f7ce97bfa4" member_type: VOTER }
I20260709 12:29:55.678653 15282 sys_catalog.cc:565] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:55.683010 15287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56d6fdfd5599486f8a9770f7ce97bfa4. Latest consensus state: current_term: 1 leader_uuid: "56d6fdfd5599486f8a9770f7ce97bfa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6fdfd5599486f8a9770f7ce97bfa4" member_type: VOTER } }
I20260709 12:29:55.683894 15287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:55.686636 15286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56d6fdfd5599486f8a9770f7ce97bfa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56d6fdfd5599486f8a9770f7ce97bfa4" member_type: VOTER } }
I20260709 12:29:55.687453 15286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:55.690452 15291 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:55.695578 15291 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:55.699783  3431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:55.705542 15291 catalog_manager.cc:1383] Generated new cluster ID: 4342d646a8f84519ac2d73163fe98ff2
I20260709 12:29:55.705924 15291 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:55.720309 15291 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:55.721698 15291 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:55.737645 15291 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4: Generated new TSK 0
I20260709 12:29:55.738291 15291 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:55.767388  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:55.773849 15303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:55.774874 15304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:55.776294 15306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:55.776918  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:55.777981  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:55.778215  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:55.778389  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600195778366 us; error 0 us; skew 500 ppm
I20260709 12:29:55.778926  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:55.781348  3431 webserver.cc:533] Webserver started at http://127.3.89.193:34999/ using document root <none> and password file <none>
I20260709 12:29:55.781853  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:55.782063  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:55.782335  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:55.783447  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-0-root/instance:
uuid: "16adde14591b47b6a9732fe4a957bd14"
format_stamp: "Formatted at 2026-07-09 12:29:55 on dist-test-slave-3hrd"
I20260709 12:29:55.787964  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:29:55.791419 15311 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:55.792232  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:29:55.792572  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-0-root
uuid: "16adde14591b47b6a9732fe4a957bd14"
format_stamp: "Formatted at 2026-07-09 12:29:55 on dist-test-slave-3hrd"
I20260709 12:29:55.792901  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:55.810608  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:55.812088  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:55.813761  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:55.816188  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:55.816473  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:55.816740  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:55.816938  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:55.860837  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:45781
I20260709 12:29:55.860958 15373 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.193:45781 every 8 connection(s)
I20260709 12:29:55.866837  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:55.874918 15378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:55.876507 15379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:55.879057 15381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:55.879683  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:55.881114  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:55.881440  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:55.881671  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600195881654 us; error 0 us; skew 500 ppm
I20260709 12:29:55.882467  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:55.885311 15374 heartbeater.cc:344] Connected to a master server at 127.3.89.254:40851
I20260709 12:29:55.885505  3431 webserver.cc:533] Webserver started at http://127.3.89.194:40259/ using document root <none> and password file <none>
I20260709 12:29:55.885753 15374 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:55.886268  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:55.886533  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:55.886716 15374 heartbeater.cc:507] Master 127.3.89.254:40851 requested a full tablet report, sending...
I20260709 12:29:55.886826  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:55.888500  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-1-root/instance:
uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
format_stamp: "Formatted at 2026-07-09 12:29:55 on dist-test-slave-3hrd"
I20260709 12:29:55.889312 15247 ts_manager.cc:194] Registered new tserver with Master: 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781)
I20260709 12:29:55.891482 15247 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52732
I20260709 12:29:55.894029  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:29:55.897876 15386 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:55.898785  3431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:29:55.899044  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-1-root
uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
format_stamp: "Formatted at 2026-07-09 12:29:55 on dist-test-slave-3hrd"
I20260709 12:29:55.899336  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:55.917909  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:55.919068  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:55.920493  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:55.922850  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:55.923031  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:55.923341  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:55.923484  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:55.965456  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:33189
I20260709 12:29:55.965665 15448 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.194:33189 every 8 connection(s)
I20260709 12:29:55.970396  3431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:55.977948 15452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:55.980017 15453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:55.983412  3431 server_base.cc:1061] running on GCE node
I20260709 12:29:55.984191 15449 heartbeater.cc:344] Connected to a master server at 127.3.89.254:40851
W20260709 12:29:55.984294 15455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:55.984721 15449 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:55.985461  3431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:29:55.985653 15449 heartbeater.cc:507] Master 127.3.89.254:40851 requested a full tablet report, sending...
W20260709 12:29:55.985842  3431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:55.986112  3431 hybrid_clock.cc:648] HybridClock initialized: now 1783600195986088 us; error 0 us; skew 500 ppm
I20260709 12:29:55.986840  3431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:55.988207 15247 ts_manager.cc:194] Registered new tserver with Master: 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:29:55.989681  3431 webserver.cc:533] Webserver started at http://127.3.89.195:39605/ using document root <none> and password file <none>
I20260709 12:29:55.990249  3431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:55.990203 15247 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52748
I20260709 12:29:55.990648  3431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:55.991044  3431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:55.992393  3431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-2-root/instance:
uuid: "5e95a763e18e4814893fad06e1ea9d80"
format_stamp: "Formatted at 2026-07-09 12:29:55 on dist-test-slave-3hrd"
I20260709 12:29:55.997116  3431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:29:56.000921 15460 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:56.002034  3431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:29:56.002327  3431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-2-root
uuid: "5e95a763e18e4814893fad06e1ea9d80"
format_stamp: "Formatted at 2026-07-09 12:29:55 on dist-test-slave-3hrd"
I20260709 12:29:56.002614  3431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskHSPRNA/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599794910850-3431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:56.022213  3431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:56.023505  3431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:56.025033  3431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:56.027531  3431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:56.027729  3431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:56.028072  3431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:56.028229  3431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:56.072366  3431 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:38943
I20260709 12:29:56.072486 15523 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.3.89.195:38943 every 8 connection(s)
I20260709 12:29:56.087694 15524 heartbeater.cc:344] Connected to a master server at 127.3.89.254:40851
I20260709 12:29:56.088235 15524 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:56.089102 15524 heartbeater.cc:507] Master 127.3.89.254:40851 requested a full tablet report, sending...
I20260709 12:29:56.091399 15247 ts_manager.cc:194] Registered new tserver with Master: 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:56.092259  3431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016263769s
I20260709 12:29:56.093029 15247 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52752
I20260709 12:29:56.120663 15247 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52756:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:56.123778 15247 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:56.206002 15487 tablet_service.cc:1511] Processing CreateTablet for tablet e48cab88c8f84967baa4e76847f252da (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:56.206064 15486 tablet_service.cc:1511] Processing CreateTablet for tablet 71e7aac918ff414b80c722862b23d801 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:56.207674 15487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e48cab88c8f84967baa4e76847f252da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.208444 15486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71e7aac918ff414b80c722862b23d801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.209200 15489 tablet_service.cc:1511] Processing CreateTablet for tablet 25474d56fa3c452ea0f06683eaf900f1 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:56.210651 15489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25474d56fa3c452ea0f06683eaf900f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.213546 15488 tablet_service.cc:1511] Processing CreateTablet for tablet 1e1136e261f24c7b84f1331f507d74f5 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:56.215162 15488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e1136e261f24c7b84f1331f507d74f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.230851 15338 tablet_service.cc:1511] Processing CreateTablet for tablet 1e1136e261f24c7b84f1331f507d74f5 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:56.233049 15338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e1136e261f24c7b84f1331f507d74f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.230957 15339 tablet_service.cc:1511] Processing CreateTablet for tablet 25474d56fa3c452ea0f06683eaf900f1 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:56.234875 15485 tablet_service.cc:1511] Processing CreateTablet for tablet a81f9e32762c40f0bc6a7faa29e39744 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:56.235651 15339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25474d56fa3c452ea0f06683eaf900f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.236428 15485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a81f9e32762c40f0bc6a7faa29e39744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.238533 15539 tablet_bootstrap.cc:492] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:29:56.242036 15336 tablet_service.cc:1511] Processing CreateTablet for tablet 71e7aac918ff414b80c722862b23d801 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:56.243561 15336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71e7aac918ff414b80c722862b23d801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.246714 15335 tablet_service.cc:1511] Processing CreateTablet for tablet a81f9e32762c40f0bc6a7faa29e39744 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:56.247995 15414 tablet_service.cc:1511] Processing CreateTablet for tablet 25474d56fa3c452ea0f06683eaf900f1 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:56.248158 15411 tablet_service.cc:1511] Processing CreateTablet for tablet 71e7aac918ff414b80c722862b23d801 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:56.249003 15337 tablet_service.cc:1511] Processing CreateTablet for tablet e48cab88c8f84967baa4e76847f252da (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:56.248167 15335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a81f9e32762c40f0bc6a7faa29e39744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.251166 15337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e48cab88c8f84967baa4e76847f252da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.250908 15413 tablet_service.cc:1511] Processing CreateTablet for tablet 1e1136e261f24c7b84f1331f507d74f5 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:56.252797 15410 tablet_service.cc:1511] Processing CreateTablet for tablet a81f9e32762c40f0bc6a7faa29e39744 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:56.253506 15412 tablet_service.cc:1511] Processing CreateTablet for tablet e48cab88c8f84967baa4e76847f252da (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:56.255704 15414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25474d56fa3c452ea0f06683eaf900f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.256608 15410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a81f9e32762c40f0bc6a7faa29e39744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.257403 15412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e48cab88c8f84967baa4e76847f252da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.260524 15413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e1136e261f24c7b84f1331f507d74f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.266086 15411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71e7aac918ff414b80c722862b23d801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.256146 15539 tablet_bootstrap.cc:654] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.276111 15540 tablet_bootstrap.cc:492] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:29:56.296485 15337 tablet_service.cc:1511] Processing CreateTablet for tablet a65d04ca2b97426c86a48fa9fd9825c2 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:56.298108 15337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a65d04ca2b97426c86a48fa9fd9825c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.286401 15540 tablet_bootstrap.cc:654] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.304382 15409 tablet_service.cc:1511] Processing CreateTablet for tablet a65d04ca2b97426c86a48fa9fd9825c2 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:56.306016 15409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a65d04ca2b97426c86a48fa9fd9825c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.326649 15486 tablet_service.cc:1511] Processing CreateTablet for tablet a65d04ca2b97426c86a48fa9fd9825c2 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:56.338376 15486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a65d04ca2b97426c86a48fa9fd9825c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.341398 15539 tablet_bootstrap.cc:492] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:29:56.350078 15539 ts_tablet_manager.cc:1403] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.112s	user 0.015s	sys 0.013s
I20260709 12:29:56.350600 15540 tablet_bootstrap.cc:492] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:29:56.351219 15540 ts_tablet_manager.cc:1403] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.016s
I20260709 12:29:56.354667 15540 raft_consensus.cc:359] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.355484 15540 raft_consensus.cc:385] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.355818 15540 raft_consensus.cc:740] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.356792 15540 consensus_queue.cc:260] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.342865 15336 tablet_service.cc:1511] Processing CreateTablet for tablet 4fdaa067b5184307b4e22d101add7e12 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:56.346761 15410 tablet_service.cc:1511] Processing CreateTablet for tablet 4fdaa067b5184307b4e22d101add7e12 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:56.357877 15539 raft_consensus.cc:359] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.359395 15539 raft_consensus.cc:385] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.359587 15410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fdaa067b5184307b4e22d101add7e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.359812 15539 raft_consensus.cc:740] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.363862 15336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fdaa067b5184307b4e22d101add7e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.364799 15485 tablet_service.cc:1511] Processing CreateTablet for tablet 4fdaa067b5184307b4e22d101add7e12 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:56.366757 15485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fdaa067b5184307b4e22d101add7e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.367305 15338 tablet_service.cc:1511] Processing CreateTablet for tablet eb71faa206ff442ea23f3a704dda68de (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:56.362059 15539 consensus_queue.cc:260] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.369585 15338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb71faa206ff442ea23f3a704dda68de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.377166 15414 tablet_service.cc:1511] Processing CreateTablet for tablet eb71faa206ff442ea23f3a704dda68de (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:56.379314 15414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb71faa206ff442ea23f3a704dda68de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.392496 15488 tablet_service.cc:1511] Processing CreateTablet for tablet eb71faa206ff442ea23f3a704dda68de (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:56.394630 15488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb71faa206ff442ea23f3a704dda68de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.418239 15543 tablet_bootstrap.cc:492] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:29:56.418516 15335 tablet_service.cc:1511] Processing CreateTablet for tablet cb09d8084bee4073bc05ba1ac0899d04 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:56.420447 15335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb09d8084bee4073bc05ba1ac0899d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.422340 15374 heartbeater.cc:499] Master 127.3.89.254:40851 was elected leader, sending a full tablet report...
I20260709 12:29:56.433089 15409 tablet_service.cc:1511] Processing CreateTablet for tablet cb09d8084bee4073bc05ba1ac0899d04 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:56.434593 15409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb09d8084bee4073bc05ba1ac0899d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.438674 15540 ts_tablet_manager.cc:1434] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.087s	user 0.016s	sys 0.028s
I20260709 12:29:56.439880 15540 tablet_bootstrap.cc:492] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:29:56.449776 15524 heartbeater.cc:499] Master 127.3.89.254:40851 was elected leader, sending a full tablet report...
I20260709 12:29:56.450364 15539 ts_tablet_manager.cc:1434] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.100s	user 0.013s	sys 0.047s
I20260709 12:29:56.450047 15488 tablet_service.cc:1511] Processing CreateTablet for tablet cb09d8084bee4073bc05ba1ac0899d04 (DEFAULT_TABLE table=test-workload [id=28a77ab371cb439ab8d16fd0fd2646d0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:56.451511 15488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb09d8084bee4073bc05ba1ac0899d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:56.454998 15543 tablet_bootstrap.cc:654] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.451480 15539 tablet_bootstrap.cc:492] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:29:56.466154 15539 tablet_bootstrap.cc:654] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.484351 15540 tablet_bootstrap.cc:654] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.494515 15543 tablet_bootstrap.cc:492] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:29:56.494988 15543 ts_tablet_manager.cc:1403] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.002s
I20260709 12:29:56.500801 15539 tablet_bootstrap.cc:492] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:29:56.497363 15543 raft_consensus.cc:359] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.501446 15539 ts_tablet_manager.cc:1403] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.050s	user 0.010s	sys 0.025s
I20260709 12:29:56.501719 15543 raft_consensus.cc:385] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.502085 15543 raft_consensus.cc:740] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.503278 15540 tablet_bootstrap.cc:492] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:29:56.503100 15543 consensus_queue.cc:260] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.503965 15540 ts_tablet_manager.cc:1403] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.064s	user 0.010s	sys 0.016s
I20260709 12:29:56.504803 15539 raft_consensus.cc:359] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:29:56.508365 15539 raft_consensus.cc:385] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.508796 15539 raft_consensus.cc:740] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.509560 15539 consensus_queue.cc:260] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:29:56.510524 15543 ts_tablet_manager.cc:1434] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.005s
I20260709 12:29:56.511296 15449 heartbeater.cc:499] Master 127.3.89.254:40851 was elected leader, sending a full tablet report...
I20260709 12:29:56.511297 15540 raft_consensus.cc:359] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:29:56.511897 15539 ts_tablet_manager.cc:1434] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 12:29:56.512255 15540 raft_consensus.cc:385] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.512558 15540 raft_consensus.cc:740] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.513491 15539 tablet_bootstrap.cc:492] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:29:56.515441 15543 tablet_bootstrap.cc:492] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:29:56.517073 15540 consensus_queue.cc:260] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:29:56.519465 15540 ts_tablet_manager.cc:1434] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 12:29:56.520541 15540 tablet_bootstrap.cc:492] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:29:56.523557 15543 tablet_bootstrap.cc:654] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.527290 15540 tablet_bootstrap.cc:654] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.528753 15539 tablet_bootstrap.cc:654] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.535723 15543 tablet_bootstrap.cc:492] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:29:56.536286 15543 ts_tablet_manager.cc:1403] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 12:29:56.539146 15540 tablet_bootstrap.cc:492] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:29:56.538998 15543 raft_consensus.cc:359] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.539713 15543 raft_consensus.cc:385] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.539762 15540 ts_tablet_manager.cc:1403] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.001s
I20260709 12:29:56.540032 15543 raft_consensus.cc:740] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.542163 15543 consensus_queue.cc:260] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.543632 15540 raft_consensus.cc:359] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.544456 15543 ts_tablet_manager.cc:1434] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:56.544715 15540 raft_consensus.cc:385] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.545192 15540 raft_consensus.cc:740] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.545851 15543 tablet_bootstrap.cc:492] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:29:56.546550 15540 consensus_queue.cc:260] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.549041 15540 ts_tablet_manager.cc:1434] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:29:56.549678 15539 tablet_bootstrap.cc:492] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:29:56.550086 15540 tablet_bootstrap.cc:492] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:29:56.550209 15539 ts_tablet_manager.cc:1403] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.006s
I20260709 12:29:56.552925 15543 tablet_bootstrap.cc:654] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.553385 15539 raft_consensus.cc:359] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.554268 15539 raft_consensus.cc:385] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.554585 15539 raft_consensus.cc:740] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.555586 15539 consensus_queue.cc:260] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.557624 15540 tablet_bootstrap.cc:654] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.565054 15539 ts_tablet_manager.cc:1434] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:29:56.566383 15539 tablet_bootstrap.cc:492] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:29:56.567152 15543 tablet_bootstrap.cc:492] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:29:56.567737 15543 ts_tablet_manager.cc:1403] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.007s
I20260709 12:29:56.570600 15543 raft_consensus.cc:359] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.571182 15543 raft_consensus.cc:385] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.571440 15543 raft_consensus.cc:740] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.572178 15543 consensus_queue.cc:260] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.572774 15539 tablet_bootstrap.cc:654] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.575194 15543 ts_tablet_manager.cc:1434] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.003s
I20260709 12:29:56.576257 15543 tablet_bootstrap.cc:492] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:29:56.576725 15540 tablet_bootstrap.cc:492] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
W20260709 12:29:56.577008 15525 tablet.cc:2406] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:29:56.577479 15540 ts_tablet_manager.cc:1403] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.006s
I20260709 12:29:56.581766 15540 raft_consensus.cc:359] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.582762 15540 raft_consensus.cc:385] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.582944 15543 tablet_bootstrap.cc:654] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.583209 15540 raft_consensus.cc:740] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.584635 15540 consensus_queue.cc:260] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.592993 15540 ts_tablet_manager.cc:1434] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:29:56.594583 15540 tablet_bootstrap.cc:492] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:29:56.595408 15543 tablet_bootstrap.cc:492] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:29:56.595979 15543 ts_tablet_manager.cc:1403] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 12:29:56.598997 15543 raft_consensus.cc:359] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.599776 15543 raft_consensus.cc:385] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.600090 15543 raft_consensus.cc:740] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.601132 15543 consensus_queue.cc:260] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.603245 15539 tablet_bootstrap.cc:492] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:29:56.603826 15539 ts_tablet_manager.cc:1403] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.022s
I20260709 12:29:56.604527 15540 tablet_bootstrap.cc:654] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.605297 15543 ts_tablet_manager.cc:1434] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:29:56.606415 15543 tablet_bootstrap.cc:492] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:29:56.607303 15539 raft_consensus.cc:359] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.608341 15539 raft_consensus.cc:385] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.608716 15539 raft_consensus.cc:740] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.609908 15539 consensus_queue.cc:260] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.613727 15543 tablet_bootstrap.cc:654] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.616091 15539 ts_tablet_manager.cc:1434] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:29:56.617240 15539 tablet_bootstrap.cc:492] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:29:56.620810 15540 tablet_bootstrap.cc:492] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:29:56.621454 15540 ts_tablet_manager.cc:1403] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.004s
I20260709 12:29:56.623878 15539 tablet_bootstrap.cc:654] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.624689 15540 raft_consensus.cc:359] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.625495 15540 raft_consensus.cc:385] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.625955 15540 raft_consensus.cc:740] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.626912 15540 consensus_queue.cc:260] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.629992 15540 ts_tablet_manager.cc:1434] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:56.631414 15540 tablet_bootstrap.cc:492] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:29:56.635537 15539 tablet_bootstrap.cc:492] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:29:56.636130 15539 ts_tablet_manager.cc:1403] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 12:29:56.638909 15544 raft_consensus.cc:493] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:56.639235 15539 raft_consensus.cc:359] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.639449 15544 raft_consensus.cc:515] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.640041 15539 raft_consensus.cc:385] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.640280 15543 tablet_bootstrap.cc:492] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:29:56.640434 15539 raft_consensus.cc:740] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.640862 15543 ts_tablet_manager.cc:1403] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.019s
I20260709 12:29:56.642000 15545 raft_consensus.cc:493] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:56.642566 15545 raft_consensus.cc:515] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:29:56.643736 15544 leader_election.cc:290] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:56.644006 15543 raft_consensus.cc:359] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:29:56.648491 15543 raft_consensus.cc:385] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.649238 15543 raft_consensus.cc:740] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.645110 15545 leader_election.cc:290] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:29:56.645486 15540 tablet_bootstrap.cc:654] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.641539 15539 consensus_queue.cc:260] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.650982 15543 consensus_queue.cc:260] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:29:56.654870 15543 ts_tablet_manager.cc:1434] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.002s
I20260709 12:29:56.657956 15543 tablet_bootstrap.cc:492] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:29:56.663328 15549 raft_consensus.cc:493] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:56.663803 15549 raft_consensus.cc:515] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:29:56.666621 15549 leader_election.cc:290] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943), 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781)
I20260709 12:29:56.675352 15540 tablet_bootstrap.cc:492] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:29:56.676047 15540 ts_tablet_manager.cc:1403] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.003s
I20260709 12:29:56.678098 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:29:56.679060 15349 raft_consensus.cc:2468] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 0.
I20260709 12:29:56.679792 15539 ts_tablet_manager.cc:1434] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.043s	user 0.010s	sys 0.002s
I20260709 12:29:56.681054 15461 leader_election.cc:304] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:29:56.687320 15539 tablet_bootstrap.cc:492] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:29:56.687884 15540 raft_consensus.cc:359] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.688743 15540 raft_consensus.cc:385] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.689038 15540 raft_consensus.cc:740] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.689916 15543 tablet_bootstrap.cc:654] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.692694 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e48cab88c8f84967baa4e76847f252da" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:29:56.692862 15540 consensus_queue.cc:260] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.693430 15424 raft_consensus.cc:2468] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
I20260709 12:29:56.695297 15565 raft_consensus.cc:2804] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:56.695973 15565 raft_consensus.cc:493] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:56.696382 15565 raft_consensus.cc:3060] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:56.695465 15295 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:56.698463 15549 raft_consensus.cc:493] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:56.698976 15549 raft_consensus.cc:515] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.701062 15549 leader_election.cc:290] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:56.702653 15544 raft_consensus.cc:493] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:56.704284 15312 leader_election.cc:304] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:29:56.705616 15544 raft_consensus.cc:515] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.709789 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:29:56.711843 15424 raft_consensus.cc:2468] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 0.
I20260709 12:29:56.712280 15544 leader_election.cc:290] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:56.716253 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e1136e261f24c7b84f1331f507d74f5" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:29:56.716398 15499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e48cab88c8f84967baa4e76847f252da" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:29:56.717025 15498 raft_consensus.cc:2468] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
I20260709 12:29:56.717357 15499 raft_consensus.cc:2468] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
I20260709 12:29:56.718685 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e1136e261f24c7b84f1331f507d74f5" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:29:56.719367 15424 raft_consensus.cc:2468] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
I20260709 12:29:56.720415 15312 leader_election.cc:304] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:29:56.721629 15570 raft_consensus.cc:2804] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:56.722028 15570 raft_consensus.cc:493] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:56.722368 15570 raft_consensus.cc:3060] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.712836 15544 raft_consensus.cc:2804] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:56.724208 15544 raft_consensus.cc:493] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:56.724573 15544 raft_consensus.cc:3060] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.727187 15545 raft_consensus.cc:493] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:56.727753 15545 raft_consensus.cc:515] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.729924 15545 leader_election.cc:290] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:29:56.730840 15570 raft_consensus.cc:515] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.731604 15539 tablet_bootstrap.cc:654] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.732215 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e7aac918ff414b80c722862b23d801" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:29:56.732899 15349 raft_consensus.cc:2468] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 0.
I20260709 12:29:56.733610 15570 leader_election.cc:290] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:56.733924 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e7aac918ff414b80c722862b23d801" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:29:56.734611 15424 raft_consensus.cc:2468] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 0.
I20260709 12:29:56.734169 15540 ts_tablet_manager.cc:1434] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.058s	user 0.010s	sys 0.000s
I20260709 12:29:56.736020 15499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:29:56.736404 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e7aac918ff414b80c722862b23d801" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:29:56.736228 15540 tablet_bootstrap.cc:492] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:29:56.736696 15461 leader_election.cc:304] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:29:56.737089 15498 raft_consensus.cc:2468] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 0.
I20260709 12:29:56.737715 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e1136e261f24c7b84f1331f507d74f5" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:29:56.738183 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e1136e261f24c7b84f1331f507d74f5" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:29:56.738368 15497 raft_consensus.cc:3060] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.738700 15388 leader_election.cc:304] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:29:56.738751 15424 raft_consensus.cc:3060] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.741753 15549 raft_consensus.cc:2804] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:56.742292 15549 raft_consensus.cc:493] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:56.742640 15549 raft_consensus.cc:3060] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.743968 15565 raft_consensus.cc:515] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:29:56.744481 15540 tablet_bootstrap.cc:654] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.745189 15499 raft_consensus.cc:2393] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9f1ed34c92d5440fa430a0d48f8fb05d in current term 1: Already voted for candidate 5e95a763e18e4814893fad06e1ea9d80 in this term.
I20260709 12:29:56.745496 15497 raft_consensus.cc:2468] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 1.
I20260709 12:29:56.745867 15545 raft_consensus.cc:2804] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:56.748565 15545 raft_consensus.cc:493] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:56.748641 15313 leader_election.cc:304] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:29:56.753823 15545 raft_consensus.cc:3060] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.754354 15570 raft_consensus.cc:2804] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:56.758417 15348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:29:56.759406 15348 raft_consensus.cc:2468] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 0.
I20260709 12:29:56.752300 15544 raft_consensus.cc:515] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.761044 15347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e7aac918ff414b80c722862b23d801" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:29:56.761826 15347 raft_consensus.cc:2468] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 0.
I20260709 12:29:56.762842 15544 leader_election.cc:290] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:56.763885 15422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e48cab88c8f84967baa4e76847f252da" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:29:56.764734 15422 raft_consensus.cc:3060] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.767285 15570 raft_consensus.cc:697] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Becoming Leader. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:29:56.768252 15570 consensus_queue.cc:237] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.770741 15387 leader_election.cc:304] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 5e95a763e18e4814893fad06e1ea9d80
I20260709 12:29:56.748095 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:29:56.770622 15549 raft_consensus.cc:515] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.773017 15349 raft_consensus.cc:3060] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.748102 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:29:56.773854 15423 raft_consensus.cc:3060] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.774505 15499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e48cab88c8f84967baa4e76847f252da" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:29:56.775110 15499 raft_consensus.cc:3060] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.746313 15565 leader_election.cc:290] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:29:56.783195 15422 raft_consensus.cc:2468] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 1.
I20260709 12:29:56.783929 15499 raft_consensus.cc:2468] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 1.
I20260709 12:29:56.785683 15313 leader_election.cc:304] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:29:56.787695 15574 raft_consensus.cc:2804] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:56.788185 15574 raft_consensus.cc:697] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Becoming Leader. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:29:56.789124 15423 raft_consensus.cc:2468] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 1.
I20260709 12:29:56.789026 15574 consensus_queue.cc:237] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.790201 15543 tablet_bootstrap.cc:492] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:29:56.791118 15499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e7aac918ff414b80c722862b23d801" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:29:56.795627 15575 raft_consensus.cc:2764] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:29:56.796990 15543 ts_tablet_manager.cc:1403] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.139s	user 0.016s	sys 0.011s
I20260709 12:29:56.800143 15539 tablet_bootstrap.cc:492] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:29:56.800263 15424 raft_consensus.cc:2468] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 1.
I20260709 12:29:56.800844 15539 ts_tablet_manager.cc:1403] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.114s	user 0.012s	sys 0.004s
I20260709 12:29:56.792204 15549 leader_election.cc:290] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:56.803943 15349 raft_consensus.cc:2468] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 1.
I20260709 12:29:56.804105 15539 raft_consensus.cc:359] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.804862 15539 raft_consensus.cc:385] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.805362 15539 raft_consensus.cc:740] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.805670 15461 leader_election.cc:304] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:29:56.806195 15539 consensus_queue.cc:260] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.807067 15565 raft_consensus.cc:2804] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:56.803507 15545 raft_consensus.cc:515] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.815585 15565 raft_consensus.cc:697] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Becoming Leader. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:29:56.816916 15499 raft_consensus.cc:2393] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9f1ed34c92d5440fa430a0d48f8fb05d in current term 1: Already voted for candidate 5e95a763e18e4814893fad06e1ea9d80 in this term.
I20260709 12:29:56.817009 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e7aac918ff414b80c722862b23d801" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:29:56.819600 15349 raft_consensus.cc:3060] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.818637 15565 consensus_queue.cc:237] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:29:56.804787 15543 raft_consensus.cc:359] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.825620 15245 catalog_manager.cc:5697] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "16adde14591b47b6a9732fe4a957bd14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:56.830932 15543 raft_consensus.cc:385] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.832904 15543 raft_consensus.cc:740] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.833250 15540 tablet_bootstrap.cc:492] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:29:56.833488 15347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e7aac918ff414b80c722862b23d801" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:29:56.833918 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e7aac918ff414b80c722862b23d801" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:29:56.833850 15540 ts_tablet_manager.cc:1403] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.098s	user 0.034s	sys 0.038s
I20260709 12:29:56.836213 15543 consensus_queue.cc:260] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.835774 15545 leader_election.cc:290] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:29:56.838507 15539 ts_tablet_manager.cc:1434] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.037s	user 0.015s	sys 0.013s
I20260709 12:29:56.839080 15540 raft_consensus.cc:359] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.839895 15540 raft_consensus.cc:385] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.840070 15539 tablet_bootstrap.cc:492] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:29:56.840214 15540 raft_consensus.cc:740] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.847321 15543 ts_tablet_manager.cc:1434] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.003s
I20260709 12:29:56.848553 15543 tablet_bootstrap.cc:492] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:29:56.853921 15424 raft_consensus.cc:2393] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e95a763e18e4814893fad06e1ea9d80 in current term 1: Already voted for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in this term.
I20260709 12:29:56.855221 15461 leader_election.cc:304] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80; no voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:29:56.841255 15540 consensus_queue.cc:260] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.859356 15540 ts_tablet_manager.cc:1434] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.002s
I20260709 12:29:56.860443 15540 tablet_bootstrap.cc:492] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:29:56.861891 15349 raft_consensus.cc:2468] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 1.
I20260709 12:29:56.863420 15387 leader_election.cc:304] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 5e95a763e18e4814893fad06e1ea9d80
I20260709 12:29:56.864848 15549 raft_consensus.cc:2804] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:56.865355 15549 raft_consensus.cc:697] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Becoming Leader. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:29:56.866256 15549 consensus_queue.cc:237] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.866998 15540 tablet_bootstrap.cc:654] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.856627 15545 raft_consensus.cc:2749] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:56.871970 15543 tablet_bootstrap.cc:654] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.876863 15539 tablet_bootstrap.cc:654] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.880846 15246 catalog_manager.cc:5697] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "5e95a763e18e4814893fad06e1ea9d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:56.888613 15543 tablet_bootstrap.cc:492] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:29:56.888978 15540 tablet_bootstrap.cc:492] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:29:56.889263 15543 ts_tablet_manager.cc:1403] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.041s	user 0.006s	sys 0.008s
I20260709 12:29:56.889513 15540 ts_tablet_manager.cc:1403] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.029s	user 0.006s	sys 0.009s
I20260709 12:29:56.890246 15539 tablet_bootstrap.cc:492] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:29:56.890770 15539 ts_tablet_manager.cc:1403] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.051s	user 0.009s	sys 0.007s
I20260709 12:29:56.893164 15543 raft_consensus.cc:359] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.894137 15543 raft_consensus.cc:385] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.893669 15539 raft_consensus.cc:359] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.894623 15539 raft_consensus.cc:385] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.894476 15543 raft_consensus.cc:740] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.895046 15539 raft_consensus.cc:740] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.895447 15570 raft_consensus.cc:493] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:56.895044 15247 catalog_manager.cc:5697] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194). New cstate: current_term: 1 leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:56.895978 15570 raft_consensus.cc:515] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.896221 15539 consensus_queue.cc:260] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.898231 15570 leader_election.cc:290] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:56.899142 15549 raft_consensus.cc:493] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:56.899389 15539 ts_tablet_manager.cc:1434] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.001s
I20260709 12:29:56.899741 15549 raft_consensus.cc:515] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.901101 15539 tablet_bootstrap.cc:492] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:29:56.901422 15540 raft_consensus.cc:359] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.902144 15540 raft_consensus.cc:385] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:56.902385 15549 leader_election.cc:290] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:56.902710 15540 raft_consensus.cc:740] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:29:56.903391 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81f9e32762c40f0bc6a7faa29e39744" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:29:56.903545 15540 consensus_queue.cc:260] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.904076 15349 raft_consensus.cc:2468] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 0.
I20260709 12:29:56.904664 15574 raft_consensus.cc:493] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:56.905915 15540 ts_tablet_manager.cc:1434] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:29:56.906342 15387 leader_election.cc:304] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:29:56.906838 15499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81f9e32762c40f0bc6a7faa29e39744" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:29:56.907548 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65d04ca2b97426c86a48fa9fd9825c2" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:29:56.907758 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65d04ca2b97426c86a48fa9fd9825c2" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:29:56.905174 15574 raft_consensus.cc:515] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.908291 15424 raft_consensus.cc:2468] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
W20260709 12:29:56.909299 15313 leader_election.cc:343] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:56.910187 15312 leader_election.cc:304] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 5e95a763e18e4814893fad06e1ea9d80
I20260709 12:29:56.910336 15574 leader_election.cc:290] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:56.907562 15499 raft_consensus.cc:2468] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 0.
I20260709 12:29:56.911600 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81f9e32762c40f0bc6a7faa29e39744" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:29:56.911888 15549 raft_consensus.cc:2804] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:56.912400 15549 raft_consensus.cc:493] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:56.912997 15549 raft_consensus.cc:3060] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.911993 15544 raft_consensus.cc:2804] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:56.914263 15544 raft_consensus.cc:493] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:56.914700 15544 raft_consensus.cc:3060] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.920239 15245 catalog_manager.cc:5697] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "16adde14591b47b6a9732fe4a957bd14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:56.922505 15545 raft_consensus.cc:493] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:56.923142 15545 raft_consensus.cc:515] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.924649 15544 raft_consensus.cc:515] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.925462 15545 leader_election.cc:290] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:29:56.912355 15497 raft_consensus.cc:2468] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
I20260709 12:29:56.926301 15543 consensus_queue.cc:260] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.931294 15543 ts_tablet_manager.cc:1434] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 12:29:56.931829 15539 tablet_bootstrap.cc:654] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.932440 15543 tablet_bootstrap.cc:492] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:29:56.908222 15540 tablet_bootstrap.cc:492] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:29:56.927045 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81f9e32762c40f0bc6a7faa29e39744" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:29:56.928285 15313 leader_election.cc:304] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:29:56.936290 15574 raft_consensus.cc:2804] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:56.927420 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb71faa206ff442ea23f3a704dda68de" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:29:56.936800 15574 raft_consensus.cc:493] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:56.928417 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb71faa206ff442ea23f3a704dda68de" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:29:56.938773 15574 raft_consensus.cc:3060] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:56.940022 15461 leader_election.cc:343] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:56.941962 15540 tablet_bootstrap.cc:654] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.937500 15349 raft_consensus.cc:2468] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 0.
I20260709 12:29:56.945290 15461 leader_election.cc:304] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:29:56.937785 15549 raft_consensus.cc:515] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.947322 15545 raft_consensus.cc:2804] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:56.947543 15424 raft_consensus.cc:2393] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16adde14591b47b6a9732fe4a957bd14 in current term 1: Already voted for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in this term.
I20260709 12:29:56.947883 15545 raft_consensus.cc:493] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:56.948361 15545 raft_consensus.cc:3060] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.952114 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65d04ca2b97426c86a48fa9fd9825c2" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:29:56.952756 15424 raft_consensus.cc:3060] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.954497 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65d04ca2b97426c86a48fa9fd9825c2" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
W20260709 12:29:56.955982 15313 leader_election.cc:343] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:56.957531 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81f9e32762c40f0bc6a7faa29e39744" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:29:56.958148 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81f9e32762c40f0bc6a7faa29e39744" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:29:56.958798 15497 raft_consensus.cc:3060] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.952736 15574 raft_consensus.cc:515] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.960096 15549 leader_election.cc:290] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:56.960858 15349 raft_consensus.cc:2393] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9f1ed34c92d5440fa430a0d48f8fb05d in current term 1: Already voted for candidate 16adde14591b47b6a9732fe4a957bd14 in this term.
I20260709 12:29:56.962420 15424 raft_consensus.cc:2468] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 1.
I20260709 12:29:56.964541 15312 leader_election.cc:304] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 5e95a763e18e4814893fad06e1ea9d80
I20260709 12:29:56.965425 15570 raft_consensus.cc:2804] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:56.959906 15544 leader_election.cc:290] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:56.970724 15497 raft_consensus.cc:2468] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 1.
I20260709 12:29:56.971303 15543 tablet_bootstrap.cc:654] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:56.971513 15545 raft_consensus.cc:515] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.972373 15388 leader_election.cc:304] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 16adde14591b47b6a9732fe4a957bd14
I20260709 12:29:56.974195 15549 raft_consensus.cc:2804] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:56.974702 15549 raft_consensus.cc:697] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Becoming Leader. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:29:56.976397 15539 tablet_bootstrap.cc:492] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:29:56.977711 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb71faa206ff442ea23f3a704dda68de" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:29:56.978027 15539 ts_tablet_manager.cc:1403] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.077s	user 0.018s	sys 0.009s
I20260709 12:29:56.978426 15349 raft_consensus.cc:3060] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:56.978562 15549 consensus_queue.cc:237] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.976766 15570 raft_consensus.cc:697] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Becoming Leader. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:29:56.981109 15570 consensus_queue.cc:237] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:56.976565 15545 leader_election.cc:290] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:29:56.979987 15574 leader_election.cc:290] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:56.985486 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81f9e32762c40f0bc6a7faa29e39744" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:29:56.986683 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb71faa206ff442ea23f3a704dda68de" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:29:56.987140 15497 raft_consensus.cc:2393] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 16adde14591b47b6a9732fe4a957bd14 in current term 1: Already voted for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in this term.
I20260709 12:29:56.979871 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a81f9e32762c40f0bc6a7faa29e39744" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
W20260709 12:29:56.989380 15461 leader_election.cc:343] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:56.991401 15312 leader_election.cc:304] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14; no voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:29:57.016644 15574 raft_consensus.cc:2749] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:57.018153 15545 raft_consensus.cc:493] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:57.018910 15545 raft_consensus.cc:515] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.033908 15539 raft_consensus.cc:359] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.034669 15539 raft_consensus.cc:385] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:57.034920 15539 raft_consensus.cc:740] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:29:57.035986 15347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fdaa067b5184307b4e22d101add7e12" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:29:57.036804 15545 leader_election.cc:290] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:29:57.037034 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fdaa067b5184307b4e22d101add7e12" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
W20260709 12:29:57.037633 15461 leader_election.cc:343] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:57.037796 15424 raft_consensus.cc:2468] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 0.
I20260709 12:29:57.039381 15461 leader_election.cc:304] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 16adde14591b47b6a9732fe4a957bd14
I20260709 12:29:57.040536 15545 raft_consensus.cc:2804] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:57.040910 15545 raft_consensus.cc:493] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:57.041277 15545 raft_consensus.cc:3060] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:57.041466 15540 tablet_bootstrap.cc:492] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:29:57.042100 15539 consensus_queue.cc:260] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.042793 15540 ts_tablet_manager.cc:1403] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.135s	user 0.010s	sys 0.024s
I20260709 12:29:57.044464 15539 ts_tablet_manager.cc:1434] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.002s
I20260709 12:29:57.045425 15539 tablet_bootstrap.cc:492] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:29:57.037750 15349 raft_consensus.cc:2468] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 1.
I20260709 12:29:57.045601 15540 raft_consensus.cc:359] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.046545 15540 raft_consensus.cc:385] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:57.046829 15540 raft_consensus.cc:740] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:29:57.044730 15246 catalog_manager.cc:5697] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "16adde14591b47b6a9732fe4a957bd14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:57.049878 15461 leader_election.cc:304] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:29:57.051167 15565 raft_consensus.cc:2804] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:57.051928 15565 raft_consensus.cc:697] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Becoming Leader. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:29:57.052677 15565 consensus_queue.cc:237] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.053117 15540 consensus_queue.cc:260] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.053718 15539 tablet_bootstrap.cc:654] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:57.054486 15543 tablet_bootstrap.cc:492] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:29:57.055022 15543 ts_tablet_manager.cc:1403] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.123s	user 0.020s	sys 0.022s
I20260709 12:29:57.055323 15540 ts_tablet_manager.cc:1434] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:29:57.058216 15543 raft_consensus.cc:359] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.058717 15539 tablet_bootstrap.cc:492] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:29:57.059062 15543 raft_consensus.cc:385] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:57.059259 15539 ts_tablet_manager.cc:1403] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 12:29:57.059448 15543 raft_consensus.cc:740] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:29:57.060127 15543 consensus_queue.cc:260] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.061921 15543 ts_tablet_manager.cc:1434] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:57.062857 15543 tablet_bootstrap.cc:492] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:29:57.051142 15247 catalog_manager.cc:5697] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194). New cstate: current_term: 1 leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:57.068421 15543 tablet_bootstrap.cc:654] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:57.073896 15545 raft_consensus.cc:515] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.076174 15545 leader_election.cc:290] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:29:57.077780 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fdaa067b5184307b4e22d101add7e12" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:29:57.078495 15349 raft_consensus.cc:3060] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:57.081964 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fdaa067b5184307b4e22d101add7e12" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:29:57.082901 15424 raft_consensus.cc:3060] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:57.090538 15349 raft_consensus.cc:2468] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 1.
I20260709 12:29:57.091869 15461 leader_election.cc:304] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:29:57.092811 15545 raft_consensus.cc:2804] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:57.093327 15545 raft_consensus.cc:697] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Becoming Leader. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:29:57.094202 15545 consensus_queue.cc:237] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.081915 15539 raft_consensus.cc:359] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.104699 15539 raft_consensus.cc:385] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:57.105134 15539 raft_consensus.cc:740] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:29:57.106284 15539 consensus_queue.cc:260] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.108363 15543 tablet_bootstrap.cc:492] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:29:57.108850 15543 ts_tablet_manager.cc:1403] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.046s	user 0.008s	sys 0.009s
I20260709 12:29:57.109194 15539 ts_tablet_manager.cc:1434] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.000s
I20260709 12:29:57.111057 15543 raft_consensus.cc:359] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.111838 15543 raft_consensus.cc:385] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:57.112100 15543 raft_consensus.cc:740] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:29:57.112460 15424 raft_consensus.cc:2468] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 1.
I20260709 12:29:57.112779 15543 consensus_queue.cc:260] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.115023 15543 ts_tablet_manager.cc:1434] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:29:57.116535 15245 catalog_manager.cc:5697] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "5e95a763e18e4814893fad06e1ea9d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:57.132122 15245 catalog_manager.cc:5697] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "5e95a763e18e4814893fad06e1ea9d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:57.170843 15570 consensus_queue.cc:1048] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:57.183068 15590 consensus_queue.cc:1048] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:57.209403 15545 consensus_queue.cc:1048] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:57.221161 15565 consensus_queue.cc:1048] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:57.327512 15590 raft_consensus.cc:493] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:57.328140 15590 raft_consensus.cc:515] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.330124 15590 leader_election.cc:290] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:57.330752 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb09d8084bee4073bc05ba1ac0899d04" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:29:57.331209 15570 consensus_queue.cc:1048] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:57.331240 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb09d8084bee4073bc05ba1ac0899d04" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:29:57.331710 15424 raft_consensus.cc:2468] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
I20260709 12:29:57.332388 15497 raft_consensus.cc:2468] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
I20260709 12:29:57.333215 15312 leader_election.cc:304] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:29:57.334159 15570 raft_consensus.cc:2804] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:57.334605 15570 raft_consensus.cc:493] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:57.334955 15570 raft_consensus.cc:3060] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:57.344756 15570 raft_consensus.cc:515] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.345070 15590 consensus_queue.cc:1048] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:57.348042 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb09d8084bee4073bc05ba1ac0899d04" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:29:57.348757 15423 raft_consensus.cc:3060] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:57.355077 15570 leader_election.cc:290] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:57.356210 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb09d8084bee4073bc05ba1ac0899d04" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:29:57.356917 15497 raft_consensus.cc:3060] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:57.357621 15423 raft_consensus.cc:2468] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 1.
I20260709 12:29:57.359019 15312 leader_election.cc:304] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:29:57.360028 15570 raft_consensus.cc:2804] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:57.360682 15570 raft_consensus.cc:697] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Becoming Leader. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:29:57.361657 15570 consensus_queue.cc:237] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.364092 15497 raft_consensus.cc:2468] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 1.
I20260709 12:29:57.370121 15245 catalog_manager.cc:5697] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "16adde14591b47b6a9732fe4a957bd14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:57.397037 15497 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1"
dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
mode: GRACEFUL
new_leader_uuid: "16adde14591b47b6a9732fe4a957bd14"
 from {username='slave'} at 127.0.0.1:54066
I20260709 12:29:57.397678 15497 raft_consensus.cc:606] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Received request to transfer leadership to TS 16adde14591b47b6a9732fe4a957bd14
I20260709 12:29:57.452840 15423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71e7aac918ff414b80c722862b23d801"
dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
mode: GRACEFUL
new_leader_uuid: "16adde14591b47b6a9732fe4a957bd14"
 from {username='slave'} at 127.0.0.1:38700
I20260709 12:29:57.453879 15423 raft_consensus.cc:606] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Received request to transfer leadership to TS 16adde14591b47b6a9732fe4a957bd14
I20260709 12:29:57.461726 15549 consensus_queue.cc:1048] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:57.463109 15575 raft_consensus.cc:493] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:57.464119 15575 raft_consensus.cc:515] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.467356 15575 leader_election.cc:290] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:57.471009 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb71faa206ff442ea23f3a704dda68de" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:29:57.472115 15499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb71faa206ff442ea23f3a704dda68de" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:29:57.472744 15349 raft_consensus.cc:2393] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9f1ed34c92d5440fa430a0d48f8fb05d in current term 1: Already voted for candidate 5e95a763e18e4814893fad06e1ea9d80 in this term.
I20260709 12:29:57.476352 15388 leader_election.cc:304] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80
I20260709 12:29:57.479645 15549 raft_consensus.cc:3060] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:57.483340 15575 consensus_queue.cc:1048] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:57.507117 15549 raft_consensus.cc:2749] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:57.507517 15575 consensus_queue.cc:1048] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:57.519411 15570 consensus_queue.cc:1048] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:57.521404 15565 raft_consensus.cc:493] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:57.522313 15565 raft_consensus.cc:515] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.524012 15617 consensus_queue.cc:1048] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:57.525933 15617 raft_consensus.cc:993] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d: : Instructing follower 16adde14591b47b6a9732fe4a957bd14 to start an election
I20260709 12:29:57.526548 15617 raft_consensus.cc:1081] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Signalling peer 16adde14591b47b6a9732fe4a957bd14 to start an election
I20260709 12:29:57.527308 15545 consensus_queue.cc:1048] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:57.528597 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65d04ca2b97426c86a48fa9fd9825c2" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:29:57.534940 15349 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71e7aac918ff414b80c722862b23d801"
dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
 from {username='slave'} at 127.0.0.1:57940
I20260709 12:29:57.535573 15348 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a65d04ca2b97426c86a48fa9fd9825c2"
dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
mode: GRACEFUL
new_leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
 from {username='slave'} at 127.0.0.1:57956
I20260709 12:29:57.535815 15349 raft_consensus.cc:493] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:57.536561 15348 raft_consensus.cc:606] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Received request to transfer leadership to TS 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:29:57.536589 15349 raft_consensus.cc:3060] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.567834 15497 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fdaa067b5184307b4e22d101add7e12"
dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
mode: GRACEFUL
new_leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
 from {username='slave'} at 127.0.0.1:54066
I20260709 12:29:57.567977 15499 raft_consensus.cc:3060] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:57.579973 15349 raft_consensus.cc:515] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.580454 15545 consensus_queue.cc:1048] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:57.581454 15346 raft_consensus.cc:1240] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Rejecting Update request from peer 9f1ed34c92d5440fa430a0d48f8fb05d for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:57.584676 15575 consensus_queue.cc:1059] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:57.585501 15565 leader_election.cc:290] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:29:57.585790 15622 raft_consensus.cc:3055] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:57.586460 15622 raft_consensus.cc:740] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:29:57.586972 15422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e7aac918ff414b80c722862b23d801" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:29:57.587831 15622 consensus_queue.cc:260] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.588640 15497 raft_consensus.cc:606] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Received request to transfer leadership to TS 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:29:57.589428 15349 leader_election.cc:290] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:57.589524 15622 raft_consensus.cc:3060] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.597893 15424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65d04ca2b97426c86a48fa9fd9825c2" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:29:57.597301 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71e7aac918ff414b80c722862b23d801" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:29:57.601245 15498 raft_consensus.cc:3060] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.601873 15461 leader_election.cc:304] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80; no voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:29:57.603142 15545 raft_consensus.cc:2749] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:57.606227 15570 consensus_queue.cc:1048] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:57.625666 15570 raft_consensus.cc:993] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14: : Instructing follower 9f1ed34c92d5440fa430a0d48f8fb05d to start an election
I20260709 12:29:57.626454 15570 raft_consensus.cc:1081] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Signalling peer 9f1ed34c92d5440fa430a0d48f8fb05d to start an election
I20260709 12:29:57.630934 15424 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a65d04ca2b97426c86a48fa9fd9825c2"
dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
 from {username='slave'} at 127.0.0.1:38674
I20260709 12:29:57.632014 15424 raft_consensus.cc:493] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:57.632587 15424 raft_consensus.cc:3060] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.618575 15499 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb71faa206ff442ea23f3a704dda68de"
dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
mode: GRACEFUL
new_leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
 from {username='slave'} at 127.0.0.1:54066
I20260709 12:29:57.627712 15498 raft_consensus.cc:2468] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 2.
I20260709 12:29:57.639920 15313 leader_election.cc:304] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:29:57.642581 15499 raft_consensus.cc:606] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Received request to transfer leadership to TS 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:29:57.642377 15574 raft_consensus.cc:2804] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:57.645632 15574 raft_consensus.cc:697] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Becoming Leader. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:29:57.647737 15422 raft_consensus.cc:2468] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 2.
I20260709 12:29:57.648940 15574 consensus_queue.cc:237] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.651312 15347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb09d8084bee4073bc05ba1ac0899d04"
dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
mode: GRACEFUL
new_leader_uuid: "5e95a763e18e4814893fad06e1ea9d80"
 from {username='slave'} at 127.0.0.1:57956
I20260709 12:29:57.652060 15347 raft_consensus.cc:606] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Received request to transfer leadership to TS 5e95a763e18e4814893fad06e1ea9d80
I20260709 12:29:57.653307 15545 consensus_queue.cc:1048] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:57.658560 15424 raft_consensus.cc:515] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.659888 15423 raft_consensus.cc:1240] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Rejecting Update request from peer 16adde14591b47b6a9732fe4a957bd14 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:57.669310 15577 raft_consensus.cc:993] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80: : Instructing follower 16adde14591b47b6a9732fe4a957bd14 to start an election
I20260709 12:29:57.669534 15347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65d04ca2b97426c86a48fa9fd9825c2" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:29:57.670001 15424 leader_election.cc:290] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 2 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:57.670306 15347 raft_consensus.cc:3055] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:57.670687 15577 raft_consensus.cc:1081] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Signalling peer 16adde14591b47b6a9732fe4a957bd14 to start an election
I20260709 12:29:57.670786 15347 raft_consensus.cc:740] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:29:57.672030 15347 consensus_queue.cc:260] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.673112 15347 raft_consensus.cc:3060] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.675674 15349 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1"
dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
 from {username='slave'} at 127.0.0.1:57924
I20260709 12:29:57.681543 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a65d04ca2b97426c86a48fa9fd9825c2" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:29:57.683674 15497 raft_consensus.cc:3060] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.682504 15349 raft_consensus.cc:493] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:57.682361 15545 consensus_queue.cc:1048] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:57.684535 15349 raft_consensus.cc:3060] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.693037 15347 raft_consensus.cc:2468] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 2.
I20260709 12:29:57.695004 15387 leader_election.cc:304] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:29:57.696010 15622 raft_consensus.cc:2804] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:57.696569 15622 raft_consensus.cc:697] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 LEADER]: Becoming Leader. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:29:57.697686 15622 consensus_queue.cc:237] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.710186 15497 raft_consensus.cc:2468] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 2.
I20260709 12:29:57.709110 15246 catalog_manager.cc:5697] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 reported cstate change: term changed from 1 to 2, leader changed from 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194) to 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "16adde14591b47b6a9732fe4a957bd14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:57.716771 15349 raft_consensus.cc:515] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:29:57.718348 15545 raft_consensus.cc:993] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80: : Instructing follower 9f1ed34c92d5440fa430a0d48f8fb05d to start an election
I20260709 12:29:57.719098 15578 raft_consensus.cc:1081] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Signalling peer 9f1ed34c92d5440fa430a0d48f8fb05d to start an election
I20260709 12:29:57.720157 15349 leader_election.cc:290] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 2 election: Requested vote from peers 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:29:57.721756 15424 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fdaa067b5184307b4e22d101add7e12"
dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:29:57.722522 15424 raft_consensus.cc:493] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:57.722632 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:29:57.721683 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:29:57.723069 15424 raft_consensus.cc:3060] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.723577 15423 raft_consensus.cc:3060] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.723656 15497 raft_consensus.cc:3055] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:57.724277 15497 raft_consensus.cc:740] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:29:57.725587 15497 consensus_queue.cc:260] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:29:57.727403 15497 raft_consensus.cc:3060] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.732995 15424 raft_consensus.cc:515] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.733863 15423 raft_consensus.cc:2468] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 2.
I20260709 12:29:57.735407 15312 leader_election.cc:304] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:29:57.735627 15424 leader_election.cc:290] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 2 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:57.736250 15497 raft_consensus.cc:2468] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 2.
I20260709 12:29:57.738024 15574 raft_consensus.cc:2804] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:57.738664 15574 raft_consensus.cc:697] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Becoming Leader. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:29:57.739517 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fdaa067b5184307b4e22d101add7e12" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:29:57.739730 15574 consensus_queue.cc:237] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:29:57.740185 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fdaa067b5184307b4e22d101add7e12" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:29:57.740984 15349 raft_consensus.cc:3060] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.740223 15497 raft_consensus.cc:3055] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:57.746129 15497 raft_consensus.cc:740] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:29:57.747131 15497 consensus_queue.cc:260] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.748826 15497 raft_consensus.cc:3060] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.747462 15246 catalog_manager.cc:5697] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d reported cstate change: term changed from 1 to 2, leader changed from 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193) to 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194). New cstate: current_term: 2 leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:57.751132 15349 raft_consensus.cc:2468] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 2.
I20260709 12:29:57.751204 15247 catalog_manager.cc:5697] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 reported cstate change: term changed from 1 to 2, leader changed from 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195) to 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "16adde14591b47b6a9732fe4a957bd14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:57.754825 15387 leader_election.cc:304] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:29:57.755846 15622 raft_consensus.cc:2804] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:57.756363 15622 raft_consensus.cc:697] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 LEADER]: Becoming Leader. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:29:57.756474 15497 raft_consensus.cc:2468] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 2.
I20260709 12:29:57.757457 15622 consensus_queue.cc:237] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.766420 15247 catalog_manager.cc:5697] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d reported cstate change: term changed from 1 to 2, leader changed from 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195) to 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194). New cstate: current_term: 2 leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:57.813937 15574 consensus_queue.cc:1048] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:57.825732 15544 consensus_queue.cc:1048] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:57.834352 15570 raft_consensus.cc:993] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14: : Instructing follower 5e95a763e18e4814893fad06e1ea9d80 to start an election
I20260709 12:29:57.834904 15544 raft_consensus.cc:1081] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Signalling peer 5e95a763e18e4814893fad06e1ea9d80 to start an election
I20260709 12:29:57.836542 15497 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb09d8084bee4073bc05ba1ac0899d04"
dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
 from {username='slave'} at 127.0.0.1:54042
I20260709 12:29:57.837044 15497 raft_consensus.cc:493] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:57.837370 15497 raft_consensus.cc:3060] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.843441 15497 raft_consensus.cc:515] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.845567 15497 leader_election.cc:290] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 2 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:29:57.846671 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb09d8084bee4073bc05ba1ac0899d04" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:29:57.846917 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb09d8084bee4073bc05ba1ac0899d04" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:29:57.847635 15349 raft_consensus.cc:3055] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:57.847685 15423 raft_consensus.cc:3060] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.848342 15349 raft_consensus.cc:740] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:29:57.849311 15349 consensus_queue.cc:260] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.850556 15349 raft_consensus.cc:3060] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:57.854662 15423 raft_consensus.cc:2468] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 2.
I20260709 12:29:57.856081 15461 leader_election.cc:304] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:29:57.856876 15349 raft_consensus.cc:2468] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 2.
I20260709 12:29:57.856870 15578 raft_consensus.cc:2804] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:57.857918 15578 raft_consensus.cc:697] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 LEADER]: Becoming Leader. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:29:57.858914 15578 consensus_queue.cc:237] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:57.865654 15246 catalog_manager.cc:5697] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 reported cstate change: term changed from 1 to 2, leader changed from 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193) to 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "5e95a763e18e4814893fad06e1ea9d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:58.004448 15545 raft_consensus.cc:993] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80: : Instructing follower 9f1ed34c92d5440fa430a0d48f8fb05d to start an election
I20260709 12:29:58.004974 15578 raft_consensus.cc:1081] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Signalling peer 9f1ed34c92d5440fa430a0d48f8fb05d to start an election
I20260709 12:29:58.006645 15423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb71faa206ff442ea23f3a704dda68de"
dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:29:58.007189 15423 raft_consensus.cc:493] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:58.007467 15423 raft_consensus.cc:3060] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:58.012857 15423 raft_consensus.cc:515] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:58.014583 15423 leader_election.cc:290] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 2 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:29:58.015614 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb71faa206ff442ea23f3a704dda68de" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:29:58.015765 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb71faa206ff442ea23f3a704dda68de" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:29:58.016319 15349 raft_consensus.cc:3060] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:58.016389 15497 raft_consensus.cc:3055] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:58.016875 15497 raft_consensus.cc:740] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:29:58.017802 15497 consensus_queue.cc:260] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:58.018815 15497 raft_consensus.cc:3060] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:58.022074 15349 raft_consensus.cc:2468] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 2.
I20260709 12:29:58.023356 15387 leader_election.cc:304] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:29:58.024183 15549 raft_consensus.cc:2804] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:58.024654 15497 raft_consensus.cc:2468] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 2.
I20260709 12:29:58.024681 15549 raft_consensus.cc:697] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 LEADER]: Becoming Leader. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:29:58.026000 15549 consensus_queue.cc:237] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:29:58.033195 15246 catalog_manager.cc:5697] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d reported cstate change: term changed from 1 to 2, leader changed from 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195) to 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194). New cstate: current_term: 2 leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:58.051491 15423 raft_consensus.cc:1275] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Refusing update from remote peer 16adde14591b47b6a9732fe4a957bd14: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:58.052991 15544 consensus_queue.cc:1048] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:58.063062 15497 raft_consensus.cc:1275] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Refusing update from remote peer 16adde14591b47b6a9732fe4a957bd14: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:58.064687 15590 consensus_queue.cc:1048] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:58.238449 15497 raft_consensus.cc:1275] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Refusing update from remote peer 16adde14591b47b6a9732fe4a957bd14: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:58.239998 15590 consensus_queue.cc:1048] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:58.240918 15423 raft_consensus.cc:1275] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Refusing update from remote peer 16adde14591b47b6a9732fe4a957bd14: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:58.242640 15590 consensus_queue.cc:1048] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:58.281095 15349 raft_consensus.cc:1275] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Refusing update from remote peer 9f1ed34c92d5440fa430a0d48f8fb05d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:58.282629 15549 consensus_queue.cc:1048] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:58.291781 15497 raft_consensus.cc:1275] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Refusing update from remote peer 9f1ed34c92d5440fa430a0d48f8fb05d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:58.293874 15632 consensus_queue.cc:1048] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:58.338617 15497 raft_consensus.cc:1275] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Refusing update from remote peer 9f1ed34c92d5440fa430a0d48f8fb05d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:58.339944 15632 consensus_queue.cc:1048] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:58.348107 15349 raft_consensus.cc:1275] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Refusing update from remote peer 9f1ed34c92d5440fa430a0d48f8fb05d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:58.349910 15575 consensus_queue.cc:1048] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:58.415019 15349 raft_consensus.cc:1275] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Refusing update from remote peer 5e95a763e18e4814893fad06e1ea9d80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:58.416594 15578 consensus_queue.cc:1048] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:58.427001 15423 raft_consensus.cc:1275] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Refusing update from remote peer 5e95a763e18e4814893fad06e1ea9d80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:58.428681 15633 consensus_queue.cc:1048] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:58.498709 15349 raft_consensus.cc:1275] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Refusing update from remote peer 9f1ed34c92d5440fa430a0d48f8fb05d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:58.500087 15575 consensus_queue.cc:1048] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:58.508752 15497 raft_consensus.cc:1275] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Refusing update from remote peer 9f1ed34c92d5440fa430a0d48f8fb05d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:58.510751 15632 consensus_queue.cc:1048] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:00.686070 15246 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52770:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:30:00.688668 15246 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:30:00.749118 15335 tablet_service.cc:1511] Processing CreateTablet for tablet 5c92e0c0305648ac83c7b229b8577fc8 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:00.750425 15409 tablet_service.cc:1511] Processing CreateTablet for tablet 5c92e0c0305648ac83c7b229b8577fc8 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:00.751488 15335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c92e0c0305648ac83c7b229b8577fc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.752434 15488 tablet_service.cc:1511] Processing CreateTablet for tablet 5c92e0c0305648ac83c7b229b8577fc8 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:00.754320 15488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c92e0c0305648ac83c7b229b8577fc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.754701 15336 tablet_service.cc:1511] Processing CreateTablet for tablet 03709ab5a8864cadb7b3242b2855ab95 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:00.755510 15414 tablet_service.cc:1511] Processing CreateTablet for tablet 03709ab5a8864cadb7b3242b2855ab95 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:00.756156 15336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03709ab5a8864cadb7b3242b2855ab95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.759505 15338 tablet_service.cc:1511] Processing CreateTablet for tablet a9feb97705d048d688949803485eb490 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:00.760507 15486 tablet_service.cc:1511] Processing CreateTablet for tablet 03709ab5a8864cadb7b3242b2855ab95 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:00.761044 15338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9feb97705d048d688949803485eb490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.761982 15486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03709ab5a8864cadb7b3242b2855ab95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.766968 15485 tablet_service.cc:1511] Processing CreateTablet for tablet a9feb97705d048d688949803485eb490 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:00.768644 15485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9feb97705d048d688949803485eb490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.771916 15410 tablet_service.cc:1511] Processing CreateTablet for tablet a9feb97705d048d688949803485eb490 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
W20260709 12:30:00.772507 15295 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:00.775949 15487 tablet_service.cc:1511] Processing CreateTablet for tablet e7dba2e76ec54b77ac5733aaa7b716e5 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:00.778810 15489 tablet_service.cc:1511] Processing CreateTablet for tablet d468370aee07411eb5b47b1e09128986 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:00.780155 15413 tablet_service.cc:1511] Processing CreateTablet for tablet e7dba2e76ec54b77ac5733aaa7b716e5 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:00.781613 15337 tablet_service.cc:1511] Processing CreateTablet for tablet e7dba2e76ec54b77ac5733aaa7b716e5 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:00.783129 15337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7dba2e76ec54b77ac5733aaa7b716e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.786229 15339 tablet_service.cc:1511] Processing CreateTablet for tablet d468370aee07411eb5b47b1e09128986 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:00.787017 15484 tablet_service.cc:1511] Processing CreateTablet for tablet 05c6479826a143d2a68b6d9ff6fc56c5 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:00.787746 15339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d468370aee07411eb5b47b1e09128986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.777504 15487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7dba2e76ec54b77ac5733aaa7b716e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.792429 15484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05c6479826a143d2a68b6d9ff6fc56c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.799484 15483 tablet_service.cc:1511] Processing CreateTablet for tablet cbde0675b7ae445490115e46663ca4d8 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:00.801105 15483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbde0675b7ae445490115e46663ca4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.803445 15482 tablet_service.cc:1511] Processing CreateTablet for tablet 2105bdf1d73b43cfa43a714a15343c92 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:00.805182 15482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2105bdf1d73b43cfa43a714a15343c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.805220 15411 tablet_service.cc:1511] Processing CreateTablet for tablet d468370aee07411eb5b47b1e09128986 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:00.808815 15489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d468370aee07411eb5b47b1e09128986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.810884 15334 tablet_service.cc:1511] Processing CreateTablet for tablet 05c6479826a143d2a68b6d9ff6fc56c5 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:00.811390 15412 tablet_service.cc:1511] Processing CreateTablet for tablet 05c6479826a143d2a68b6d9ff6fc56c5 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:00.812675 15334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05c6479826a143d2a68b6d9ff6fc56c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.752446 15409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c92e0c0305648ac83c7b229b8577fc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.815785 15411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d468370aee07411eb5b47b1e09128986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.818130 15410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9feb97705d048d688949803485eb490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.820756 15413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7dba2e76ec54b77ac5733aaa7b716e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.826118 15412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05c6479826a143d2a68b6d9ff6fc56c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.830232 15414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03709ab5a8864cadb7b3242b2855ab95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.832630 15481 tablet_service.cc:1511] Processing CreateTablet for tablet 48cd247fcd954875a222541b22e88e41 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:00.834424 15481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48cd247fcd954875a222541b22e88e41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.857867 15336 tablet_service.cc:1511] Processing CreateTablet for tablet cbde0675b7ae445490115e46663ca4d8 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:00.860098 15336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbde0675b7ae445490115e46663ca4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.865041 15408 tablet_service.cc:1511] Processing CreateTablet for tablet cbde0675b7ae445490115e46663ca4d8 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:00.867745 15408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbde0675b7ae445490115e46663ca4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.872233 15335 tablet_service.cc:1511] Processing CreateTablet for tablet 48cd247fcd954875a222541b22e88e41 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:00.872258 15407 tablet_service.cc:1511] Processing CreateTablet for tablet 2105bdf1d73b43cfa43a714a15343c92 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:00.874126 15335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48cd247fcd954875a222541b22e88e41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.874240 15407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2105bdf1d73b43cfa43a714a15343c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.890105 15338 tablet_service.cc:1511] Processing CreateTablet for tablet 2105bdf1d73b43cfa43a714a15343c92 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:00.898690 15645 tablet_bootstrap.cc:492] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:30:00.900267 15406 tablet_service.cc:1511] Processing CreateTablet for tablet 48cd247fcd954875a222541b22e88e41 (DEFAULT_TABLE table=soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:00.910786 15406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48cd247fcd954875a222541b22e88e41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.911402 15338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2105bdf1d73b43cfa43a714a15343c92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:00.948293 15645 tablet_bootstrap.cc:654] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.015381 15645 tablet_bootstrap.cc:492] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:30:01.018060 15645 ts_tablet_manager.cc:1403] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.120s	user 0.014s	sys 0.006s
I20260709 12:30:01.026000 15645 raft_consensus.cc:359] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.026954 15645 raft_consensus.cc:385] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.027360 15645 raft_consensus.cc:740] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.028272 15645 consensus_queue.cc:260] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.029147 15647 tablet_bootstrap.cc:492] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:30:01.038691 15647 tablet_bootstrap.cc:654] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.039541 15648 tablet_bootstrap.cc:492] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:30:01.039486 15651 raft_consensus.cc:493] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.044641 15645 ts_tablet_manager.cc:1434] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.006s
I20260709 12:30:01.045992 15645 tablet_bootstrap.cc:492] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:30:01.052495 15648 tablet_bootstrap.cc:654] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.052677 15645 tablet_bootstrap.cc:654] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.056226 15651 raft_consensus.cc:515] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.059098 15651 leader_election.cc:290] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.059698 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9feb97705d048d688949803485eb490" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:30:01.062362 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9feb97705d048d688949803485eb490" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:30:01.063057 15647 tablet_bootstrap.cc:492] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:30:01.064559 15645 tablet_bootstrap.cc:492] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
W20260709 12:30:01.064620 15312 leader_election.cc:343] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.065007 15647 ts_tablet_manager.cc:1403] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.005s
I20260709 12:30:01.065235 15645 ts_tablet_manager.cc:1403] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.007s
W20260709 12:30:01.066016 15313 leader_election.cc:343] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.066568 15313 leader_election.cc:304] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14; no voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:30:01.067664 15650 raft_consensus.cc:2749] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:01.068850 15645 raft_consensus.cc:359] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.068612 15647 raft_consensus.cc:359] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.069628 15645 raft_consensus.cc:385] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.069764 15647 raft_consensus.cc:385] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.070076 15645 raft_consensus.cc:740] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.070181 15647 raft_consensus.cc:740] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.071128 15645 consensus_queue.cc:260] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.071097 15647 consensus_queue.cc:260] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.072760 15648 tablet_bootstrap.cc:492] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:30:01.073361 15648 ts_tablet_manager.cc:1403] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.004s
I20260709 12:30:01.073918 15645 ts_tablet_manager.cc:1434] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:30:01.074191 15647 ts_tablet_manager.cc:1434] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.006s
I20260709 12:30:01.075107 15647 tablet_bootstrap.cc:492] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:30:01.076304 15645 tablet_bootstrap.cc:492] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:30:01.081883 15647 tablet_bootstrap.cc:654] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.086642 15645 tablet_bootstrap.cc:654] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.086840 15648 raft_consensus.cc:359] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.087865 15648 raft_consensus.cc:385] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.088160 15648 raft_consensus.cc:740] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.095161 15648 consensus_queue.cc:260] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.099896 15647 tablet_bootstrap.cc:492] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:30:01.100181 15645 tablet_bootstrap.cc:492] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:30:01.100503 15647 ts_tablet_manager.cc:1403] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.026s	user 0.006s	sys 0.015s
I20260709 12:30:01.100761 15645 ts_tablet_manager.cc:1403] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.000s
I20260709 12:30:01.103456 15648 ts_tablet_manager.cc:1434] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.003s
I20260709 12:30:01.103806 15647 raft_consensus.cc:359] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.104594 15647 raft_consensus.cc:385] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.104765 15648 tablet_bootstrap.cc:492] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:30:01.104923 15647 raft_consensus.cc:740] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.105249 15645 raft_consensus.cc:359] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.106195 15645 raft_consensus.cc:385] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.106032 15647 consensus_queue.cc:260] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.106566 15645 raft_consensus.cc:740] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.107508 15645 consensus_queue.cc:260] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.110634 15645 ts_tablet_manager.cc:1434] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:30:01.111821 15645 tablet_bootstrap.cc:492] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:30:01.112444 15647 ts_tablet_manager.cc:1434] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.005s
I20260709 12:30:01.113739 15647 tablet_bootstrap.cc:492] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:30:01.119799 15645 tablet_bootstrap.cc:654] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.120980 15648 tablet_bootstrap.cc:654] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.126524 15647 tablet_bootstrap.cc:654] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.129984 15651 raft_consensus.cc:493] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.130604 15651 raft_consensus.cc:515] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.132812 15651 leader_election.cc:290] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.134215 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03709ab5a8864cadb7b3242b2855ab95" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
W20260709 12:30:01.136029 15312 leader_election.cc:343] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.136970 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03709ab5a8864cadb7b3242b2855ab95" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
W20260709 12:30:01.138273 15313 leader_election.cc:343] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.138680 15313 leader_election.cc:304] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14; no voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:30:01.139364 15651 raft_consensus.cc:2749] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:01.140612 15645 tablet_bootstrap.cc:492] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:30:01.140944 15648 tablet_bootstrap.cc:492] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:30:01.141225 15645 ts_tablet_manager.cc:1403] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.030s	user 0.024s	sys 0.000s
I20260709 12:30:01.141433 15648 ts_tablet_manager.cc:1403] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.037s	user 0.006s	sys 0.011s
I20260709 12:30:01.144570 15645 raft_consensus.cc:359] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.144739 15648 raft_consensus.cc:359] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.145360 15645 raft_consensus.cc:385] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.145434 15648 raft_consensus.cc:385] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.145761 15645 raft_consensus.cc:740] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.145859 15648 raft_consensus.cc:740] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.146658 15645 consensus_queue.cc:260] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.146658 15648 consensus_queue.cc:260] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.149027 15648 ts_tablet_manager.cc:1434] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:30:01.149535 15645 ts_tablet_manager.cc:1434] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:30:01.150378 15648 tablet_bootstrap.cc:492] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:30:01.151118 15647 tablet_bootstrap.cc:492] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:30:01.151645 15647 ts_tablet_manager.cc:1403] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.038s	user 0.018s	sys 0.002s
I20260709 12:30:01.152963 15645 tablet_bootstrap.cc:492] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:30:01.156847 15647 raft_consensus.cc:359] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.157655 15647 raft_consensus.cc:385] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.157967 15648 tablet_bootstrap.cc:654] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.158185 15647 raft_consensus.cc:740] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.159250 15647 consensus_queue.cc:260] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.160679 15645 tablet_bootstrap.cc:654] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.165833 15647 ts_tablet_manager.cc:1434] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260709 12:30:01.167635 15647 tablet_bootstrap.cc:492] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:30:01.190397 15647 tablet_bootstrap.cc:654] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.195104 15645 tablet_bootstrap.cc:492] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:30:01.195818 15645 ts_tablet_manager.cc:1403] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.043s	user 0.019s	sys 0.022s
I20260709 12:30:01.195840 15648 tablet_bootstrap.cc:492] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:30:01.196523 15648 ts_tablet_manager.cc:1403] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.022s
I20260709 12:30:01.199383 15645 raft_consensus.cc:359] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.199678 15648 raft_consensus.cc:359] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.200248 15645 raft_consensus.cc:385] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.200372 15648 raft_consensus.cc:385] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.200711 15648 raft_consensus.cc:740] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.200605 15645 raft_consensus.cc:740] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.201761 15645 consensus_queue.cc:260] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.201797 15648 consensus_queue.cc:260] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.202653 15647 tablet_bootstrap.cc:492] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:30:01.203384 15647 ts_tablet_manager.cc:1403] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.003s
I20260709 12:30:01.203601 15633 raft_consensus.cc:493] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.204151 15633 raft_consensus.cc:515] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.204846 15648 ts_tablet_manager.cc:1434] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:30:01.205468 15632 raft_consensus.cc:493] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.206133 15648 tablet_bootstrap.cc:492] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:30:01.205992 15632 raft_consensus.cc:515] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.206350 15633 leader_election.cc:290] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:30:01.207094 15647 raft_consensus.cc:359] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.207861 15647 raft_consensus.cc:385] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.208154 15647 raft_consensus.cc:740] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.208328 15632 leader_election.cc:290] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.211192 15632 raft_consensus.cc:493] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.210055 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:30:01.211848 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:30:01.213316 15498 raft_consensus.cc:2468] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 0.
I20260709 12:30:01.213353 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:30:01.214182 15423 raft_consensus.cc:2468] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 0.
I20260709 12:30:01.214859 15388 leader_election.cc:304] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:01.216008 15646 raft_consensus.cc:2804] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:30:01.216337 15387 leader_election.cc:343] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.216449 15646 raft_consensus.cc:493] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:01.216930 15646 raft_consensus.cc:3060] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.209010 15647 consensus_queue.cc:260] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.217548 15648 tablet_bootstrap.cc:654] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.217942 15461 leader_election.cc:304] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:01.218276 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:30:01.220126 15663 raft_consensus.cc:2804] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:01.220510 15663 raft_consensus.cc:493] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:01.220831 15663 raft_consensus.cc:3060] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.221774 15632 raft_consensus.cc:515] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
W20260709 12:30:01.222781 15461 leader_election.cc:343] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.225026 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d468370aee07411eb5b47b1e09128986" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:30:01.226351 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d468370aee07411eb5b47b1e09128986" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
W20260709 12:30:01.227106 15387 leader_election.cc:343] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:01.227833 15388 leader_election.cc:343] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.228298 15388 leader_election.cc:304] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80
I20260709 12:30:01.229537 15663 raft_consensus.cc:515] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.232978 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
W20260709 12:30:01.234776 15461 leader_election.cc:343] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.236806 15632 leader_election.cc:290] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.237372 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:30:01.238327 15663 leader_election.cc:290] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:30:01.241402 15632 raft_consensus.cc:2749] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:01.241817 15648 tablet_bootstrap.cc:492] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:30:01.242193 15632 raft_consensus.cc:493] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.242379 15648 ts_tablet_manager.cc:1403] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.004s
I20260709 12:30:01.242736 15632 raft_consensus.cc:515] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.245826 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03709ab5a8864cadb7b3242b2855ab95" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:30:01.245675 15648 raft_consensus.cc:359] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.246635 15349 raft_consensus.cc:2468] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 0.
I20260709 12:30:01.246677 15648 raft_consensus.cc:385] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.247206 15648 raft_consensus.cc:740] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.248036 15645 ts_tablet_manager.cc:1434] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.001s
I20260709 12:30:01.248116 15387 leader_election.cc:304] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:01.248279 15648 consensus_queue.cc:260] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.253531 15632 leader_election.cc:290] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.253676 15648 ts_tablet_manager.cc:1434] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:30:01.255336 15632 raft_consensus.cc:2804] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:01.255528 15648 tablet_bootstrap.cc:492] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:30:01.255752 15632 raft_consensus.cc:493] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:01.256107 15632 raft_consensus.cc:3060] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.261360 15645 tablet_bootstrap.cc:492] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:30:01.245545 15646 raft_consensus.cc:515] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.263474 15423 raft_consensus.cc:2393] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e95a763e18e4814893fad06e1ea9d80 in current term 1: Already voted for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in this term.
I20260709 12:30:01.269238 15461 leader_election.cc:304] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80; no voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:30:01.265507 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:30:01.272136 15663 raft_consensus.cc:2749] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:30:01.272637 15387 leader_election.cc:343] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.273625 15648 tablet_bootstrap.cc:654] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.273953 15646 leader_election.cc:290] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.281406 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03709ab5a8864cadb7b3242b2855ab95" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:30:01.282783 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:01.283654 15497 raft_consensus.cc:2393] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9f1ed34c92d5440fa430a0d48f8fb05d in current term 1: Already voted for candidate 5e95a763e18e4814893fad06e1ea9d80 in this term.
I20260709 12:30:01.282975 15632 raft_consensus.cc:515] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
W20260709 12:30:01.285516 15388 leader_election.cc:343] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.286636 15388 leader_election.cc:304] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80
I20260709 12:30:01.287708 15665 raft_consensus.cc:2749] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:01.289669 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03709ab5a8864cadb7b3242b2855ab95" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:30:01.290547 15349 raft_consensus.cc:3060] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.291505 15647 ts_tablet_manager.cc:1434] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.088s	user 0.018s	sys 0.022s
I20260709 12:30:01.292114 15645 tablet_bootstrap.cc:654] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.292734 15647 tablet_bootstrap.cc:492] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:30:01.300458 15647 tablet_bootstrap.cc:654] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.307749 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03709ab5a8864cadb7b3242b2855ab95" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
W20260709 12:30:01.309445 15388 leader_election.cc:343] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.309657 15645 tablet_bootstrap.cc:492] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:30:01.310758 15645 ts_tablet_manager.cc:1403] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.052s	user 0.013s	sys 0.007s
I20260709 12:30:01.311306 15349 raft_consensus.cc:2468] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 1.
I20260709 12:30:01.313145 15387 leader_election.cc:304] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 5e95a763e18e4814893fad06e1ea9d80
I20260709 12:30:01.313448 15632 leader_election.cc:290] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.314934 15632 raft_consensus.cc:2804] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:01.315428 15632 raft_consensus.cc:697] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Becoming Leader. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:30:01.316111 15645 raft_consensus.cc:359] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.316351 15632 consensus_queue.cc:237] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.316970 15645 raft_consensus.cc:385] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.317049 15647 tablet_bootstrap.cc:492] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:30:01.317318 15645 raft_consensus.cc:740] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.317704 15647 ts_tablet_manager.cc:1403] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.013s
I20260709 12:30:01.318250 15648 tablet_bootstrap.cc:492] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:30:01.318993 15648 ts_tablet_manager.cc:1403] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.064s	user 0.024s	sys 0.026s
I20260709 12:30:01.318295 15645 consensus_queue.cc:260] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.320973 15647 raft_consensus.cc:359] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.321810 15647 raft_consensus.cc:385] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.322085 15647 raft_consensus.cc:740] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.322293 15645 ts_tablet_manager.cc:1434] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.002s
I20260709 12:30:01.325814 15648 raft_consensus.cc:359] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.323065 15647 consensus_queue.cc:260] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.330750 15648 raft_consensus.cc:385] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.331132 15648 raft_consensus.cc:740] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.332442 15647 ts_tablet_manager.cc:1434] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.003s
I20260709 12:30:01.333837 15647 tablet_bootstrap.cc:492] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:30:01.333370 15648 consensus_queue.cc:260] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.336386 15648 ts_tablet_manager.cc:1434] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.004s
I20260709 12:30:01.337463 15648 tablet_bootstrap.cc:492] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:30:01.340636 15645 tablet_bootstrap.cc:492] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:30:01.344638 15648 tablet_bootstrap.cc:654] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.357659 15648 tablet_bootstrap.cc:492] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:30:01.358217 15648 ts_tablet_manager.cc:1403] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 12:30:01.358348 15645 tablet_bootstrap.cc:654] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.358873 15246 catalog_manager.cc:5697] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194). New cstate: current_term: 1 leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:01.362016 15648 raft_consensus.cc:359] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.363206 15648 raft_consensus.cc:385] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.363612 15648 raft_consensus.cc:740] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.364696 15648 consensus_queue.cc:260] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.368170 15648 ts_tablet_manager.cc:1434] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:30:01.369486 15648 tablet_bootstrap.cc:492] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:30:01.376307 15632 raft_consensus.cc:493] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.376919 15632 raft_consensus.cc:515] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.379062 15632 leader_election.cc:290] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.379170 15648 tablet_bootstrap.cc:654] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.380926 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7dba2e76ec54b77ac5733aaa7b716e5" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:30:01.381844 15497 raft_consensus.cc:2468] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 0.
I20260709 12:30:01.380926 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7dba2e76ec54b77ac5733aaa7b716e5" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:30:01.383147 15663 raft_consensus.cc:493] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.383566 15349 raft_consensus.cc:2468] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 0.
I20260709 12:30:01.383904 15663 raft_consensus.cc:515] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.385398 15387 leader_election.cc:304] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:01.386458 15632 raft_consensus.cc:2804] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:01.386822 15632 raft_consensus.cc:493] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:01.387151 15632 raft_consensus.cc:3060] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.387835 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cd247fcd954875a222541b22e88e41" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
W20260709 12:30:01.389752 15461 leader_election.cc:343] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.390703 15647 tablet_bootstrap.cc:654] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.391088 15663 leader_election.cc:290] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:30:01.393507 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cd247fcd954875a222541b22e88e41" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
W20260709 12:30:01.395792 15461 leader_election.cc:343] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.396060 15632 raft_consensus.cc:515] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.396446 15461 leader_election.cc:304] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80; no voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:30:01.397987 15663 raft_consensus.cc:2749] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:01.399587 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7dba2e76ec54b77ac5733aaa7b716e5" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:30:01.400426 15349 raft_consensus.cc:3060] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.401383 15645 tablet_bootstrap.cc:492] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:30:01.402124 15645 ts_tablet_manager.cc:1403] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.062s	user 0.023s	sys 0.012s
I20260709 12:30:01.407413 15645 raft_consensus.cc:359] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.408453 15645 raft_consensus.cc:385] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.408833 15645 raft_consensus.cc:740] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.410902 15645 consensus_queue.cc:260] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.412228 15648 tablet_bootstrap.cc:492] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:30:01.412925 15648 ts_tablet_manager.cc:1403] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.016s
I20260709 12:30:01.414157 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7dba2e76ec54b77ac5733aaa7b716e5" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:01.415030 15497 raft_consensus.cc:3060] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.416123 15648 raft_consensus.cc:359] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.416893 15648 raft_consensus.cc:385] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.417174 15648 raft_consensus.cc:740] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.417979 15648 consensus_queue.cc:260] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.419788 15648 ts_tablet_manager.cc:1434] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:30:01.423975 15648 tablet_bootstrap.cc:492] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:30:01.425315 15645 ts_tablet_manager.cc:1434] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.001s
I20260709 12:30:01.436451 15645 tablet_bootstrap.cc:492] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:30:01.437325 15648 tablet_bootstrap.cc:654] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.442639 15651 raft_consensus.cc:493] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.443388 15651 raft_consensus.cc:515] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.446695 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c92e0c0305648ac83c7b229b8577fc8" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:30:01.448694 15648 tablet_bootstrap.cc:492] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:30:01.449251 15648 ts_tablet_manager.cc:1403] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.026s	user 0.003s	sys 0.007s
W20260709 12:30:01.448697 15312 leader_election.cc:343] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.450472 15632 leader_election.cc:290] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.450812 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c92e0c0305648ac83c7b229b8577fc8" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:30:01.452262 15648 raft_consensus.cc:359] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.411572 15349 raft_consensus.cc:2468] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 1.
W20260709 12:30:01.452890 15313 leader_election.cc:343] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:01.452927 15648 raft_consensus.cc:385] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.453521 15648 raft_consensus.cc:740] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.453488 15313 leader_election.cc:304] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14; no voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:30:01.454429 15648 consensus_queue.cc:260] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.455302 15387 leader_election.cc:304] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:01.456487 15497 raft_consensus.cc:2468] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 1.
I20260709 12:30:01.456857 15648 ts_tablet_manager.cc:1434] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:30:01.456826 15664 raft_consensus.cc:2804] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:01.454484 15650 raft_consensus.cc:2749] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:01.457626 15664 raft_consensus.cc:697] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Becoming Leader. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:30:01.458577 15664 consensus_queue.cc:237] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.461629 15647 tablet_bootstrap.cc:492] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:30:01.451329 15651 leader_election.cc:290] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.462255 15647 ts_tablet_manager.cc:1403] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.129s	user 0.024s	sys 0.040s
I20260709 12:30:01.463886 15648 tablet_bootstrap.cc:492] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80: Bootstrap starting.
I20260709 12:30:01.463967 15645 tablet_bootstrap.cc:654] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.472023 15647 raft_consensus.cc:359] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.473065 15647 raft_consensus.cc:385] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.473455 15647 raft_consensus.cc:740] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.474567 15647 consensus_queue.cc:260] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.477949 15647 ts_tablet_manager.cc:1434] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:30:01.479228 15648 tablet_bootstrap.cc:654] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.479424 15647 tablet_bootstrap.cc:492] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:30:01.489256 15645 tablet_bootstrap.cc:492] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:30:01.489822 15645 ts_tablet_manager.cc:1403] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.000s
I20260709 12:30:01.493016 15645 raft_consensus.cc:359] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.493870 15645 raft_consensus.cc:385] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.494516 15647 tablet_bootstrap.cc:654] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.494278 15645 raft_consensus.cc:740] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.495676 15645 consensus_queue.cc:260] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.496821 15245 catalog_manager.cc:5697] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194). New cstate: current_term: 1 leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:01.498961 15645 ts_tablet_manager.cc:1434] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:30:01.500051 15645 tablet_bootstrap.cc:492] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14: Bootstrap starting.
I20260709 12:30:01.500982 15664 raft_consensus.cc:493] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.502384 15664 raft_consensus.cc:515] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.506775 15648 tablet_bootstrap.cc:492] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80: No bootstrap required, opened a new log
I20260709 12:30:01.507364 15648 ts_tablet_manager.cc:1403] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.001s
I20260709 12:30:01.505844 15664 leader_election.cc:290] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.511529 15647 tablet_bootstrap.cc:492] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:30:01.511209 15648 raft_consensus.cc:359] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.512127 15648 raft_consensus.cc:385] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.512264 15647 ts_tablet_manager.cc:1403] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.003s
I20260709 12:30:01.512498 15648 raft_consensus.cc:740] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.513554 15648 consensus_queue.cc:260] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.517789 15645 tablet_bootstrap.cc:654] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.517848 15647 raft_consensus.cc:359] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.519044 15647 raft_consensus.cc:385] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.519421 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9feb97705d048d688949803485eb490" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:30:01.519431 15647 raft_consensus.cc:740] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.520231 15349 raft_consensus.cc:2468] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 0.
I20260709 12:30:01.520769 15647 consensus_queue.cc:260] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.521677 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9feb97705d048d688949803485eb490" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:30:01.522382 15497 raft_consensus.cc:2468] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 0.
I20260709 12:30:01.522523 15648 ts_tablet_manager.cc:1434] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 12:30:01.524405 15647 ts_tablet_manager.cc:1434] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.012s	user 0.012s	sys 0.000s
I20260709 12:30:01.524282 15388 leader_election.cc:304] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:01.525645 15647 tablet_bootstrap.cc:492] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:30:01.525557 15664 raft_consensus.cc:2804] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:01.526510 15664 raft_consensus.cc:493] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:01.527079 15664 raft_consensus.cc:3060] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.533432 15647 tablet_bootstrap.cc:654] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.538486 15664 raft_consensus.cc:515] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.543115 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9feb97705d048d688949803485eb490" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:30:01.543154 15664 leader_election.cc:290] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.543776 15349 raft_consensus.cc:3060] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.546757 15645 tablet_bootstrap.cc:492] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14: No bootstrap required, opened a new log
I20260709 12:30:01.547446 15645 ts_tablet_manager.cc:1403] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.013s
I20260709 12:30:01.548460 15647 tablet_bootstrap.cc:492] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:30:01.548928 15647 ts_tablet_manager.cc:1403] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.002s
I20260709 12:30:01.548197 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9feb97705d048d688949803485eb490" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:01.550616 15497 raft_consensus.cc:3060] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.550570 15645 raft_consensus.cc:359] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.551467 15645 raft_consensus.cc:385] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.551414 15647 raft_consensus.cc:359] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.551962 15645 raft_consensus.cc:740] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.552207 15647 raft_consensus.cc:385] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.552549 15647 raft_consensus.cc:740] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.552781 15645 consensus_queue.cc:260] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.553223 15647 consensus_queue.cc:260] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.554040 15650 raft_consensus.cc:493] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.555368 15647 ts_tablet_manager.cc:1434] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:30:01.554785 15650 raft_consensus.cc:515] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.556365 15647 tablet_bootstrap.cc:492] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d: Bootstrap starting.
I20260709 12:30:01.565191 15497 raft_consensus.cc:2468] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 1.
I20260709 12:30:01.616765 15647 tablet_bootstrap.cc:654] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:01.573778 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c6479826a143d2a68b6d9ff6fc56c5" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:30:01.617865 15388 leader_election.cc:304] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:01.577008 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c6479826a143d2a68b6d9ff6fc56c5" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:30:01.620517 15423 raft_consensus.cc:2468] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
I20260709 12:30:01.622036 15664 raft_consensus.cc:2804] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:01.558367 15650 leader_election.cc:290] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.622747 15664 raft_consensus.cc:697] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Becoming Leader. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:30:01.623502 15498 raft_consensus.cc:2468] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
I20260709 12:30:01.624233 15664 consensus_queue.cc:237] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.628374 15349 raft_consensus.cc:2468] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 1.
I20260709 12:30:01.629637 15313 leader_election.cc:304] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:30:01.633320 15650 raft_consensus.cc:2804] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:01.633985 15650 raft_consensus.cc:493] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:01.634507 15650 raft_consensus.cc:3060] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.643533 15664 raft_consensus.cc:493] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.644881 15664 raft_consensus.cc:515] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.647249 15664 leader_election.cc:290] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.643855 15645 ts_tablet_manager.cc:1434] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14: Time spent starting tablet: real 0.096s	user 0.030s	sys 0.049s
I20260709 12:30:01.648345 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbde0675b7ae445490115e46663ca4d8" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:30:01.648394 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbde0675b7ae445490115e46663ca4d8" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:30:01.649071 15498 raft_consensus.cc:2468] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 0.
I20260709 12:30:01.649067 15349 raft_consensus.cc:2468] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 0.
I20260709 12:30:01.649317 15650 raft_consensus.cc:515] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.650529 15387 leader_election.cc:304] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:01.651249 15664 raft_consensus.cc:2804] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:01.651633 15664 raft_consensus.cc:493] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:01.651996 15664 raft_consensus.cc:3060] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.652141 15650 leader_election.cc:290] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.653347 15647 tablet_bootstrap.cc:492] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d: No bootstrap required, opened a new log
I20260709 12:30:01.653462 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c6479826a143d2a68b6d9ff6fc56c5" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:30:01.654032 15647 ts_tablet_manager.cc:1403] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent bootstrapping tablet: real 0.098s	user 0.031s	sys 0.046s
I20260709 12:30:01.654203 15423 raft_consensus.cc:3060] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.655910 15245 catalog_manager.cc:5697] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194). New cstate: current_term: 1 leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:01.657249 15647 raft_consensus.cc:359] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.658073 15647 raft_consensus.cc:385] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:01.658388 15647 raft_consensus.cc:740] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Initialized, Role: FOLLOWER
I20260709 12:30:01.659360 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c6479826a143d2a68b6d9ff6fc56c5" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:01.659329 15647 consensus_queue.cc:260] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.660292 15498 raft_consensus.cc:3060] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.660601 15664 raft_consensus.cc:515] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.662480 15647 ts_tablet_manager.cc:1434] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:30:01.663021 15664 leader_election.cc:290] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.664248 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbde0675b7ae445490115e46663ca4d8" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:01.664613 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbde0675b7ae445490115e46663ca4d8" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:30:01.664921 15497 raft_consensus.cc:3060] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.665210 15349 raft_consensus.cc:3060] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.669011 15650 raft_consensus.cc:493] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.670092 15498 raft_consensus.cc:2468] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 1.
I20260709 12:30:01.663965 15423 raft_consensus.cc:2468] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 1.
I20260709 12:30:01.671202 15313 leader_election.cc:304] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:30:01.669890 15650 raft_consensus.cc:515] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.672243 15497 raft_consensus.cc:2468] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 1.
I20260709 12:30:01.672243 15349 raft_consensus.cc:2468] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 1.
I20260709 12:30:01.673750 15388 leader_election.cc:304] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:01.673969 15650 leader_election.cc:290] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.674779 15664 raft_consensus.cc:2804] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:01.675508 15664 raft_consensus.cc:697] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Becoming Leader. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:30:01.676245 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:30:01.676627 15686 raft_consensus.cc:2804] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:01.677104 15423 raft_consensus.cc:2393] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16adde14591b47b6a9732fe4a957bd14 in current term 1: Already voted for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in this term.
I20260709 12:30:01.677382 15651 raft_consensus.cc:493] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.677958 15651 raft_consensus.cc:515] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.680172 15651 leader_election.cc:290] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.680657 15686 raft_consensus.cc:697] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Becoming Leader. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:30:01.680197 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:30:01.681054 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cd247fcd954875a222541b22e88e41" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:30:01.680837 15664 consensus_queue.cc:237] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.681510 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cd247fcd954875a222541b22e88e41" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:30:01.682070 15498 raft_consensus.cc:2393] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 16adde14591b47b6a9732fe4a957bd14 in current term 1: Already voted for candidate 5e95a763e18e4814893fad06e1ea9d80 in this term.
I20260709 12:30:01.682019 15423 raft_consensus.cc:2468] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
I20260709 12:30:01.682507 15497 raft_consensus.cc:2468] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
I20260709 12:30:01.683283 15312 leader_election.cc:304] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:01.683657 15313 leader_election.cc:304] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14; no voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:30:01.684155 15651 raft_consensus.cc:2804] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:01.684454 15650 raft_consensus.cc:3060] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.684584 15651 raft_consensus.cc:493] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:01.685294 15651 raft_consensus.cc:3060] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.685940 15686 consensus_queue.cc:237] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.695685 15686 raft_consensus.cc:493] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.704181 15246 catalog_manager.cc:5697] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194). New cstate: current_term: 1 leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:01.707449 15247 catalog_manager.cc:5697] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "16adde14591b47b6a9732fe4a957bd14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:01.709758 15686 raft_consensus.cc:515] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.712051 15686 leader_election.cc:290] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.713758 15650 raft_consensus.cc:2749] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:01.714071 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d468370aee07411eb5b47b1e09128986" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:30:01.714923 15423 raft_consensus.cc:2468] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
I20260709 12:30:01.715972 15312 leader_election.cc:304] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:01.716796 15651 raft_consensus.cc:515] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.717772 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d468370aee07411eb5b47b1e09128986" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80" is_pre_election: true
I20260709 12:30:01.718513 15498 raft_consensus.cc:2468] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 0.
I20260709 12:30:01.720321 15651 leader_election.cc:290] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.720739 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cd247fcd954875a222541b22e88e41" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:30:01.721038 15651 raft_consensus.cc:2804] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:01.721438 15423 raft_consensus.cc:3060] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.721832 15651 raft_consensus.cc:493] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:01.722266 15651 raft_consensus.cc:3060] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.726676 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cd247fcd954875a222541b22e88e41" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:01.727527 15497 raft_consensus.cc:3060] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.730139 15423 raft_consensus.cc:2468] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 1.
I20260709 12:30:01.731359 15312 leader_election.cc:304] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:01.731175 15651 raft_consensus.cc:515] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.732282 15650 raft_consensus.cc:2804] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:01.732795 15650 raft_consensus.cc:697] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Becoming Leader. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:30:01.733582 15651 leader_election.cc:290] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:01.733759 15650 consensus_queue.cc:237] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.734917 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d468370aee07411eb5b47b1e09128986" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:01.735564 15498 raft_consensus.cc:3060] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.736584 15497 raft_consensus.cc:2468] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 1.
I20260709 12:30:01.738149 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d468370aee07411eb5b47b1e09128986" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:30:01.738855 15423 raft_consensus.cc:3060] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.742835 15498 raft_consensus.cc:2468] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 1.
I20260709 12:30:01.744172 15313 leader_election.cc:304] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:30:01.744831 15423 raft_consensus.cc:2468] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 1.
I20260709 12:30:01.745100 15650 raft_consensus.cc:2804] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:01.745721 15650 raft_consensus.cc:697] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Becoming Leader. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:30:01.746649 15650 consensus_queue.cc:237] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.747758 15246 catalog_manager.cc:5697] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "16adde14591b47b6a9732fe4a957bd14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:01.761289 15246 catalog_manager.cc:5697] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "16adde14591b47b6a9732fe4a957bd14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:01.774408 15295 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:01.802147 15498 raft_consensus.cc:3060] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.808954 15664 consensus_queue.cc:1048] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:01.821696 15632 consensus_queue.cc:1048] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:01.913362 15663 raft_consensus.cc:493] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:01.913925 15663 raft_consensus.cc:515] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.915593 15663 leader_election.cc:290] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:30:01.916419 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c92e0c0305648ac83c7b229b8577fc8" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:30:01.916581 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c92e0c0305648ac83c7b229b8577fc8" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:30:01.917049 15349 raft_consensus.cc:2468] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 0.
I20260709 12:30:01.917201 15423 raft_consensus.cc:2468] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 0.
I20260709 12:30:01.917994 15461 leader_election.cc:304] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:30:01.918635 15663 raft_consensus.cc:2804] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:01.918943 15663 raft_consensus.cc:493] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:01.919224 15663 raft_consensus.cc:3060] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.924988 15663 raft_consensus.cc:515] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.926653 15663 leader_election.cc:290] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:30:01.927431 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c92e0c0305648ac83c7b229b8577fc8" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:30:01.927642 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c92e0c0305648ac83c7b229b8577fc8" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:30:01.928009 15349 raft_consensus.cc:3060] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.928221 15423 raft_consensus.cc:3060] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:01.933022 15349 raft_consensus.cc:2468] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 1.
I20260709 12:30:01.933085 15423 raft_consensus.cc:2468] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 1.
I20260709 12:30:01.934309 15461 leader_election.cc:304] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:30:01.935029 15663 raft_consensus.cc:2804] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:01.935894 15663 raft_consensus.cc:697] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Becoming Leader. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:30:01.936594 15663 consensus_queue.cc:237] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:01.944344 15246 catalog_manager.cc:5697] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "5e95a763e18e4814893fad06e1ea9d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:01.970918 15632 consensus_queue.cc:1048] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:01.983654 15664 consensus_queue.cc:1048] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:02.079247 15632 consensus_queue.cc:1048] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:02.081454 15632 consensus_queue.cc:1048] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:02.092545 15632 consensus_queue.cc:1048] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:02.100844 15664 consensus_queue.cc:1048] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:02.118432 15650 consensus_queue.cc:1048] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:02.132440 15686 consensus_queue.cc:1048] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:02.146234 15686 consensus_queue.cc:1048] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:02.157505 15650 consensus_queue.cc:1048] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:02.219141 15650 consensus_queue.cc:1048] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:02.230973 15686 consensus_queue.cc:1048] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:02.378983 15663 consensus_queue.cc:1048] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:02.391131 15699 consensus_queue.cc:1048] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:30:02.781287 15295 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:03.032083 15699 raft_consensus.cc:493] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:03.032805 15699 raft_consensus.cc:515] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.034860 15699 leader_election.cc:290] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:30:03.035538 15347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14" is_pre_election: true
I20260709 12:30:03.036140 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" is_pre_election: true
I20260709 12:30:03.036345 15347 raft_consensus.cc:2468] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 1.
I20260709 12:30:03.036854 15423 raft_consensus.cc:2468] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 1.
I20260709 12:30:03.037708 15461 leader_election.cc:304] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:30:03.038429 15699 raft_consensus.cc:2804] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:03.038808 15699 raft_consensus.cc:493] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:03.039101 15699 raft_consensus.cc:3060] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.044577 15699 raft_consensus.cc:515] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.046355 15699 leader_election.cc:290] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 2 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:30:03.047228 15347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:30:03.047399 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:30:03.048013 15347 raft_consensus.cc:3060] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.048041 15423 raft_consensus.cc:3060] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.055277 15423 raft_consensus.cc:2468] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 2.
I20260709 12:30:03.055277 15347 raft_consensus.cc:2468] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 2.
I20260709 12:30:03.056507 15461 leader_election.cc:304] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:30:03.057468 15699 raft_consensus.cc:2804] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:03.057962 15699 raft_consensus.cc:697] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 LEADER]: Becoming Leader. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:30:03.058772 15699 consensus_queue.cc:237] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.067026 15246 catalog_manager.cc:5697] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 reported cstate change: term changed from 0 to 2, leader changed from <none> to 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "5e95a763e18e4814893fad06e1ea9d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:03.114280 15496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c92e0c0305648ac83c7b229b8577fc8"
dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
mode: GRACEFUL
new_leader_uuid: "16adde14591b47b6a9732fe4a957bd14"
 from {username='slave'} at 127.0.0.1:54066
I20260709 12:30:03.114912 15496 raft_consensus.cc:606] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Received request to transfer leadership to TS 16adde14591b47b6a9732fe4a957bd14
I20260709 12:30:03.120558 15423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03709ab5a8864cadb7b3242b2855ab95"
dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
mode: GRACEFUL
new_leader_uuid: "16adde14591b47b6a9732fe4a957bd14"
 from {username='slave'} at 127.0.0.1:38700
I20260709 12:30:03.121094 15423 raft_consensus.cc:606] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Received request to transfer leadership to TS 16adde14591b47b6a9732fe4a957bd14
I20260709 12:30:03.126334 15423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9feb97705d048d688949803485eb490"
dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
mode: GRACEFUL
new_leader_uuid: "16adde14591b47b6a9732fe4a957bd14"
 from {username='slave'} at 127.0.0.1:38700
I20260709 12:30:03.126814 15423 raft_consensus.cc:606] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Received request to transfer leadership to TS 16adde14591b47b6a9732fe4a957bd14
I20260709 12:30:03.133361 15423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e7dba2e76ec54b77ac5733aaa7b716e5"
dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
mode: GRACEFUL
new_leader_uuid: "16adde14591b47b6a9732fe4a957bd14"
 from {username='slave'} at 127.0.0.1:38700
I20260709 12:30:03.133940 15423 raft_consensus.cc:606] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Received request to transfer leadership to TS 16adde14591b47b6a9732fe4a957bd14
I20260709 12:30:03.139180 15347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d468370aee07411eb5b47b1e09128986"
dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
mode: GRACEFUL
new_leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
 from {username='slave'} at 127.0.0.1:57956
I20260709 12:30:03.139777 15347 raft_consensus.cc:606] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Received request to transfer leadership to TS 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:30:03.145516 15347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05c6479826a143d2a68b6d9ff6fc56c5"
dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
mode: GRACEFUL
new_leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
 from {username='slave'} at 127.0.0.1:57956
I20260709 12:30:03.146083 15347 raft_consensus.cc:606] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Received request to transfer leadership to TS 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:30:03.154359 15496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92"
dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
mode: GRACEFUL
new_leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
 from {username='slave'} at 127.0.0.1:54066
I20260709 12:30:03.155014 15496 raft_consensus.cc:606] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 LEADER]: Received request to transfer leadership to TS 9f1ed34c92d5440fa430a0d48f8fb05d
I20260709 12:30:03.160337 15347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48cd247fcd954875a222541b22e88e41"
dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
mode: GRACEFUL
new_leader_uuid: "5e95a763e18e4814893fad06e1ea9d80"
 from {username='slave'} at 127.0.0.1:57956
I20260709 12:30:03.160872 15347 raft_consensus.cc:606] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Received request to transfer leadership to TS 5e95a763e18e4814893fad06e1ea9d80
I20260709 12:30:03.232714 15664 raft_consensus.cc:993] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d: : Instructing follower 16adde14591b47b6a9732fe4a957bd14 to start an election
I20260709 12:30:03.233223 15665 raft_consensus.cc:1081] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Signalling peer 16adde14591b47b6a9732fe4a957bd14 to start an election
I20260709 12:30:03.234839 15347 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03709ab5a8864cadb7b3242b2855ab95"
dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
 from {username='slave'} at 127.0.0.1:57940
I20260709 12:30:03.235325 15347 raft_consensus.cc:493] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:03.235605 15347 raft_consensus.cc:3060] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.242141 15347 raft_consensus.cc:515] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.244354 15347 leader_election.cc:290] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:03.245607 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03709ab5a8864cadb7b3242b2855ab95" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:30:03.245592 15496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03709ab5a8864cadb7b3242b2855ab95" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:03.246601 15423 raft_consensus.cc:3055] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:03.246606 15496 raft_consensus.cc:3060] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.247680 15423 raft_consensus.cc:740] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:30:03.248430 15423 consensus_queue.cc:260] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.249356 15423 raft_consensus.cc:3060] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.252835 15496 raft_consensus.cc:2468] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 2.
I20260709 12:30:03.254190 15313 leader_election.cc:304] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:30:03.254246 15423 raft_consensus.cc:2468] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 2.
I20260709 12:30:03.255018 15691 raft_consensus.cc:2804] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:03.255903 15691 raft_consensus.cc:697] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Becoming Leader. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:30:03.256801 15691 consensus_queue.cc:237] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.266223 15246 catalog_manager.cc:5697] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 reported cstate change: term changed from 1 to 2, leader changed from 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194) to 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "16adde14591b47b6a9732fe4a957bd14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:03.270879 15721 raft_consensus.cc:993] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14: : Instructing follower 5e95a763e18e4814893fad06e1ea9d80 to start an election
I20260709 12:30:03.271453 15721 raft_consensus.cc:1081] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Signalling peer 5e95a763e18e4814893fad06e1ea9d80 to start an election
I20260709 12:30:03.273175 15496 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48cd247fcd954875a222541b22e88e41"
dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
 from {username='slave'} at 127.0.0.1:54042
I20260709 12:30:03.273757 15496 raft_consensus.cc:493] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:03.274082 15496 raft_consensus.cc:3060] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.280174 15496 raft_consensus.cc:515] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.282143 15496 leader_election.cc:290] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 2 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:30:03.282840 15347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cd247fcd954875a222541b22e88e41" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:30:03.283280 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48cd247fcd954875a222541b22e88e41" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:30:03.283491 15347 raft_consensus.cc:3055] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:03.283874 15423 raft_consensus.cc:3060] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.284660 15691 raft_consensus.cc:993] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14: : Instructing follower 9f1ed34c92d5440fa430a0d48f8fb05d to start an election
I20260709 12:30:03.285086 15691 raft_consensus.cc:1081] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Signalling peer 9f1ed34c92d5440fa430a0d48f8fb05d to start an election
I20260709 12:30:03.283911 15347 raft_consensus.cc:740] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:30:03.286567 15424 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d468370aee07411eb5b47b1e09128986"
dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
 from {username='slave'} at 127.0.0.1:38674
I20260709 12:30:03.286415 15347 consensus_queue.cc:260] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.287127 15424 raft_consensus.cc:493] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:03.287492 15424 raft_consensus.cc:3060] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.287657 15347 raft_consensus.cc:3060] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.290639 15423 raft_consensus.cc:2468] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 2.
I20260709 12:30:03.291735 15461 leader_election.cc:304] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:03.292713 15663 raft_consensus.cc:2804] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:03.293190 15663 raft_consensus.cc:697] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 LEADER]: Becoming Leader. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:30:03.294087 15663 consensus_queue.cc:237] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.294643 15347 raft_consensus.cc:2468] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 2.
I20260709 12:30:03.294013 15424 raft_consensus.cc:515] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.299226 15424 leader_election.cc:290] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 2 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:03.300082 15496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d468370aee07411eb5b47b1e09128986" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:03.300693 15496 raft_consensus.cc:3060] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.301923 15347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d468370aee07411eb5b47b1e09128986" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:30:03.302479 15347 raft_consensus.cc:3055] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:03.302760 15347 raft_consensus.cc:740] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:30:03.303515 15347 consensus_queue.cc:260] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.304889 15347 raft_consensus.cc:3060] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.306578 15246 catalog_manager.cc:5697] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 reported cstate change: term changed from 1 to 2, leader changed from 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193) to 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195). New cstate: current_term: 2 leader_uuid: "5e95a763e18e4814893fad06e1ea9d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:03.308353 15496 raft_consensus.cc:2468] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 2.
I20260709 12:30:03.309659 15388 leader_election.cc:304] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:03.310431 15665 raft_consensus.cc:2804] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:03.310874 15665 raft_consensus.cc:697] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 LEADER]: Becoming Leader. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:30:03.311672 15665 consensus_queue.cc:237] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.312455 15347 raft_consensus.cc:2468] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 2.
I20260709 12:30:03.319556 15246 catalog_manager.cc:5697] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d reported cstate change: term changed from 1 to 2, leader changed from 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193) to 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194). New cstate: current_term: 2 leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:03.437494 15699 raft_consensus.cc:993] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80: : Instructing follower 16adde14591b47b6a9732fe4a957bd14 to start an election
I20260709 12:30:03.437974 15663 raft_consensus.cc:1081] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Signalling peer 16adde14591b47b6a9732fe4a957bd14 to start an election
I20260709 12:30:03.439579 15347 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c92e0c0305648ac83c7b229b8577fc8"
dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
 from {username='slave'} at 127.0.0.1:57924
I20260709 12:30:03.440251 15347 raft_consensus.cc:493] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:03.440584 15347 raft_consensus.cc:3060] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.446712 15347 raft_consensus.cc:515] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.448388 15347 leader_election.cc:290] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:03.449035 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c92e0c0305648ac83c7b229b8577fc8" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:30:03.449524 15496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c92e0c0305648ac83c7b229b8577fc8" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:03.449697 15423 raft_consensus.cc:3060] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.450150 15496 raft_consensus.cc:3055] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:03.450454 15496 raft_consensus.cc:740] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:30:03.451165 15496 consensus_queue.cc:260] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.452116 15496 raft_consensus.cc:3060] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.454625 15423 raft_consensus.cc:2468] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 2.
I20260709 12:30:03.455602 15312 leader_election.cc:304] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:03.456318 15691 raft_consensus.cc:2804] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:03.456724 15691 raft_consensus.cc:697] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Becoming Leader. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:30:03.457484 15691 consensus_queue.cc:237] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.457821 15496 raft_consensus.cc:2468] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 2.
I20260709 12:30:03.465080 15246 catalog_manager.cc:5697] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 reported cstate change: term changed from 1 to 2, leader changed from 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195) to 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "16adde14591b47b6a9732fe4a957bd14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:03.529810 15664 raft_consensus.cc:993] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d: : Instructing follower 16adde14591b47b6a9732fe4a957bd14 to start an election
I20260709 12:30:03.530187 15664 raft_consensus.cc:1081] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Signalling peer 16adde14591b47b6a9732fe4a957bd14 to start an election
I20260709 12:30:03.531574 15347 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e7dba2e76ec54b77ac5733aaa7b716e5"
dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
 from {username='slave'} at 127.0.0.1:57940
I20260709 12:30:03.532111 15347 raft_consensus.cc:493] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:03.532421 15347 raft_consensus.cc:3060] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.537935 15347 raft_consensus.cc:515] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.540231 15347 leader_election.cc:290] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:03.540764 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7dba2e76ec54b77ac5733aaa7b716e5" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:30:03.541397 15423 raft_consensus.cc:3055] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:03.541772 15423 raft_consensus.cc:740] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:30:03.541548 15496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7dba2e76ec54b77ac5733aaa7b716e5" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:03.542361 15496 raft_consensus.cc:3060] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.542685 15423 consensus_queue.cc:260] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.543676 15423 raft_consensus.cc:3060] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.548014 15496 raft_consensus.cc:2468] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 2.
I20260709 12:30:03.548713 15423 raft_consensus.cc:2468] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 2.
I20260709 12:30:03.549528 15313 leader_election.cc:304] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:30:03.550518 15691 raft_consensus.cc:2804] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:03.550956 15691 raft_consensus.cc:697] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Becoming Leader. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:30:03.551673 15691 consensus_queue.cc:237] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.559449 15246 catalog_manager.cc:5697] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 reported cstate change: term changed from 1 to 2, leader changed from 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194) to 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "16adde14591b47b6a9732fe4a957bd14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:03.669373 15423 raft_consensus.cc:1275] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Refusing update from remote peer 5e95a763e18e4814893fad06e1ea9d80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:30:03.669377 15347 raft_consensus.cc:1275] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Refusing update from remote peer 5e95a763e18e4814893fad06e1ea9d80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:30:03.671101 15699 consensus_queue.cc:1048] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:03.671972 15663 consensus_queue.cc:1048] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:03.679724 15699 raft_consensus.cc:993] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80: : Instructing follower 9f1ed34c92d5440fa430a0d48f8fb05d to start an election
I20260709 12:30:03.681725 15729 raft_consensus.cc:1081] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 LEADER]: Signalling peer 9f1ed34c92d5440fa430a0d48f8fb05d to start an election
I20260709 12:30:03.684103 15424 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92"
dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:30:03.700639 15424 raft_consensus.cc:493] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:03.701126 15424 raft_consensus.cc:3060] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:03.703640 15665 raft_consensus.cc:993] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d: : Instructing follower 16adde14591b47b6a9732fe4a957bd14 to start an election
I20260709 12:30:03.704288 15664 raft_consensus.cc:1081] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Signalling peer 16adde14591b47b6a9732fe4a957bd14 to start an election
I20260709 12:30:03.705937 15349 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9feb97705d048d688949803485eb490"
dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
 from {username='slave'} at 127.0.0.1:57940
I20260709 12:30:03.706544 15349 raft_consensus.cc:493] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:03.706955 15349 raft_consensus.cc:3060] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.707814 15721 raft_consensus.cc:993] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14: : Instructing follower 9f1ed34c92d5440fa430a0d48f8fb05d to start an election
I20260709 12:30:03.708336 15721 raft_consensus.cc:1081] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Signalling peer 9f1ed34c92d5440fa430a0d48f8fb05d to start an election
I20260709 12:30:03.710137 15422 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05c6479826a143d2a68b6d9ff6fc56c5"
dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
 from {username='slave'} at 127.0.0.1:38674
I20260709 12:30:03.710691 15422 raft_consensus.cc:493] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:03.710760 15497 raft_consensus.cc:1275] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Refusing update from remote peer 16adde14591b47b6a9732fe4a957bd14: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:03.711189 15422 raft_consensus.cc:3060] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.715443 15424 raft_consensus.cc:515] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.718076 15691 consensus_queue.cc:1048] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:03.719334 15422 raft_consensus.cc:515] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.723081 15497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c6479826a143d2a68b6d9ff6fc56c5" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:03.723713 15497 raft_consensus.cc:3060] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.723399 15349 raft_consensus.cc:515] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.725430 15346 raft_consensus.cc:1275] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Refusing update from remote peer 9f1ed34c92d5440fa430a0d48f8fb05d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:03.726830 15349 leader_election.cc:290] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:03.727043 15423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9feb97705d048d688949803485eb490" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:30:03.727654 15423 raft_consensus.cc:3055] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:03.727927 15423 raft_consensus.cc:740] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:30:03.728159 15498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9feb97705d048d688949803485eb490" candidate_uuid: "16adde14591b47b6a9732fe4a957bd14" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:03.728847 15498 raft_consensus.cc:3060] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.731257 15424 leader_election.cc:290] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 3 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:03.731387 15665 consensus_queue.cc:1048] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:03.732438 15499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
I20260709 12:30:03.733100 15499 raft_consensus.cc:3055] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:03.733409 15499 raft_consensus.cc:740] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:30:03.728667 15423 consensus_queue.cc:260] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.734318 15499 consensus_queue.cc:260] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.735687 15499 raft_consensus.cc:3060] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:03.722748 15347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05c6479826a143d2a68b6d9ff6fc56c5" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:30:03.734828 15423 raft_consensus.cc:3060] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.739009 15424 raft_consensus.cc:1275] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Refusing update from remote peer 5e95a763e18e4814893fad06e1ea9d80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:03.740708 15729 consensus_queue.cc:1048] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:03.741722 15422 leader_election.cc:290] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 2 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195:38943)
I20260709 12:30:03.742082 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2105bdf1d73b43cfa43a714a15343c92" candidate_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:30:03.742688 15349 raft_consensus.cc:3060] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:03.738948 15347 raft_consensus.cc:3055] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:03.743857 15347 raft_consensus.cc:740] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:30:03.744944 15347 consensus_queue.cc:260] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.746380 15347 raft_consensus.cc:3060] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.748500 15497 raft_consensus.cc:2468] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 2.
I20260709 12:30:03.750602 15388 leader_election.cc:304] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:03.751833 15499 raft_consensus.cc:2468] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 3.
I20260709 12:30:03.752107 15665 raft_consensus.cc:2804] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:03.752631 15665 raft_consensus.cc:697] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 LEADER]: Becoming Leader. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:30:03.753217 15388 leader_election.cc:304] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e95a763e18e4814893fad06e1ea9d80, 9f1ed34c92d5440fa430a0d48f8fb05d; no voters: 
I20260709 12:30:03.753515 15665 consensus_queue.cc:237] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.758138 15664 raft_consensus.cc:2804] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:03.758805 15664 raft_consensus.cc:697] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 3 LEADER]: Becoming Leader. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:30:03.759932 15664 consensus_queue.cc:237] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.762490 15423 raft_consensus.cc:2468] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 2.
I20260709 12:30:03.772862 15349 raft_consensus.cc:2468] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 3.
I20260709 12:30:03.769109 15498 raft_consensus.cc:2468] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16adde14591b47b6a9732fe4a957bd14 in term 2.
I20260709 12:30:03.772817 15347 raft_consensus.cc:2468] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ed34c92d5440fa430a0d48f8fb05d in term 2.
I20260709 12:30:03.826988 15313 leader_election.cc:304] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
W20260709 12:30:03.818817 15387 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.193:45781, user_credentials={real_user=slave}} blocked reactor thread for 55228.1us
I20260709 12:30:03.828754 15691 raft_consensus.cc:2804] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:03.832074 15691 raft_consensus.cc:697] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Becoming Leader. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:30:03.837961 15423 raft_consensus.cc:1275] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Refusing update from remote peer 16adde14591b47b6a9732fe4a957bd14: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:03.839205 15496 raft_consensus.cc:1275] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Refusing update from remote peer 9f1ed34c92d5440fa430a0d48f8fb05d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:03.833093 15691 consensus_queue.cc:237] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:03.843534 15349 raft_consensus.cc:1275] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Refusing update from remote peer 5e95a763e18e4814893fad06e1ea9d80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:03.850188 15247 catalog_manager.cc:5697] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d reported cstate change: term changed from 1 to 2, leader changed from 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193) to 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194). New cstate: current_term: 2 leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:03.855144 15247 catalog_manager.cc:5697] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d reported cstate change: term changed from 2 to 3, leader changed from 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195) to 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194). New cstate: current_term: 3 leader_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:03.863430 15729 consensus_queue.cc:1048] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:03.852304 15665 consensus_queue.cc:1048] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:30:03.885092 15721 consensus_queue.cc:1048] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:03.895648 15246 catalog_manager.cc:5697] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 reported cstate change: term changed from 1 to 2, leader changed from 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194) to 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193). New cstate: current_term: 2 leader_uuid: "16adde14591b47b6a9732fe4a957bd14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:03.906524 15312 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.89.194:33189, user_credentials={real_user=slave}} blocked reactor thread for 48229.5us
I20260709 12:30:03.917791 15422 raft_consensus.cc:1275] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Refusing update from remote peer 16adde14591b47b6a9732fe4a957bd14: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:03.930903 15721 consensus_queue.cc:1048] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:03.934603 15496 raft_consensus.cc:1275] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Refusing update from remote peer 16adde14591b47b6a9732fe4a957bd14: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:03.938045 15721 consensus_queue.cc:1048] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:03.994805 15496 raft_consensus.cc:1275] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Refusing update from remote peer 16adde14591b47b6a9732fe4a957bd14: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:03.996569 15745 consensus_queue.cc:1048] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:04.006484 15422 raft_consensus.cc:1275] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Refusing update from remote peer 16adde14591b47b6a9732fe4a957bd14: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:04.008882 15745 consensus_queue.cc:1048] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:04.156718 15496 raft_consensus.cc:1275] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Refusing update from remote peer 9f1ed34c92d5440fa430a0d48f8fb05d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:04.158286 15665 consensus_queue.cc:1048] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:04.169344 15347 raft_consensus.cc:1275] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Refusing update from remote peer 9f1ed34c92d5440fa430a0d48f8fb05d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:04.171857 15665 consensus_queue.cc:1048] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:04.301541 15347 raft_consensus.cc:1275] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 3 FOLLOWER]: Refusing update from remote peer 9f1ed34c92d5440fa430a0d48f8fb05d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:30:04.302937 15665 consensus_queue.cc:1048] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:04.312891 15496 raft_consensus.cc:1275] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 FOLLOWER]: Refusing update from remote peer 9f1ed34c92d5440fa430a0d48f8fb05d: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:30:04.314567 15739 consensus_queue.cc:1048] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:04.406121 15422 raft_consensus.cc:1275] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Refusing update from remote peer 16adde14591b47b6a9732fe4a957bd14: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:04.407892 15721 consensus_queue.cc:1048] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:04.419982 15496 raft_consensus.cc:1275] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Refusing update from remote peer 16adde14591b47b6a9732fe4a957bd14: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:04.422178 15742 consensus_queue.cc:1048] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:06.164100 15246 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52770:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:30:06.184517 15246 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52770:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:30:06.237754  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:06.350934 15801 tablet.cc:1724] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.422152 15799 tablet.cc:1724] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:06.430673 15788 log.cc:927] Time spent T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:30:06.424111 15811 tablet.cc:1724] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:06.469950 15796 log.cc:927] Time spent T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:30:06.470546 15814 tablet.cc:1724] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.394887 15797 tablet.cc:1724] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:06.488454 15809 log.cc:927] Time spent T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:30:06.501436 15818 tablet.cc:1724] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:06.503561 15798 log.cc:927] Time spent T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:30:06.509709 15814 tablet.cc:1724] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.522456 15815 tablet.cc:1724] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.535848 15819 tablet.cc:1724] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.567126 15827 tablet.cc:1724] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:06.572590 15816 log.cc:927] Time spent T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d: Append to log took a long time: real 0.074s	user 0.003s	sys 0.000s
I20260709 12:30:06.585018 15814 tablet.cc:1724] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.598037 15349 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1"
dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
mode: GRACEFUL
new_leader_uuid: "5e95a763e18e4814893fad06e1ea9d80"
 from {username='slave'} at 127.0.0.1:56640
I20260709 12:30:06.598620 15349 raft_consensus.cc:606] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Received request to transfer leadership to TS 5e95a763e18e4814893fad06e1ea9d80
I20260709 12:30:06.542117 15817 tablet.cc:1724] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.591342 15801 tablet.cc:1724] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.607568 15818 tablet.cc:1724] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.612697 15799 tablet.cc:1724] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.615653 15826 tablet.cc:1724] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.619437 15246 catalog_manager.cc:5789] TS 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781) does not have the latest schema for tablet 5c92e0c0305648ac83c7b229b8577fc8 (table soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]). Expected version 1 got 0
I20260709 12:30:06.643527 15811 tablet.cc:1724] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.679316 15827 tablet.cc:1724] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.710323 15247 catalog_manager.cc:5789] TS 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189) does not have the latest schema for tablet 05c6479826a143d2a68b6d9ff6fc56c5 (table soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]). Expected version 1 got 0
I20260709 12:30:06.711295 15247 catalog_manager.cc:5789] TS 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189) does not have the latest schema for tablet 2105bdf1d73b43cfa43a714a15343c92 (table soft_deleted_table [id=7afca1411b744af8b942e44846f30c69]). Expected version 1 got 0
I20260709 12:30:06.715322 15817 tablet.cc:1724] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.699834 15801 tablet.cc:1724] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.721642 15826 tablet.cc:1724] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.723800 15818 tablet.cc:1724] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.694978 15797 tablet.cc:1724] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.737828 15829 tablet.cc:1724] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.689783 15819 tablet.cc:1724] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.750047 15797 tablet.cc:1724] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.751173 15819 tablet.cc:1724] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:06.770325 15784 raft_consensus.cc:993] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14: : Instructing follower 5e95a763e18e4814893fad06e1ea9d80 to start an election
I20260709 12:30:06.770916 15784 raft_consensus.cc:1081] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Signalling peer 5e95a763e18e4814893fad06e1ea9d80 to start an election
I20260709 12:30:06.772940 15499 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1"
dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
 from {username='slave'} at 127.0.0.1:54042
I20260709 12:30:06.777669 15499 raft_consensus.cc:493] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:06.778193 15499 raft_consensus.cc:3060] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:06.793253 15422 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fdaa067b5184307b4e22d101add7e12"
dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
mode: GRACEFUL
new_leader_uuid: "5e95a763e18e4814893fad06e1ea9d80"
 from {username='slave'} at 127.0.0.1:57378
I20260709 12:30:06.794317 15422 raft_consensus.cc:606] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 LEADER]: Received request to transfer leadership to TS 5e95a763e18e4814893fad06e1ea9d80
I20260709 12:30:06.796298  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:30:06.796973  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:06.798739 15499 raft_consensus.cc:515] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:30:06.800830 15499 leader_election.cc:290] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 3 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:30:06.801517 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:30:06.802354 15349 raft_consensus.cc:3055] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:06.802702 15349 raft_consensus.cc:740] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 16adde14591b47b6a9732fe4a957bd14, State: Running, Role: LEADER
I20260709 12:30:06.803460 15349 consensus_queue.cc:260] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:30:06.803655 15422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25474d56fa3c452ea0f06683eaf900f1" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:30:06.804884 15349 raft_consensus.cc:3060] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:06.805145 15422 raft_consensus.cc:3060] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:06.812052 15349 raft_consensus.cc:2468] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 3.
I20260709 12:30:06.813289 15422 raft_consensus.cc:2468] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 3.
I20260709 12:30:06.813688 15461 leader_election.cc:304] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:30:06.814698 15828 raft_consensus.cc:2804] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:06.815201 15828 raft_consensus.cc:697] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 LEADER]: Becoming Leader. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:30:06.816337 15828 consensus_queue.cc:237] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } }
I20260709 12:30:06.817209 15805 raft_consensus.cc:993] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d: : Instructing follower 5e95a763e18e4814893fad06e1ea9d80 to start an election
I20260709 12:30:06.812368 15247 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4: soft_deleted_table [id=7afca1411b744af8b942e44846f30c69] alter complete (version 1)
I20260709 12:30:06.817799 15805 raft_consensus.cc:1081] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 LEADER]: Signalling peer 5e95a763e18e4814893fad06e1ea9d80 to start an election
I20260709 12:30:06.822641 15499 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fdaa067b5184307b4e22d101add7e12"
dest_uuid: "5e95a763e18e4814893fad06e1ea9d80"
 from {username='slave'} at 127.0.0.1:54054
I20260709 12:30:06.823442 15499 raft_consensus.cc:493] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:06.823998 15499 raft_consensus.cc:3060] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:06.832639 15499 raft_consensus.cc:515] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:06.834558 15499 leader_election.cc:290] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 3 election: Requested vote from peers 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781), 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189)
I20260709 12:30:06.835171 15349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fdaa067b5184307b4e22d101add7e12" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "16adde14591b47b6a9732fe4a957bd14"
I20260709 12:30:06.835606 15422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fdaa067b5184307b4e22d101add7e12" candidate_uuid: "5e95a763e18e4814893fad06e1ea9d80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d"
I20260709 12:30:06.835793 15349 raft_consensus.cc:3060] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:06.834338 15247 catalog_manager.cc:5697] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 reported cstate change: term changed from 2 to 3, leader changed from 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193) to 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "5e95a763e18e4814893fad06e1ea9d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:06.836220 15422 raft_consensus.cc:3055] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:06.836570 15422 raft_consensus.cc:740] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9f1ed34c92d5440fa430a0d48f8fb05d, State: Running, Role: LEADER
I20260709 12:30:06.837384 15422 consensus_queue.cc:260] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:06.838780 15422 raft_consensus.cc:3060] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:06.843173 15349 raft_consensus.cc:2468] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 3.
I20260709 12:30:06.844476 15461 leader_election.cc:304] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16adde14591b47b6a9732fe4a957bd14, 5e95a763e18e4814893fad06e1ea9d80; no voters: 
I20260709 12:30:06.845207 15828 raft_consensus.cc:2804] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:06.845605 15422 raft_consensus.cc:2468] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e95a763e18e4814893fad06e1ea9d80 in term 3.
I20260709 12:30:06.845649 15828 raft_consensus.cc:697] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 LEADER]: Becoming Leader. State: Replica: 5e95a763e18e4814893fad06e1ea9d80, State: Running, Role: LEADER
I20260709 12:30:06.846798 15828 consensus_queue.cc:237] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } }
I20260709 12:30:06.853742 15247 catalog_manager.cc:5697] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 reported cstate change: term changed from 2 to 3, leader changed from 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194) to 5e95a763e18e4814893fad06e1ea9d80 (127.3.89.195). New cstate: current_term: 3 leader_uuid: "5e95a763e18e4814893fad06e1ea9d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e95a763e18e4814893fad06e1ea9d80" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38943 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:07.243563 15422 raft_consensus.cc:1275] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 3 FOLLOWER]: Refusing update from remote peer 5e95a763e18e4814893fad06e1ea9d80: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:07.244704 15806 consensus_queue.cc:1048] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:07.254055 15349 raft_consensus.cc:1275] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [term 3 FOLLOWER]: Refusing update from remote peer 5e95a763e18e4814893fad06e1ea9d80: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:07.255679 15828 consensus_queue.cc:1048] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:07.311923 15422 raft_consensus.cc:1275] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 3 FOLLOWER]: Refusing update from remote peer 5e95a763e18e4814893fad06e1ea9d80: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:07.313268 15828 consensus_queue.cc:1048] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1ed34c92d5440fa430a0d48f8fb05d" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 33189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:07.322556 15349 raft_consensus.cc:1275] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 3 FOLLOWER]: Refusing update from remote peer 5e95a763e18e4814893fad06e1ea9d80: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:07.324003 15839 consensus_queue.cc:1048] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16adde14591b47b6a9732fe4a957bd14" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 45781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:07.798318  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:07.806253  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:07.806774  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:08.807891  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:08.812932  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:08.813351  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:09.814397  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:09.820066  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:09.820596  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:10.821698  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:10.826752  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:10.827247  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:11.828365  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:11.833983  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:11.834575  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:12.835616  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:12.840994  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:12.841495  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:13.842597  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:13.848752  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:13.849257  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:14.850339  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:14.855558  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:14.856076  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:16.707445  3431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:16.712824  3431 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:16.713346  3431 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:30:17.041599 15520 debug-util.cc:398] Leaking SignalData structure 0x7b080021de80 after lost signal to thread 3434
W20260709 12:30:17.042704 15520 debug-util.cc:398] Leaking SignalData structure 0x7b080009af80 after lost signal to thread 15281
W20260709 12:30:17.043944 15520 debug-util.cc:398] Leaking SignalData structure 0x7b080021d6c0 after lost signal to thread 15373
W20260709 12:30:17.045019 15520 debug-util.cc:398] Leaking SignalData structure 0x7b08000ebaa0 after lost signal to thread 15448
W20260709 12:30:17.046155 15520 debug-util.cc:398] Leaking SignalData structure 0x7b08000fbb40 after lost signal to thread 15523
I20260709 12:30:17.801801  3431 tablet_server.cc:179] TabletServer@127.3.89.193:0 shutting down...
W20260709 12:30:17.817350 15387 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 26 similar messages]
W20260709 12:30:17.821300 15387 consensus_peers.cc:597] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d -> Peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Couldn't send request to peer 16adde14591b47b6a9732fe4a957bd14. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:17.873342  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:17.874816  3431 tablet_replica.cc:333] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.876507  3431 raft_consensus.cc:2243] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:17.877794  3431 raft_consensus.cc:2272] T 48cd247fcd954875a222541b22e88e41 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.882627  3431 tablet_replica.cc:333] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.883862  3431 raft_consensus.cc:2243] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:17.884827  3431 raft_consensus.cc:2272] T 2105bdf1d73b43cfa43a714a15343c92 P 16adde14591b47b6a9732fe4a957bd14 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.890122  3431 tablet_replica.cc:333] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.891597  3431 raft_consensus.cc:2243] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:17.892787  3431 raft_consensus.cc:2272] T d468370aee07411eb5b47b1e09128986 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:17.893368 15387 consensus_peers.cc:597] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d -> Peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Couldn't send request to peer 16adde14591b47b6a9732fe4a957bd14. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:17.897598  3431 tablet_replica.cc:333] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.898828  3431 raft_consensus.cc:2243] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:17.900105  3431 raft_consensus.cc:2272] T 05c6479826a143d2a68b6d9ff6fc56c5 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:17.903877 15461 consensus_peers.cc:597] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 -> Peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Couldn't send request to peer 16adde14591b47b6a9732fe4a957bd14. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:17.905133  3431 tablet_replica.cc:333] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.906301  3431 raft_consensus.cc:2243] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:17.908586  3431 raft_consensus.cc:2272] T 03709ab5a8864cadb7b3242b2855ab95 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.913029  3431 tablet_replica.cc:333] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.913955  3431 raft_consensus.cc:2243] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:17.915488  3431 raft_consensus.cc:2272] T 5c92e0c0305648ac83c7b229b8577fc8 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.918615  3431 tablet_replica.cc:333] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.919309  3431 raft_consensus.cc:2243] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:17.920467  3431 raft_consensus.cc:2272] T e48cab88c8f84967baa4e76847f252da P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.923714  3431 tablet_replica.cc:333] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.924398  3431 raft_consensus.cc:2243] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:17.925170  3431 raft_consensus.cc:2272] T 4fdaa067b5184307b4e22d101add7e12 P 16adde14591b47b6a9732fe4a957bd14 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.928736  3431 tablet_replica.cc:333] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.929461  3431 raft_consensus.cc:2243] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:17.930423  3431 raft_consensus.cc:2272] T eb71faa206ff442ea23f3a704dda68de P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.934374  3431 tablet_replica.cc:333] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.935119  3431 raft_consensus.cc:2243] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:17.935851  3431 raft_consensus.cc:2272] T cbde0675b7ae445490115e46663ca4d8 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.938781  3431 tablet_replica.cc:333] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.939466  3431 raft_consensus.cc:2243] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:17.940670  3431 raft_consensus.cc:2272] T a9feb97705d048d688949803485eb490 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.943743  3431 tablet_replica.cc:333] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.944527  3431 raft_consensus.cc:2243] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:17.945300  3431 raft_consensus.cc:2272] T cb09d8084bee4073bc05ba1ac0899d04 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.948212  3431 tablet_replica.cc:333] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.948837  3431 raft_consensus.cc:2243] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:17.950348  3431 raft_consensus.cc:2272] T 1e1136e261f24c7b84f1331f507d74f5 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.953490  3431 tablet_replica.cc:333] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.954159  3431 raft_consensus.cc:2243] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:17.955196  3431 raft_consensus.cc:2272] T 71e7aac918ff414b80c722862b23d801 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.958148  3431 tablet_replica.cc:333] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.958808  3431 raft_consensus.cc:2243] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:17.959434  3431 raft_consensus.cc:2272] T a65d04ca2b97426c86a48fa9fd9825c2 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.962302  3431 tablet_replica.cc:333] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.962900  3431 raft_consensus.cc:2243] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:17.963953  3431 raft_consensus.cc:2272] T e7dba2e76ec54b77ac5733aaa7b716e5 P 16adde14591b47b6a9732fe4a957bd14 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.966838  3431 tablet_replica.cc:333] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.967334  3431 raft_consensus.cc:2243] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:17.967895  3431 raft_consensus.cc:2272] T a81f9e32762c40f0bc6a7faa29e39744 P 16adde14591b47b6a9732fe4a957bd14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:17.970976  3431 tablet_replica.cc:333] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14: stopping tablet replica
I20260709 12:30:17.971529  3431 raft_consensus.cc:2243] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:17.972160  3431 raft_consensus.cc:2272] T 25474d56fa3c452ea0f06683eaf900f1 P 16adde14591b47b6a9732fe4a957bd14 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:17.992625 15461 consensus_peers.cc:597] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 -> Peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Couldn't send request to peer 16adde14591b47b6a9732fe4a957bd14. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:18.020816 15387 consensus_peers.cc:597] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d -> Peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Couldn't send request to peer 16adde14591b47b6a9732fe4a957bd14. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:18.021940  3431 tablet_server.cc:196] TabletServer@127.3.89.193:0 shutdown complete.
W20260709 12:30:18.083039 15461 consensus_peers.cc:597] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 -> Peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Couldn't send request to peer 16adde14591b47b6a9732fe4a957bd14. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:18.100553  3431 tablet_server.cc:179] TabletServer@127.3.89.194:0 shutting down...
W20260709 12:30:18.113359 15461 consensus_peers.cc:597] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 -> Peer 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189): Couldn't send request to peer 9f1ed34c92d5440fa430a0d48f8fb05d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:18.143002 15387 consensus_peers.cc:597] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d -> Peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Couldn't send request to peer 16adde14591b47b6a9732fe4a957bd14. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:18.145756 15387 consensus_peers.cc:597] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d -> Peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Couldn't send request to peer 16adde14591b47b6a9732fe4a957bd14. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:18.150781  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:18.152020  3431 tablet_replica.cc:333] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.153462  3431 raft_consensus.cc:2243] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.154551  3431 raft_consensus.cc:2272] T 48cd247fcd954875a222541b22e88e41 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.158679  3431 tablet_replica.cc:333] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.159931  3431 raft_consensus.cc:2243] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:18.161541  3431 raft_consensus.cc:2272] T 05c6479826a143d2a68b6d9ff6fc56c5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.166321  3431 tablet_replica.cc:333] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.167425  3431 raft_consensus.cc:2243] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.168383  3431 raft_consensus.cc:2272] T 5c92e0c0305648ac83c7b229b8577fc8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.172693  3431 tablet_replica.cc:333] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.173686  3431 raft_consensus.cc:2243] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.174659  3431 raft_consensus.cc:2272] T 03709ab5a8864cadb7b3242b2855ab95 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.179045  3431 tablet_replica.cc:333] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.180269  3431 raft_consensus.cc:2243] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:18.181866  3431 raft_consensus.cc:2272] T d468370aee07411eb5b47b1e09128986 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.186057  3431 tablet_replica.cc:333] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.187091  3431 raft_consensus.cc:2243] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:18.188798  3431 raft_consensus.cc:2272] T eb71faa206ff442ea23f3a704dda68de P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.193097  3431 tablet_replica.cc:333] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.194267  3431 raft_consensus.cc:2243] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.195096  3431 raft_consensus.cc:2272] T e48cab88c8f84967baa4e76847f252da P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.199002  3431 tablet_replica.cc:333] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.199950  3431 raft_consensus.cc:2243] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.200596  3431 raft_consensus.cc:2272] T 4fdaa067b5184307b4e22d101add7e12 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.204596  3431 tablet_replica.cc:333] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.205480  3431 raft_consensus.cc:2243] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:18.206770  3431 raft_consensus.cc:2272] T a65d04ca2b97426c86a48fa9fd9825c2 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.210673  3431 tablet_replica.cc:333] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.211616  3431 raft_consensus.cc:2243] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.212563  3431 raft_consensus.cc:2272] T e7dba2e76ec54b77ac5733aaa7b716e5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.216749  3431 tablet_replica.cc:333] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.217675  3431 raft_consensus.cc:2243] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:18.219193  3431 raft_consensus.cc:2272] T a81f9e32762c40f0bc6a7faa29e39744 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.223294  3431 tablet_replica.cc:333] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.224237  3431 raft_consensus.cc:2243] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.224951  3431 raft_consensus.cc:2272] T 71e7aac918ff414b80c722862b23d801 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.227978  3431 tablet_replica.cc:333] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.228708  3431 raft_consensus.cc:2243] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.229413  3431 raft_consensus.cc:2272] T 25474d56fa3c452ea0f06683eaf900f1 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.232384  3431 tablet_replica.cc:333] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.233003  3431 raft_consensus.cc:2243] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:18.234308  3431 raft_consensus.cc:2272] T 2105bdf1d73b43cfa43a714a15343c92 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.237169  3431 tablet_replica.cc:333] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.237877  3431 raft_consensus.cc:2243] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:18.238973  3431 raft_consensus.cc:2272] T cbde0675b7ae445490115e46663ca4d8 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.242246  3431 tablet_replica.cc:333] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.243037  3431 raft_consensus.cc:2243] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.243868  3431 raft_consensus.cc:2272] T a9feb97705d048d688949803485eb490 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.247373  3431 tablet_replica.cc:333] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
W20260709 12:30:18.247558 15461 consensus_peers.cc:597] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 -> Peer 16adde14591b47b6a9732fe4a957bd14 (127.3.89.193:45781): Couldn't send request to peer 16adde14591b47b6a9732fe4a957bd14. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.193:45781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:18.248162  3431 raft_consensus.cc:2243] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.248873  3431 raft_consensus.cc:2272] T cb09d8084bee4073bc05ba1ac0899d04 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.251813  3431 tablet_replica.cc:333] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d: stopping tablet replica
I20260709 12:30:18.252409  3431 raft_consensus.cc:2243] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.252928  3431 raft_consensus.cc:2272] T 1e1136e261f24c7b84f1331f507d74f5 P 9f1ed34c92d5440fa430a0d48f8fb05d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.298233  3431 tablet_server.cc:196] TabletServer@127.3.89.194:0 shutdown complete.
I20260709 12:30:18.350991  3431 tablet_server.cc:179] TabletServer@127.3.89.195:0 shutting down...
W20260709 12:30:18.386036 15461 consensus_peers.cc:597] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 -> Peer 9f1ed34c92d5440fa430a0d48f8fb05d (127.3.89.194:33189): Couldn't send request to peer 9f1ed34c92d5440fa430a0d48f8fb05d. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.194:33189: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:18.396296  3431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:18.397492  3431 tablet_replica.cc:333] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.398773  3431 raft_consensus.cc:2243] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.399765  3431 raft_consensus.cc:2272] T 5c92e0c0305648ac83c7b229b8577fc8 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.403600  3431 tablet_replica.cc:333] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.404613  3431 raft_consensus.cc:2243] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.405503  3431 raft_consensus.cc:2272] T 03709ab5a8864cadb7b3242b2855ab95 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.409463  3431 tablet_replica.cc:333] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.410445  3431 raft_consensus.cc:2243] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.411230  3431 raft_consensus.cc:2272] T d468370aee07411eb5b47b1e09128986 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.414759  3431 tablet_replica.cc:333] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.415620  3431 raft_consensus.cc:2243] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.416384  3431 raft_consensus.cc:2272] T 05c6479826a143d2a68b6d9ff6fc56c5 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.419984  3431 tablet_replica.cc:333] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.421015  3431 raft_consensus.cc:2243] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:18.422343  3431 raft_consensus.cc:2272] T 48cd247fcd954875a222541b22e88e41 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.426151  3431 tablet_replica.cc:333] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.428139  3431 raft_consensus.cc:2243] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.428960  3431 raft_consensus.cc:2272] T a81f9e32762c40f0bc6a7faa29e39744 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.432838  3431 tablet_replica.cc:333] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.433916  3431 raft_consensus.cc:2243] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.434738  3431 raft_consensus.cc:2272] T e7dba2e76ec54b77ac5733aaa7b716e5 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.437997  3431 tablet_replica.cc:333] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.438926  3431 raft_consensus.cc:2243] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.439599  3431 raft_consensus.cc:2272] T e48cab88c8f84967baa4e76847f252da P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.443089  3431 tablet_replica.cc:333] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.443948  3431 raft_consensus.cc:2243] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:18.445271  3431 raft_consensus.cc:2272] T 4fdaa067b5184307b4e22d101add7e12 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.448530  3431 tablet_replica.cc:333] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.449393  3431 raft_consensus.cc:2243] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.450166  3431 raft_consensus.cc:2272] T eb71faa206ff442ea23f3a704dda68de P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.453372  3431 tablet_replica.cc:333] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.454252  3431 raft_consensus.cc:2243] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.454854  3431 raft_consensus.cc:2272] T a65d04ca2b97426c86a48fa9fd9825c2 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.458160  3431 tablet_replica.cc:333] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.459054  3431 raft_consensus.cc:2243] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.459690  3431 raft_consensus.cc:2272] T 71e7aac918ff414b80c722862b23d801 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.463116  3431 tablet_replica.cc:333] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.463980  3431 raft_consensus.cc:2243] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.464709  3431 raft_consensus.cc:2272] T cbde0675b7ae445490115e46663ca4d8 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.468040  3431 tablet_replica.cc:333] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.468796  3431 raft_consensus.cc:2243] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.469604  3431 raft_consensus.cc:2272] T a9feb97705d048d688949803485eb490 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.472728  3431 tablet_replica.cc:333] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.473487  3431 raft_consensus.cc:2243] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:18.474805  3431 raft_consensus.cc:2272] T cb09d8084bee4073bc05ba1ac0899d04 P 5e95a763e18e4814893fad06e1ea9d80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.478005  3431 tablet_replica.cc:333] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.478721  3431 raft_consensus.cc:2243] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.479413  3431 raft_consensus.cc:2272] T 1e1136e261f24c7b84f1331f507d74f5 P 5e95a763e18e4814893fad06e1ea9d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.482599  3431 tablet_replica.cc:333] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.483325  3431 raft_consensus.cc:2243] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:18.484534  3431 raft_consensus.cc:2272] T 25474d56fa3c452ea0f06683eaf900f1 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:18.488042  3431 tablet_replica.cc:333] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80: stopping tablet replica
I20260709 12:30:18.488765  3431 raft_consensus.cc:2243] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:18.489507  3431 raft_consensus.cc:2272] T 2105bdf1d73b43cfa43a714a15343c92 P 5e95a763e18e4814893fad06e1ea9d80 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:19.541051  3431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 15520)
I20260709 12:30:20.252409  3431 tablet_server.cc:196] TabletServer@127.3.89.195:0 shutdown complete.
I20260709 12:30:20.291041  3431 master.cc:562] Master@127.3.89.254:40851 shutting down...
I20260709 12:30:20.315506  3431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:20.316203  3431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:20.316569  3431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 56d6fdfd5599486f8a9770f7ce97bfa4: stopping tablet replica
I20260709 12:30:20.337425  3431 master.cc:584] Master@127.3.89.254:40851 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (24839 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (46983 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (425314 ms total)
[  PASSED  ] 8 tests.
I20260709 12:30:20.397570  3431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2338 messages since previous log ~37 seconds ago
I20260709 12:30:20.397864  3431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2338 messages since previous log ~37 seconds ago
I20260709 12:30:20.398042  3431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2334 messages since previous log ~37 seconds ago
I20260709 12:30:20.398320  3431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~55 seconds ago
I20260709 12:30:20.398542  3431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~55 seconds ago
I20260709 12:30:20.398722  3431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 7 messages since previous log ~211 seconds ago
I20260709 12:30:20.398903  3431 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 4 messages since previous log ~230 seconds ago
I20260709 12:30:20.399050  3431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 4 messages since previous log ~230 seconds ago
I20260709 12:30:20.399258  3431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 11 messages since previous log ~2 seconds ago
I20260709 12:30:20.399515  3431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~23 seconds ago
